Meta releases LLaMA for research

Meta announced LLaMA on 24 February, a family of language models for research. It spans 7 billion, 13 billion, 33 billion and 65 billion parameters. Access is being granted case by case under a noncommercial licence to academic researchers, government and civil-society organisations, and industry research laboratories.

Sulat illustration. Meta symbol belongs to Meta. No endorsement.

The release gives researchers more options for studying and adapting foundation models. Meta says smaller models can make that work accessible with less infrastructure.

A base model with explicit limits

The original model card says LLaMA was trained between December 2022 and February 2023. It uses a Transformer architecture to predict subsequent text. Training sources include web crawls, GitHub, Wikipedia, books, arXiv and Stack Exchange.

LLaMA has not been trained with human feedback. Meta cautions that downstream applications need further risk evaluation and mitigation: outputs can be inaccurate, offensive or unhelpful. Most training text is English despite the inclusion of 20 languages, so the team expects stronger performance in English.

Code and model access are separate

The release repository provides example inference code. Its instructions tell researchers to apply for model files, then use download links supplied after approval. Having the code does not itself grant access to the weights.