Category: 2021 model releases

This category contains 9 pages.

• CLIP · OpenAI's January 2021 contrastive vision-language model that learns image classification from natural-language supervision and became a standard component in text-to-image systems.
• Codex (2021 model) · OpenAI's 2021 code-generation model, a GPT-3 descendant fine-tuned on public code; the original engine of GitHub Copilot and source of the HumanEval benchmark.
• DALL-E · OpenAI's January 2021 text-to-image model, a 12-billion-parameter autoregressive Transformer that generated images from natural-language prompts.
• Gopher · DeepMind's December 2021 280-billion-parameter dense language model, the baseline that motivated the Chinchilla compute-optimal scaling analysis.
• GPT-J · EleutherAI's June 2021 open-weight 6-billion-parameter autoregressive language model, an early open alternative to GPT-3-class systems.
• GPT-Neo · EleutherAI's March 2021 family of open-source autoregressive language models (up to 2.7 billion parameters), an early public replication of the GPT-3 design.
• Jurassic (models) · AI21 Labs' family of large autoregressive language models, spanning Jurassic-1 (2021) and Jurassic-2 (2023), later succeeded by Jamba.
• LaMDA · Google's 2021 dialogue-specialized language model family, a predecessor of Bard and the Gemini line, and the subject of a widely covered 2022 sentience controversy.
• Megatron-Turing NLG · A 530-billion-parameter dense language model trained jointly by Microsoft and NVIDIA, announced in October 2021 as the largest dense Transformer of its time.