Category: NVIDIA models

This category contains 3 pages.

• Megatron-LM · NVIDIA's 2019 research project and open-source framework for training multi-billion-parameter Transformer language models with model parallelism.
• Megatron-Turing NLG · A 530-billion-parameter dense language model trained jointly by Microsoft and NVIDIA, announced in October 2021 as the largest dense Transformer of its time.
• Nemotron · NVIDIA's family of open-weight language models, best known for the June 2024 Nemotron-4 340B release aimed at synthetic data generation.