Tulu 3

Tulu 3 (styled Tülu 3) is a family of instruction-following language models and, more importantly, a fully open post-training recipe released by the Allen Institute for AI (Ai2) in November 2024.¹ Rather than pretraining new base models, the project applied a documented multi-stage pipeline to Llama 3.1 base checkpoints at 8B and 70B parameters, with a 405B variant following in January 2025.² Alongside the model weights, Ai2 released the training datasets, data-curation tooling, training code, and an evaluation framework, making Tulu 3 one of the most complete public accounts of how a competitive post-trained model is built.

Post-training recipe

The Tulu 3 pipeline has three main stages. First, supervised instruction tuning on a curated prompt mixture combining public datasets with synthetic, persona-driven data targeting skills such as mathematics, coding, and precise instruction following. Second, preference optimization using a length-normalized variant of direct preference optimization on on-policy and off-policy preference data, a lighter-weight alternative to classic RLHF. Third, and most influentially, the paper introduced Reinforcement learning with verifiable rewards (RLVR): reinforcement learning in which the reward is a simple binary signal from an automatic checker, applied to domains where correctness can be verified, such as math word problems and constraint-based instruction following.¹ RLVR later became a standard ingredient in reasoning-focused post-training across the field.

Models and evaluation

Developer-reported evaluations showed Tulu 3 8B and 70B matching or exceeding the corresponding Llama 3.1 Instruct models and competing with open-weight instruct models such as Qwen2.5, while trailing leading closed systems on some axes.¹ The January 2025 Tulu 3 405B release was reported by Ai2 to surpass DeepSeek-V3 and GPT-4o on its benchmark suite, a claim based on Ai2's own evaluation setup.²³

Significance

Tulu 3 is frequently cited as a reference point for reproducible post-training research: its datasets and code have been reused by other groups, and its recipe informed the post-training of Ai2's fully open OLMo 2 models. The RLVR formulation it named is now common vocabulary in discussions of reasoning-model training.

References