Category: Post-training

This category contains 8 pages.

• Constitutional AI · Anthropic's post-training method: models critique and revise their own outputs against written principles, then train on AI-generated preferences.
• Direct preference optimization · Preference-alignment method that optimizes a policy directly on comparison data via a closed-form objective, skipping the reward model and RL loop.
• Instruction tuning · Supervised fine-tuning of pretrained language models on instruction-response pairs, teaching them to follow natural-language tasks zero-shot.
• Knowledge distillation · Training a smaller student model to imitate a larger teacher; the mechanism behind small frontier-adjacent models and a persistent IP controversy.
• Reinforcement learning from human feedback · Post-training method that optimizes a language model against a reward model trained on human preference comparisons; the technique behind ChatGPT-style assistants.
• Reinforcement learning with verifiable rewards · Post-training with programmatic reward signals (test suites, answer checkers) instead of learned reward models; the engine of the reasoning-model era.
• Reward model · A model trained to score another model's outputs against human or AI preferences, supplying the training signal for RLHF and, in process form, for reasoning search.
• Supervised fine-tuning · Post-training method that teaches a pretrained language model desired behavior by supervised next-token training on curated prompt-response demonstrations.