AI Reference 049 Large Language Models Wikipedia guided Primary sources linked

RLHF and Preference Training

Definition

Reinforcement learning from human feedback uses preference information from people to train or guide models toward preferred behavior; modern post-training also uses related preference-optimization techniques.

What is RLHF?

Reinforcement learning from human feedback, or RLHF, uses human preference judgments to shape model behavior after pretraining. People compare candidate outputs, and those preferences are used to train a reward model or another preference-learning objective.

How the pipeline works

A common historical pipeline is supervised instruction tuning, collection of ranked model responses, training of a reward model, and reinforcement-learning optimization against that reward. Newer preference-optimization methods can learn from comparison data without the exact same RL loop.

Why it is used

Pretraining teaches statistical language prediction; it does not directly teach that an assistant should follow instructions, refuse certain requests or present information in a helpful format. Preference training moves behavior toward those product goals, although it cannot eliminate hallucinations or guarantee correctness.

Related terms, defined

Large Language Models Tokens in AI Tokens are the discrete units processed by a language model. Depending on the tokenizer, a token can represent a word, part of a word, punctuation, whitespace patterns or another text fragment. Large Language Models Tokenization Tokenization converts text into token identifiers a model can process. Model families use different tokenizers, so the same text can consume different numbers of tokens in different systems. Large Language Models Context Windows A context window is the amount of tokenized information a model can consider in one working sequence or request. Larger context increases capacity for long inputs but does not guarantee perfect retrieval or reasoning over every detail. Large Language Models Embeddings in Modern AI Embeddings are dense numerical vectors used to represent text, images or other objects in spaces where distance or similarity can be computed. Large Language Models Inference Inference is the phase in which a trained model is run to make predictions or generate outputs. LLM inference usually produces tokens sequentially.

Reference guide and primary sources

Wikipedia is used here as a terminology and history reference guide. Current model versions, institutional statistics and product-specific claims are also linked to first-party or institutional sources because those details can change faster than encyclopedia articles.

← Fine-Tuning Language Models Tokens in AI →