AI Reference 055 Large Language Models Wikipedia guided Primary sources linked

Temperature and Sampling

Definition

Temperature changes the probability distribution used during sampling. Lower values concentrate probability more strongly; higher values make less likely alternatives easier to sample.

What is sampling?

An LLM produces a probability distribution over possible next tokens. Sampling is the process of choosing from that distribution rather than always selecting only the single most probable token.

What temperature changes

Temperature rescales token probabilities before sampling. Lower temperature makes the distribution sharper, so high-probability tokens dominate. Higher temperature flattens the distribution, making lower-probability alternatives easier to choose.

What temperature does not do

Temperature affects variability, not factual verification. Setting temperature to zero or near zero can make output more repeatable, but a model can still repeat the same incorrect claim consistently. Grounding and checking sources solve a different problem.

Related terms, defined

Large Language Models AI Hallucinations A hallucination is generated content that is false, unsupported or inconsistent with the evidence available to the system. Language generation optimizes likely sequences, not guaranteed factual truth, so high-stakes factual work needs grounding and verification. Large Language Models Retrieval-Augmented Generation Retrieval-augmented generation combines a generative model with retrieval from an external collection such as documents, databases or search results. Retrieved material is inserted into the model's context so the answer can be grounded in information outside the model's parameters. Large Language Models Function Calling and Tool Use Function calling and tool use let a model request structured actions such as querying a database, calling an API or using search, with the surrounding application executing the action. Large Language Models AI Agents An AI agent is a system in which a model can select or plan actions over multiple steps, usually with tools, memory and feedback from an environment. The language model is only one part of the agent; orchestration, permissions, tools and stopping conditions are equally important. Large Language Models Mixture-of-Experts Models A mixture-of-experts model contains multiple expert subnetworks and routes each token or input through only a subset of them. This can increase total parameter capacity without activating every parameter on every forward pass.

Reference guide and primary sources

Wikipedia is used here as a terminology and history reference guide. Current model versions, institutional statistics and product-specific claims are also linked to first-party or institutional sources because those details can change faster than encyclopedia articles.

← Inference AI Hallucinations →