AI Reference 053 Large Language Models Wikipedia guided Primary sources linked

Embeddings in Modern AI

Definition

Embeddings are dense numerical vectors used to represent text, images or other objects in spaces where distance or similarity can be computed.

What is an embedding?

An embedding is a dense numerical vector that represents an object such as a piece of text, an image or a user profile. Similar objects are trained or designed to occupy nearby regions of the vector space.

How semantic search uses embeddings

A document can be split into passages and converted to embeddings. A user's query is embedded with the same model, and the system finds passages whose vectors are close to the query vector. Those passages can then be returned directly or supplied to an LLM.

Embeddings inside and outside LLMs

Transformers use token embeddings internally, but application developers also use standalone embedding models for retrieval, clustering, deduplication and recommendation. The vectors are useful because distance can approximate semantic similarity.

Related terms, defined

Large Language Models Inference Inference is the phase in which a trained model is run to make predictions or generate outputs. LLM inference usually produces tokens sequentially. Large Language Models Temperature and Sampling Temperature changes the probability distribution used during sampling. Lower values concentrate probability more strongly; higher values make less likely alternatives easier to sample. Large Language Models AI Hallucinations A hallucination is generated content that is false, unsupported or inconsistent with the evidence available to the system. Language generation optimizes likely sequences, not guaranteed factual truth, so high-stakes factual work needs grounding and verification. Large Language Models Retrieval-Augmented Generation Retrieval-augmented generation combines a generative model with retrieval from an external collection such as documents, databases or search results. Retrieved material is inserted into the model's context so the answer can be grounded in information outside the model's parameters. Large Language Models Function Calling and Tool Use Function calling and tool use let a model request structured actions such as querying a database, calling an API or using search, with the surrounding application executing the action.

Reference guide and primary sources

Wikipedia is used here as a terminology and history reference guide. Current model versions, institutional statistics and product-specific claims are also linked to first-party or institutional sources because those details can change faster than encyclopedia articles.

← Context Windows Inference →