AI Reference 046 Large Language Models Wikipedia guided Primary sources linked

What Is a Large Language Model?

Definition

A large language model is a statistical model trained on large collections of text or multimodal data to predict and generate token sequences. Modern LLMs are usually transformer-based and are adapted to conversation, coding, tool use and other tasks through post-training and application-level systems.

What is an LLM?

A large language model, or LLM, is a machine-learning model trained on very large text or multimodal datasets to model sequences of tokens. Most modern LLMs use decoder-style transformer architectures and generate output one token at a time.

What does the model learn?

During pretraining, the model repeatedly predicts tokens and adjusts billions of numerical parameters. Those parameters capture statistical regularities about language, code and other data seen during training. They do not function as a conventional database in which every fact is stored as a readable record.

Why LLMs appear general-purpose

The same next-token model can be prompted to summarize, translate, write code, answer questions or use tools because all of those tasks can be represented as sequences. Post-training makes the model easier to instruct, but external retrieval and software are still needed when answers must use current or authoritative data.

Related terms, defined

Large Language Models LLM Pretraining LLM pretraining exposes a model to large datasets and optimizes a predictive objective, commonly next-token prediction, before later post-training or adaptation. Large Language Models Fine-Tuning Language Models Fine-tuning continues training a pretrained model on a narrower dataset or objective so it performs better for a particular domain, task or behavior. Large Language Models RLHF and Preference Training Reinforcement learning from human feedback uses preference information from people to train or guide models toward preferred behavior; modern post-training also uses related preference-optimization techniques. Large Language Models Tokens in AI Tokens are the discrete units processed by a language model. Depending on the tokenizer, a token can represent a word, part of a word, punctuation, whitespace patterns or another text fragment. Large Language Models Tokenization Tokenization converts text into token identifiers a model can process. Model families use different tokenizers, so the same text can consume different numbers of tokens in different systems.

Reference guide and primary sources

Wikipedia is used here as a terminology and history reference guide. Current model versions, institutional statistics and product-specific claims are also linked to first-party or institutional sources because those details can change faster than encyclopedia articles.

← Multimodal AI LLM Pretraining →