AI Reference 042 Neural Networks & Deep Learning Wikipedia guided Primary sources linked

Encoder-Decoder Models

Definition

Encoder-decoder architectures turn an input into internal representations and then generate an output sequence or structure conditioned on those representations.

What is an encoder-decoder model?

An encoder-decoder model separates representation of the input from generation of the output. The encoder converts an input sequence into internal representations; the decoder generates an output conditioned on those representations.

Where the design became common

Sequence-to-sequence models became important in machine translation, summarization and speech tasks. Early versions used recurrent neural networks; the original Transformer was also an encoder-decoder model.

How this differs from decoder-only LLMs

Many modern chat LLMs use decoder-only transformers and generate text autoregressively from one combined context. Encoder-decoder models remain useful when the task has a clear source-to-target structure, such as translating or transforming one sequence into another.

Related terms, defined

Neural Networks & Deep Learning Self-Supervised Learning Supervised learning trains a model from examples paired with known labels or target values. Classification and regression are common supervised tasks. Neural Networks & Deep Learning Foundation Models Foundation models are trained on broad data at scale and then adapted or prompted for many downstream tasks. Neural Networks & Deep Learning Multimodal AI Multimodal AI processes or generates multiple data types such as text, images, audio or video in one coordinated system. Neural Networks & Deep Learning Artificial Neural Networks Artificial neural networks are parameterized models built from layers of connected computational units. Training adjusts the parameters to reduce an objective function. Neural Networks & Deep Learning Backpropagation Backpropagation efficiently computes gradients through differentiable neural networks by applying the chain rule from outputs back toward earlier layers.

Reference guide and primary sources

Wikipedia is used here as a terminology and history reference guide. Current model versions, institutional statistics and product-specific claims are also linked to first-party or institutional sources because those details can change faster than encyclopedia articles.

← The Transformer Architecture Self-Supervised Learning →