AI Reference 039 Neural Networks & Deep Learning Wikipedia guided Primary sources linked

Deep Learning

Definition

Deep learning uses neural networks with multiple layers of learned representations. Its modern rise combined larger datasets, better training methods and parallel hardware.

What is deep learning?

Deep learning is machine learning based on neural networks with multiple layers of learned representation. 'Deep' refers to the number of successive transformations between the input and output, not to human-like depth of understanding.

What layers learn

Early layers may detect simple local structure, while later layers can combine those patterns into more task-specific representations. In language models, layers transform token embeddings using attention and feed-forward networks; in vision models, layers transform image patches or feature maps.

Why deep learning became dominant

Large datasets, parallel hardware, better optimization methods and architectural innovations made it possible to train much larger networks than before. Deep learning now underlies many leading systems in vision, speech, language, recommendation and multimodal AI.

Related terms, defined

Neural Networks & Deep Learning Attention Mechanisms Attention computes context-dependent weighted combinations of representations, allowing a model to emphasize information relevant to the current prediction. Neural Networks & Deep Learning The Transformer Architecture The Transformer was introduced in the 2017 paper “Attention Is All You Need.” It replaced recurrent sequence processing with attention-based layers, enabled highly parallel training and became the architectural basis of most modern large language models. Neural Networks & Deep Learning Encoder-Decoder Models Encoder-decoder architectures turn an input into internal representations and then generate an output sequence or structure conditioned on those representations. Neural Networks & Deep Learning Self-Supervised Learning Supervised learning trains a model from examples paired with known labels or target values. Classification and regression are common supervised tasks. Neural Networks & Deep Learning Foundation Models Foundation models are trained on broad data at scale and then adapted or prompted for many downstream tasks.

Reference guide and primary sources

Wikipedia is used here as a terminology and history reference guide. Current model versions, institutional statistics and product-specific claims are also linked to first-party or institutional sources because those details can change faster than encyclopedia articles.

← GPUs and the Rise of Deep Learning Attention Mechanisms →