AI Reference 035 Neural Networks & Deep Learning Wikipedia guided Primary sources linked

Long Short-Term Memory Networks

Definition

LSTM networks are recurrent neural networks with gated memory mechanisms designed to preserve and control information across longer sequences.

What is an LSTM?

Long short-term memory, or LSTM, is a recurrent neural-network architecture introduced by Sepp Hochreiter and Jürgen Schmidhuber in 1997. It adds a memory cell and gates that control what information is written, retained and exposed.

Why the gates matter

A standard RNN repeatedly transforms its hidden state, which can cause useful gradients to vanish over long sequences. The LSTM's gated memory path was designed to preserve information and make learning long-range dependencies easier.

Where LSTMs were used

LSTMs became widely used in speech recognition, machine translation, handwriting recognition and time-series modeling. They were later displaced by transformers in many large-scale language tasks, but they remain important historically and are still useful for some sequential problems.

Related terms, defined

Neural Networks & Deep Learning Word Embeddings Word embeddings represent words or tokens as dense numerical vectors whose geometry captures statistical relationships learned from data. Neural Networks & Deep Learning ImageNet and AlexNet AlexNet's strong result in the 2012 ImageNet competition helped demonstrate the effectiveness of deep convolutional neural networks trained on GPUs and large image datasets. Neural Networks & Deep Learning GPUs and the Rise of Deep Learning GPUs perform many numerical operations in parallel, which fits the matrix and tensor workloads used in neural-network training and inference. Neural Networks & Deep Learning Deep Learning Deep learning uses neural networks with multiple layers of learned representations. Its modern rise combined larger datasets, better training methods and parallel hardware. Neural Networks & Deep Learning Attention Mechanisms Attention computes context-dependent weighted combinations of representations, allowing a model to emphasize information relevant to the current prediction.

Reference guide and primary sources

Wikipedia is used here as a terminology and history reference guide. Current model versions, institutional statistics and product-specific claims are also linked to first-party or institutional sources because those details can change faster than encyclopedia articles.

← Recurrent Neural Networks Word Embeddings →