AI Reference 045 Neural Networks & Deep Learning Wikipedia guided Primary sources linked

Multimodal AI

Definition

Multimodal AI processes or generates multiple data types such as text, images, audio or video in one coordinated system.

What is multimodal AI?

Multimodal AI processes or generates more than one type of data, such as text, images, audio and video. A multimodal model may accept several modalities as input, produce several as output, or map between them.

How modalities are combined

Systems can use separate encoders for different data types or convert modalities into representations that a shared model can process. Vision-language models, for example, represent image content numerically and connect it with language representations.

Why multimodality matters

Real-world tasks are rarely text-only. Documents contain charts, robots use cameras, assistants hear speech and designers work with images. Multimodal models allow AI systems to operate on richer evidence, although each modality introduces its own error modes.

Related terms, defined

Neural Networks & Deep Learning Artificial Neural Networks Artificial neural networks are parameterized models built from layers of connected computational units. Training adjusts the parameters to reduce an objective function. Neural Networks & Deep Learning Backpropagation Backpropagation efficiently computes gradients through differentiable neural networks by applying the chain rule from outputs back toward earlier layers. Neural Networks & Deep Learning Convolutional Neural Networks Artificial neural networks are parameterized models built from layers of connected computational units. Training adjusts the parameters to reduce an objective function. Neural Networks & Deep Learning Recurrent Neural Networks Artificial neural networks are parameterized models built from layers of connected computational units. Training adjusts the parameters to reduce an objective function. Neural Networks & Deep Learning Long Short-Term Memory Networks LSTM networks are recurrent neural networks with gated memory mechanisms designed to preserve and control information across longer sequences.

Reference guide and primary sources

Wikipedia is used here as a terminology and history reference guide. Current model versions, institutional statistics and product-specific claims are also linked to first-party or institutional sources because those details can change faster than encyclopedia articles.

← Foundation Models What Is a Large Language Model? →