large language model

Purpose and description

Let an agent explain large language models and how they work in general terms, relay architectures, training and evaluation from research sources with attribution, describe capabilities, limitations and governance debates neutrally, and distinguish large language models from earlier language models, search engines and general artificial intelligence.

A type of language model built as a neural network with billions or more parameters, usually a transformer, trained on very large text corpora to predict tokens and then adapted through instruction tuning and feedback to follow prompts, in families such as generative pre-trained transformers and models from several developers including GPT, Claude, Gemini, Llama and GLM, with architectures including mixture of experts and extensions such as large action models that operate tools; large language models generate and analyse text, code and, in multimodal versions, images and audio, and raise questions of accuracy, bias, safety and governance.

What it is for: Generating and processing language for many tasks.

It can be explain how they work; relay architectures and training; describe capabilities and limits; distinguish related systems.

Distinguishing features

Transformer architecture

Pretraining on large corpora

Instruction tuning

Emergent general capabilities

What it looks like

Not a visible object; software accessed through interfaces and APIs.

Physical character

parameters: billions to trillions count

transformer introduced: 2017 year - Attention Is All You Need

context windows: thousands to millions tokens - varies by model

How it is recognised

Neural language model with very many parameters

GPT-style models, Claude, Gemini, Llama, GLM, mixture of experts models, instruction-tuned models, large action models

Small language models are smaller; search engines retrieve rather than generate; general AI is a broader aspiration

Related models

is a kind of - in registry terms

is built on - architecture

is trained by - among other methods

is contrasted with - which retrieves documents

In practice

Families and kinds

generative pre-trained transformers and other decoder models

instruction and chat models

mixture of experts models

multimodal models

large action models and agents that use tools

open-weight and proprietary models from various developers

Standards and regulation

EU AI Act and other emerging AI regulations

Model documentation practices such as model cards

Copyright and data protection law applied to training data, which is contested

Failure modes and hazards

Hallucinated or inaccurate output

Bias and harmful content

Overreliance without verification

Presenting any developer s claims as settled fact

Also called

Where this came from

wikidata · CC0 1.0