1. Introduction — Machines That Predict the Next Word

A large language model is, at bottom, a function. It takes a sequence of words—or, more precisely, tokens—and outputs a probability distribution over what the next token should be. That is all. There is no explicit database of facts, no hand-coded grammar, no symbolic reasoning engine. There is a single, enormous mathematical function, learned from text, that assigns probabilities to continuations.

And yet this function can write code, translate languages, prove theorems, explain quantum mechanics, compose poetry, diagnose medical conditions, and pass the bar exam. The gap between the simplicity of the objective (predict the next word) and the sophistication of the behavior is the central surprise of the field. Understanding how this happens is the goal of this article.

The explanation has several layers. At the lowest level, there are neural networks—compositions of simple differentiable functions organized into layers. Above that, there is a learning algorithm: gradient descent with backpropagation, which adjusts millions or billions of parameters to minimize prediction error on a vast corpus of text. At the architectural level, there is the transformer—a specific neural network design based on the attention mechanism, which allows the model to dynamically determine which parts of the input are relevant to predicting each output. And surrounding all of this is a deeper question from information theory: what is it about human language that makes next-word prediction so powerful? The answer, as we will see, is that prediction is compression, and compression requires understanding.

This article is aimed at someone with a strong mathematical background—comfortable with linear algebra, calculus, and probability—who wants a precise understanding of what LLMs are, how they learn, and why they work. We will develop the mathematics from first principles, trace the historical path that led here, and then step back to consider what it all means.

2. Historical Context — From Perceptrons to Transformers

The transformer did not appear from nowhere. It is the product of seven decades of ideas, false starts, and breakthroughs. Understanding the history clarifies why each piece of the modern architecture exists.

3. Information Theory — Language as a Statistical Process

Before we build the machine, we need to understand what it is learning. The deepest answer comes from Claude Shannon’s information theory.

Shannon Entropy

In 1948, Shannon defined the entropy of a discrete random variable $X$ with possible values $\{x_1, \ldots, x_n\}$ and probability distribution $P$:

Language Has Structure, and Structure Means Redundancy

Shannon applied his theory to English text. If English were a sequence of uniformly random letters (26 letters plus space), the entropy per character would be $\log_2 27 \approx 4.75$ bits. But English is highly structured. After seeing “th,” the next letter is almost certainly “e” or “a” or “i.” After seeing “The president of the United,” the next word is almost certainly “States.” Shannon estimated the true entropy of English at about 1.0–1.5 bits per character—roughly one-third to one-quarter of the maximum. The rest is redundancy imposed by the statistical structure of the language.

Cross-Entropy: The Loss Function of Language Models

When we train a language model, we minimize the cross-entropy between the true distribution $P$ (the actual next token in the training data) and the model’s predicted distribution $Q$:

Cross-entropy has a beautiful decomposition: $H(P, Q) = H(P) + D_{\text{KL}}(P \| Q)$, where $D_{\text{KL}}$ is the Kullback-Leibler divergence—a measure of the “distance” between $P$ and $Q$ (always $\geq 0$, zero iff $P = Q$). Since the entropy $H(P)$ of the true distribution is fixed, minimizing cross-entropy is equivalent to minimizing KL divergence—making $Q$ as close to $P$ as possible.

Information Embedded in Text

Here is the deeper point. Every text ever written by a human being—every book, every article, every conversation, every line of code, every medical record, every legal brief—encodes information about the world. A physics textbook encodes the laws of physics. A novel encodes models of human psychology. A recipe encodes chemistry. Code encodes logic. Even casual conversation encodes social norms, common knowledge, and causal reasoning.

This information is not stored explicitly. It is stored implicitly, in the statistical relationships between words. “Water boils at 100” is followed by “degrees Celsius” because that is a fact about the world. “She picked up the heavy box and” is more likely followed by “strained” than “floated” because gravity exists and humans have limited strength. The conditional probabilities between words reflect the structure of reality as filtered through human expression.

Kolmogorov Complexity and the Limits of Compression

Kolmogorov complexity $K(x)$ is the length of the shortest program that produces string $x$. It is the theoretical limit of compression: you cannot describe $x$ in fewer bits than $K(x)$. Kolmogorov complexity is uncomputable (by a reduction to the halting problem), but it provides a theoretical frame: a perfect language model would assign probabilities reflecting the true Kolmogorov complexity of continuations.

Real language models approximate this. They discover patterns—syntactic, semantic, factual, logical—that allow them to compress text below the naive character-level entropy. The better the model, the lower the cross-entropy, the closer it approaches the true information content of the language. Modern LLMs achieve perplexities that correspond to roughly 0.7–1.0 bits per character on English text, close to Shannon’s estimated entropy rate.

4. Neural Networks from First Principles

A neural network is a parametric function composed of simple, differentiable building blocks. Every LLM is a neural network. Let us build the concept from scratch.

The Artificial Neuron

A single artificial neuron computes:

The linear part $\mathbf{w}^T\mathbf{x} + b$ computes a weighted sum of the inputs—geometrically, a hyperplane in input space. The activation function introduces nonlinearity, without which a composition of any number of layers would still be linear (a composition of linear functions is linear).

Activation Functions

Common choices:

• Sigmoid: $\sigma(z) = \frac{1}{1 + e^{-z}}$. Maps to $(0, 1)$. Historically popular but causes vanishing gradients for large $|z|$.
• Tanh: $\sigma(z) = \tanh(z)$. Maps to $(-1, 1)$. Zero-centered, better than sigmoid but still saturates.
• ReLU: $\sigma(z) = \max(0, z)$. The modern default. Cheap to compute, no saturation for positive inputs. Can “die” (output zero for all inputs if weights drift negative).
• GELU: $\sigma(z) = z \cdot \Phi(z)$, where $\Phi$ is the standard normal CDF. A smooth approximation to ReLU used in most transformers. Allows small negative values, which aids optimization.

Layers and Depth

A layer applies the neuron computation to every component simultaneously. For a layer with $m$ neurons and input dimension $n$:

$$\mathbf{h} = \sigma(W\mathbf{x} + \mathbf{b}), \qquad W \in \mathbb{R}^{m \times n}, \;\; \mathbf{b} \in \mathbb{R}^m$$

A deep neural network is a composition of layers:

$$f(\mathbf{x}) = f_L \circ f_{L-1} \circ \cdots \circ f_1(\mathbf{x})$$

where each $f_\ell(\mathbf{h}) = \sigma(W_\ell \mathbf{h} + \mathbf{b}_\ell)$. The layers between the input and output are called hidden layers. The word “deep” in deep learning means many layers—modern transformers have dozens to over a hundred.

The Universal Approximation Theorem

This theorem says neural networks are expressive enough—the function class is rich enough to represent any continuous mapping. But it says nothing about whether gradient descent can find a good approximation, or how many neurons are needed. In practice, depth (many layers) is far more parameter-efficient than width (many neurons in one layer) for most problems. Deep networks learn hierarchical representations—early layers learn simple features, later layers compose them into complex ones.

5. Learning: Gradient Descent and Backpropagation

A neural network with fixed weights is just a function. The magic is in learning—adjusting the weights so that the function does something useful. This is an optimization problem.

The Setup

We have a dataset $\{(\mathbf{x}_i, y_i)\}_{i=1}^N$ (for language models: $\mathbf{x}_i$ is a sequence of tokens, $y_i$ is the next token). We have a neural network $f(\mathbf{x}; \boldsymbol{\theta})$ with parameters $\boldsymbol{\theta} = \{W_1, \mathbf{b}_1, W_2, \mathbf{b}_2, \ldots\}$. We have a loss function $\mathcal{L}(\boldsymbol{\theta})$ that measures how badly the network’s predictions match the data. For language models, this is the cross-entropy loss:

$$\mathcal{L}(\boldsymbol{\theta}) = -\frac{1}{N}\sum_{i=1}^{N} \log f(y_i \mid \mathbf{x}_i; \boldsymbol{\theta})$$

The goal: find $\boldsymbol{\theta}^*$ that minimizes $\mathcal{L}$. The parameter space is enormous—billions of dimensions—and the loss surface is non-convex, full of saddle points, plateaus, and local minima. Exact optimization is intractable. We use gradient descent.

Gradient Descent

Intuition: Imagine standing on a hilly landscape in fog. You cannot see the lowest valley, but you can feel the slope under your feet. Gradient descent says: step downhill. Repeat. You may not find the global minimum, but you will find a low point.

In practice, computing the gradient over the entire dataset at each step is too expensive. Stochastic gradient descent (SGD) estimates the gradient from a random subset (a mini-batch) of the data:

$$\boldsymbol{\theta}_{t+1} = \boldsymbol{\theta}_t - \eta \nabla_{\boldsymbol{\theta}} \mathcal{L}_{\text{batch}}(\boldsymbol{\theta}_t)$$

The noise from mini-batch sampling is not a bug—it is a feature. It helps escape shallow local minima and saddle points, acting as a form of implicit regularization.

Backpropagation

The gradient $\nabla_{\boldsymbol{\theta}}\mathcal{L}$ involves derivatives of the loss with respect to every parameter in every layer. For a network with $L$ layers, this means computing $\frac{\partial \mathcal{L}}{\partial W_\ell}$ and $\frac{\partial \mathcal{L}}{\partial \mathbf{b}_\ell}$ for each $\ell = 1, \ldots, L$. Backpropagation computes all of these efficiently using the chain rule.

Backpropagation is just the chain rule applied systematically to a computational graph. It is not heuristic, not approximate—it computes the exact gradient. The practical miracle is that it scales: computing the gradient of a function with a billion parameters costs roughly twice the computation of evaluating the function itself.

Modern Optimizers

Vanilla SGD is rarely used in practice. Modern training uses Adam (Kingma and Ba, 2014), which maintains per-parameter estimates of the first moment (mean) and second moment (uncentered variance) of the gradient:

$$m_t = \beta_1 m_{t-1} + (1 - \beta_1) g_t, \qquad v_t = \beta_2 v_{t-1} + (1 - \beta_2) g_t^2$$ $$\theta_{t+1} = \theta_t - \eta \frac{\hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon}$$

where $g_t = \nabla_\theta \mathcal{L}$, $\hat{m}_t$ and $\hat{v}_t$ are bias-corrected estimates, and $\epsilon \sim 10^{-8}$ prevents division by zero. Adam adapts the learning rate for each parameter independently: parameters with consistently large gradients get smaller effective learning rates, and vice versa. This is crucial for training deep networks where different parameters can have gradients that differ by orders of magnitude.

6. From Words to Vectors — Embeddings and Representation

Neural networks operate on real-valued vectors. Language is a sequence of discrete symbols. The bridge between them is the embedding.

Tokenization

The first step is breaking text into tokens. Early models used whole words, but this creates a vocabulary of hundreds of thousands of entries, with many rare words poorly represented. Modern LLMs use subword tokenization (Byte-Pair Encoding or SentencePiece), which breaks text into frequently occurring subword units. For example:

“unhappiness” → [“un”, “happiness”] or [“un”, “happ”, “iness”]

A typical LLM vocabulary has 30,000–100,000 tokens. Every piece of text—English, code, Chinese, mathematical notation—is converted into a sequence of integer token IDs.

The Embedding Matrix

Each token $t$ in the vocabulary is assigned a learned vector $\mathbf{e}_t \in \mathbb{R}^d$, where $d$ is the embedding dimension (typically 768 to 12,288 in modern models). These vectors are stored as rows of an embedding matrix $E \in \mathbb{R}^{V \times d}$, where $V$ is the vocabulary size. Looking up a token’s embedding is a matrix indexing operation.

Initially, these embeddings are random. During training, they are adjusted by gradient descent along with all other parameters. After training, the geometry of the embedding space encodes semantic relationships.

The Geometry of Meaning

7. The Transformer Architecture

The transformer is the architecture that makes modern LLMs possible. Every major language model since 2018—GPT, BERT, Claude, Gemini, LLaMA—is a transformer or a close variant. The key innovation is the attention mechanism.

The Problem with Recurrence

Before transformers, sequence models were recurrent: they processed tokens one at a time, maintaining a hidden state that was updated at each step. This has two problems:

1. Sequential computation: Token $t$ cannot be processed until token $t-1$ is done. This prevents parallelism and makes training slow.
2. Long-range dependencies: Information from early tokens must be preserved through many sequential updates to influence later tokens. Despite LSTMs, this remains a bottleneck.

The transformer solves both problems. It processes all tokens simultaneously, and each token can attend directly to any other token, regardless of distance.

Self-Attention: The Core Mechanism

Given an input sequence of $n$ token embeddings $\mathbf{x}_1, \ldots, \mathbf{x}_n \in \mathbb{R}^d$, stacked into a matrix $X \in \mathbb{R}^{n \times d}$, self-attention computes three projections:

Then the attention output is:

Let us unpack this step by step.

Step 1: Compute attention scores. $QK^T$ is an $n \times n$ matrix where entry $(i, j)$ is the dot product $\mathbf{q}_i^T\mathbf{k}_j$—a measure of how much token $i$ should attend to token $j$. High dot product means high relevance.

Step 2: Scale. Divide by $\sqrt{d_k}$ to prevent the dot products from growing too large (which would push the softmax into regions of tiny gradients). If the entries of $Q$ and $K$ have variance $\sim 1$, then $\mathbf{q}^T\mathbf{k}$ has variance $\sim d_k$, and dividing by $\sqrt{d_k}$ restores unit variance.

Step 3: Softmax. Apply softmax row-wise to convert scores into a probability distribution: $\alpha_{ij} = \frac{e^{s_{ij}}}{\sum_k e^{s_{ik}}}$. Now each row sums to 1, and $\alpha_{ij}$ is the “attention weight”—the fraction of attention that token $i$ pays to token $j$.

Step 4: Weighted sum. The output for token $i$ is $\sum_j \alpha_{ij} \mathbf{v}_j$—a weighted combination of all value vectors, where the weights are determined by the attention pattern. Token $i$ “reads” from all positions, weighted by relevance.

Causal Masking

For language models that generate text left-to-right (autoregressive models like GPT and Claude), token $i$ must not attend to tokens $j > i$—you cannot look into the future. This is enforced by setting the upper-triangular entries of $QK^T$ to $-\infty$ before the softmax, so they become zero after exponentiation. This is the causal mask.

Multi-Head Attention

A single attention head computes one set of attention weights. But a token might need to attend to different things for different reasons—to a subject for agreement, to a verb for semantics, to a nearby adjective for modification. Multi-head attention runs $h$ independent attention heads in parallel:

Different heads learn to attend to different types of relationships. Empirically, some heads learn syntactic patterns (subject-verb agreement), some learn positional patterns (attend to the previous token), some learn semantic patterns (coreference resolution), and some learn patterns that resist easy human interpretation.

Positional Encoding

Attention is permutation-equivariant: if you shuffle the input tokens, the attention computation shuffles the output in the same way. The network has no built-in notion of position or order. Since word order is crucial in language (“dog bites man” vs. “man bites dog”), we must inject positional information.

The original transformer used fixed sinusoidal encodings:

$$PE_{(pos, 2i)} = \sin\!\left(\frac{pos}{10000^{2i/d}}\right), \qquad PE_{(pos, 2i+1)} = \cos\!\left(\frac{pos}{10000^{2i/d}}\right)$$

These are added to the token embeddings before the first layer. Modern models often use Rotary Position Embeddings (RoPE), which encode relative positions by rotating the query and key vectors in 2D subspaces. RoPE has better properties for extrapolation to sequences longer than those seen in training.

The Full Transformer Block

A single transformer block combines multi-head attention with a feedforward network, connected by residual connections and layer normalization:

Residual connections ($\mathbf{x} + f(\mathbf{x})$ instead of $f(\mathbf{x})$) are critical. They allow gradients to flow directly through the network without passing through nonlinearities at every layer, mitigating the vanishing gradient problem and enabling the training of very deep networks (100+ layers).

Layer normalization normalizes the activations to have zero mean and unit variance across the feature dimension, stabilizing training. Without normalization, the scale of activations can drift across layers, making optimization fragile.

The Full Language Model

A complete autoregressive transformer language model stacks $L$ transformer blocks:

Scale

The number of parameters in a transformer scales roughly as $12Ld^2$ (where $L$ is the number of layers and $d$ is the model dimension), dominated by the weight matrices in the attention and FFN sub-layers. To give a sense of scale:

Model	Parameters	Layers	Dimension $d$	Year
GPT-1	117M	12	768	2018
GPT-2	1.5B	48	1,600	2019
GPT-3	175B	96	12,288	2020
LLaMA 3 (largest)	405B	126	16,384	2024

GPT-3 has 175 billion parameters. Each parameter is a single floating-point number (typically stored in 16 bits during training). The model is a single, differentiable function—one enormous composition of matrix multiplications, softmaxes, and GELUs—with 175 billion adjustable knobs, all tuned simultaneously by gradient descent on trillions of tokens of text.

8. How It All Fits Together — The Training Pipeline

Let us now trace the complete pipeline from raw text to a working language model, showing how every piece we have discussed connects.

Stage 1: Pre-Training

The foundation. A transformer is initialized with random weights and trained on a massive corpus of text (trillions of tokens—books, web pages, code, scientific papers, conversations). The training objective is simple: for every position in every document, predict the next token.

The training loop, repeated billions of times:

1. Sample a mini-batch of text sequences from the corpus.
2. Forward pass: Feed each sequence through the transformer. At each position $i$, the model outputs a probability distribution $P(t_{i+1} \mid t_1, \ldots, t_i)$.
3. Compute loss: The cross-entropy between the predicted distribution and the actual next token, averaged over all positions and all sequences in the batch.
4. Backward pass: Backpropagate the loss to compute $\nabla_\theta \mathcal{L}$.
5. Update: Adjust all parameters with Adam: $\theta \leftarrow \theta - \eta \frac{\hat{m}}{\sqrt{\hat{v}} + \epsilon}$.

Pre-training GPT-3 required approximately $3.14 \times 10^{23}$ floating-point operations (314 zettaFLOPs), running on thousands of GPUs for weeks. The electricity cost alone was estimated in the millions of dollars.

Stage 2: Supervised Fine-Tuning (SFT)

The pre-trained model is a powerful text predictor, but it is not yet a useful assistant. It will happily continue toxic text, generate misinformation, or produce aimless rambling—because those patterns exist in the training data. Fine-tuning on curated (prompt, response) pairs teaches the model to follow instructions and produce helpful, structured answers.

Stage 3: Reinforcement Learning from Human Feedback (RLHF)

Human evaluators rank model outputs from best to worst. A reward model is trained to predict these rankings. Then the language model is further optimized (via Proximal Policy Optimization or similar RL algorithms) to produce outputs that score highly under the reward model. This aligns the model’s behavior with human preferences—helpfulness, harmlessness, honesty—without requiring explicit rules for every situation.

Scaling Laws

One of the most remarkable empirical findings in deep learning is that LLM performance follows predictable scaling laws. Kaplan et al. (2020) showed that the cross-entropy loss decreases as a power law in three variables:

These are power laws—straight lines on a log-log plot. They hold over many orders of magnitude. This means performance is predictable: given a compute budget, you can forecast the loss before training. It also means there are diminishing returns: each 10x increase in compute buys roughly the same absolute improvement in loss. But the improvements compound: a small decrease in cross-entropy translates to a qualitative leap in capability (the difference between GPT-2 and GPT-3, or GPT-3 and GPT-4, is a fraction of a nat in loss but an enormous gap in behavior).

The Chinchilla scaling laws (Hoffmann et al., 2022) refined this: for a given compute budget, the optimal strategy is to scale parameters and data roughly equally. Many earlier models (including GPT-3) were under-trained—they had too many parameters for the amount of data they saw.

9. What Does the Model Learn?

The training objective is “predict the next token.” But what does the model actually learn in order to do this? The answer is richer than you might expect.

Representations, Not Rules

The model is never given explicit rules of grammar, logic, or factual knowledge. Instead, it develops internal representations—patterns of activation across its neurons—that capture these regularities. Probing studies have found that transformer hidden states encode:

• Syntax: Part-of-speech tags, dependency trees, and constituent structures emerge in intermediate layers.
• Semantics: Word senses, entity types, and semantic roles are linearly decodable from hidden states.
• World knowledge: Factual associations (capitals, dates, scientific constants) are stored in the feedforward layers, which function as key-value memories.
• Reasoning patterns: Models develop internal circuits for tasks like modular arithmetic, indirect object identification, and multi-step inference.

Features and Superposition

Mechanistic interpretability research (notably from Anthropic) has revealed that models represent more features than they have dimensions. If the model dimension is $d = 4096$, there may be millions of distinct “features” (concepts, patterns, or linguistic facts) represented in that space. This is possible through superposition—features are represented as nearly orthogonal directions in a high-dimensional space. In $d$ dimensions, you can pack exponentially many nearly orthogonal vectors (this is related to the Johnson-Lindenstrauss lemma). The model tolerates small amounts of interference between features because most features are rarely active simultaneously.

The Knowledge Gradient

Circuits and Algorithms

Some of what the model learns can be described as algorithms. For instance, a transformer trained on simple arithmetic develops an internal algorithm for addition that involves specific attention heads routing digits to specific positions. In-context learning (the ability to learn new tasks from examples in the prompt) appears to involve a form of internal gradient descent—the model simulates the learning process within its forward pass, using attention heads as implicit optimizers.

The model does not “know” these algorithms in the way a programmer knows them. They emerge from optimization. The model found that implementing these computational patterns reduces its prediction error, so gradient descent converged on them. The intelligence is in the data and the optimization pressure, not in the architecture.

10. The Economics and Infrastructure

The technology does not exist in a vacuum. The rise of LLMs is inseparable from the economics of compute.

The GPU Revolution

Neural network training is dominated by dense matrix multiplication—exactly the operation GPUs were designed for. NVIDIA recognized this early and invested in CUDA (2006), making their GPUs programmable for general computation. Today, NVIDIA controls roughly 80–90% of the AI training chip market. A single NVIDIA H100 GPU delivers approximately 990 teraFLOPS of FP16 computation and costs ~$30,000. Training a frontier model requires clusters of 10,000–100,000 GPUs.

The Cost of Intelligence

Model	Training Compute (FLOP)	Estimated Cost	Year
GPT-3	$3.1 \times 10^{23}$	~$5M	2020
Chinchilla	$5.8 \times 10^{23}$	~$10M	2022
GPT-4 (estimated)	~$10^{25}$	~$100M	2023
Frontier models (2025)	~$10^{26}$	$500M–$1B+	2025

Training costs are growing roughly 4x per year. This creates a steep barrier to entry and concentrates frontier AI development in a handful of organizations: OpenAI, Anthropic, Google DeepMind, Meta, and a few others. The capital requirements are comparable to semiconductor fabrication plants.

The Data

The training data for modern LLMs includes a substantial fraction of the publicly available internet: web pages (Common Crawl), books, Wikipedia, scientific papers, GitHub code, forums, social media. GPT-3 was trained on ~300 billion tokens; current frontier models use 10–15 trillion tokens or more. There are legitimate concerns about data sourcing, copyright, and the eventual exhaustion of high-quality text data.

Inference Economics

Training happens once; inference (running the model to generate responses) happens millions of times per day. The cost of generating one token is small ($\sim$0.001–0.01 cents for frontier models), but at scale this becomes the dominant cost. Major optimizations include:

• Quantization: Reducing parameter precision from 16-bit to 8-bit or 4-bit integers, cutting memory and compute by 2–4x with minimal quality loss.
• KV-cache: Caching the key and value matrices from previous tokens to avoid redundant computation during autoregressive generation.
• Speculative decoding: Using a small, fast model to generate candidate tokens, then verifying them in parallel with the large model.
• Distillation: Training a smaller model to mimic the outputs of a larger model, transferring capability at reduced cost.

The Economic Transformation

The AI industry is on a trajectory to become one of the largest sectors of the global economy. Cloud providers are spending $50–80 billion per year on AI infrastructure. NVIDIA’s market capitalization exceeded $3 trillion. The downstream economic impact—through productivity gains in software development, customer service, content creation, scientific research, education, and healthcare—is estimated in the trillions of dollars per year within this decade. Whether this materializes depends on the technology continuing to improve and on society’s ability to integrate it productively.

11. The Information-Theoretic View

Let us now step back and view the entire enterprise through the lens of information theory. This perspective unifies everything we have discussed.

An LLM Is a Compressor

A language model that assigns probability $Q(x_t \mid x_{<t})$ to the next token can be converted into a lossless compressor via arithmetic coding. The expected code length per token is the cross-entropy $H(P, Q)$. A perfect model ($Q = P$) achieves the entropy rate $H(P)$—the theoretically optimal compression. A language model’s cross-entropy on a held-out test set directly measures how well it compresses language.

This is not a metaphor. It is a mathematical identity. Predicting and compressing are the same operation viewed from different angles (Shannon’s source coding theorem, 1948). Every improvement in language model quality is, provably, an improvement in compression efficiency.

Compression Requires Understanding

Mutual Information and What the Model Captures

The mutual information between two random variables $X$ and $Y$ is:

$$I(X; Y) = H(X) - H(X \mid Y) = H(Y) - H(Y \mid X)$$

It measures how much knowing $Y$ reduces uncertainty about $X$. In the language model setting, $X$ is the next token and $Y$ is the preceding context. The model learns to capture the mutual information $I(x_t; x_{<t})$—the information that the past carries about the future. High mutual information means the context is highly informative about the continuation. The model’s hidden states are a learned sufficient statistic—a compressed representation of the context that preserves (approximately) all the information relevant to predicting the future.

The Information Bottleneck

The model’s finite-dimensional hidden state ($d = 4096$ to $16384$ dimensions) must represent arbitrarily long contexts. This is an information bottleneck: the model must compress the context, discarding irrelevant details and preserving what matters for prediction. Attention is the mechanism for this selective compression—it allows the model to dynamically determine which parts of the context are relevant at each step. The attention weights are, in information-theoretic terms, an adaptive channel capacity allocation.

The Rate-Distortion Connection

Rate-distortion theory (Shannon, 1959) characterizes the fundamental tradeoff between compression rate and reconstruction quality. A language model faces an analogous tradeoff: its finite capacity (parameters, hidden dimension) limits how accurately it can model the true distribution. Scaling laws can be understood in this framework: more parameters provide a higher “rate” (more bits of model capacity), allowing lower “distortion” (lower cross-entropy). The power-law scaling suggests a smooth, continuous tradeoff—there is no sharp threshold at which the model suddenly “understands” language; it gradually refines its approximation of the true distribution.

12. Philosophical Questions

The capabilities of LLMs raise questions that have historically belonged to philosophy. They do not have definitive answers, but they are worth stating precisely.

Understanding vs. Pattern Matching

An LLM has never seen, touched, or experienced anything. It has only seen text—sequences of tokens. When it writes about the taste of coffee or the feeling of grief, it is drawing on statistical patterns in how humans describe these experiences. Is this understanding?

The philosophical tradition offers a sharp test case. John Searle’s Chinese Room argument (1980): imagine a person in a room who follows a rulebook to respond to Chinese inputs with appropriate Chinese outputs, without understanding Chinese. Searle argues the room does not understand Chinese, even though its input-output behavior is indistinguishable from a Chinese speaker. By analogy, an LLM does not understand language—it merely simulates understanding.

The counterargument: the system (the room plus the rulebook plus the person) might understand Chinese, even if no single component does. And the analogy may be misleading: the LLM’s internal representations capture genuine structure (semantic similarity, logical relationships, factual knowledge) in a way that a simple lookup table does not. The line between “genuine understanding” and “sufficiently sophisticated pattern matching” may not be as clear as Searle assumed.

The Grounding Problem

Language refers to the world. The word “cat” refers to actual cats. LLMs learn relationships between words, but do they learn the relationship between words and things? This is the symbol grounding problem (Harnad, 1990). Some argue LLMs are “stochastic parrots”—manipulating symbols without connecting them to referents. Others argue that if the statistical structure of language faithfully reflects the structure of the world (which it must, because language evolved to describe the world), then learning the former implicitly captures the latter. The debate is unresolved.

Consciousness and Experience

Does an LLM have subjective experience? Almost certainly not in its current form, but we cannot be certain, because we have no theory of consciousness that would let us decide. The “hard problem of consciousness” (Chalmers, 1995)—why physical processes give rise to subjective experience at all—remains unsolved. Without a solution, we cannot definitively say whether any particular physical system is conscious. An LLM is a series of matrix multiplications. A brain is a network of electrochemical signals. Why one has experience and the other does not (if indeed that is the case) is not explained by any current theory.

Epistemological Implications

If a machine can produce text that is indistinguishable from expert-written text, what does this mean for how we assess knowledge? LLMs demonstrate that surface-level linguistic competence can be achieved without grounded experience. This challenges the assumption that competent communication implies deep understanding. It also raises the question: how much of what humans call “understanding” is, in fact, sophisticated pattern matching over our own training data (experience)?

The Alignment Problem

A system that is intelligent enough to be useful is intelligent enough to be dangerous if its goals do not align with ours. The alignment problem is the challenge of ensuring that AI systems do what we intend, even as they become more capable. RLHF is a partial solution, but it is empirical and brittle—it does not provide formal guarantees. As models become more capable, the alignment problem becomes more urgent and more difficult. It is, arguably, the most important open problem in AI.

13. What LLMs Cannot Do (and Why)

It is as important to understand the limitations as the capabilities.

Hallucination

LLMs sometimes generate confident, fluent text that is factually wrong. This is not a bug in the usual sense—it is a structural consequence of the training objective. The model learns to produce plausible continuations, not true ones. If a false claim would be plausible in context (because similar-sounding true claims exist in the training data), the model may produce it. The model has no mechanism for checking its outputs against a ground truth—it has no ground truth, only learned distributions.

Reasoning Limitations

Transformers are constant-depth computation: each token passes through a fixed number of layers (say, 96), regardless of the difficulty of the problem. A problem that requires $n$ sequential reasoning steps may exceed the model’s depth if $n$ is large enough. Chain-of-thought prompting partially addresses this by allowing the model to use its own generated tokens as intermediate scratch work, effectively converting depth into length. But the fundamental constraint remains: the model has no external working memory and no ability to run iterative algorithms for an arbitrary number of steps.

Lack of Persistent Learning

An LLM’s weights are fixed after training. It cannot learn new facts from a conversation (though it can use information provided in the context window). Every interaction starts from the same parameter state. This is unlike human cognition, where every experience modifies the brain. Fine-tuning and retrieval-augmented generation (RAG) partially address this, but the model itself does not learn from use.

No Formal Verification

The model’s outputs are samples from a learned probability distribution. There is no formal guarantee of correctness for any particular output. This is fundamentally different from a theorem prover or a compiler, which can verify its own work. For applications where correctness is critical (medicine, law, engineering), LLM outputs must be verified by other means.

14. The View from Above

Let us now assemble the complete picture.

Humanity has been producing text for thousands of years—in scrolls, books, letters, newspapers, and, in the last few decades, on the internet at unprecedented scale. This text encodes, implicitly, an enormous amount of knowledge about the world: physics, chemistry, biology, medicine, law, history, psychology, mathematics, common sense, social norms, and the structure of language itself. The encoding is noisy, redundant, contradictory, and incomplete. But the information is there, distributed across trillions of tokens.

A transformer is a mathematical function—a specific composition of matrix multiplications, softmax operations, and nonlinearities—with billions of adjustable parameters. It takes a sequence of tokens as input and outputs a probability distribution over the next token. Its architecture is designed to process sequences efficiently, using attention to dynamically route information between positions.

Training is the process of adjusting the parameters to minimize prediction error on the text corpus. The algorithm is gradient descent with backpropagation: compute the derivative of the loss with respect to every parameter using the chain rule, then take a small step downhill. Repeat this billions of times. The model’s parameters converge to a compressed representation of the statistical structure of human text.

The result is a function that, when given a sequence of tokens, produces a probability distribution over continuations that is remarkably close to what a knowledgeable, articulate human might write. It can do this because predicting text well—truly well, across all domains and contexts—requires capturing the structure of the world as expressed through language.

What remains unclear is deeper. Does the model understand, or does it merely simulate understanding? Is the statistical structure of language sufficient to capture the causal structure of the world, or are there aspects of reality that text alone cannot convey? Can these systems be made reliably safe and aligned with human values? How far does scaling take us—do we eventually hit a wall, or does the current paradigm lead to artificial general intelligence?

These questions are not yet answered. But the technology is real, the mathematics is precise, and the implications are immense. An intelligent math major can understand exactly how LLMs work—the architecture is not mysterious, the training is not magical, and the information theory is beautiful. What remains genuinely mysterious is why the gap between “predict the next word” and “exhibit intelligent behavior” turned out to be so small.