Academy

MLP is the model's memory: where knowledge lives

In a language model, attention layers route information, but the actual knowledge is stored elsewhere — in the MLP layers that make up the bulk of the model.

The Felesh team 4 min

When we think about language models, attention usually takes the spotlight. But if we ask where a model’s knowledge — that the capital of France is Paris, or the syntax of a programming language — is actually stored, the answer lies elsewhere: in the MLP layers.

Two different roles

A transformer block contains two primary components: attention and the MLP (feed-forward network). They perform distinct jobs. Attention routes information: it determines which parts of the text relate to one another and moves context between positions. It does not hold facts itself. The MLP holds the knowledge: it is where facts and patterns are stored.

A standard metaphor applies here: attention is the librarian who knows where to look, while the MLP represents the books themselves. To answer “what is the capital of France”, attention brings the context of “France” and “capital” together, and the MLP retrieves the answer “Paris”.

Attention routes, the MLP answersThe question “What is the capital of France?” goes into a transformer block. Attention links the tokens France and capital and passes that context on, but stores no facts. The MLP holds the knowledge: it retrieves the stored fact capital of France: Paris, and the answer is Paris.“What is the capital of France?”attentionroutes contextstores no factsFrancecapitallinked in attentionMLPholds the knowledgeFrance → Parisstored in weightsanswer: Paris
Attention links “France” and “capital”, then the MLP, which holds the fact, returns Paris.

A key-value kind of memory

Mechanically, the MLP operates as a key-value memory store. The first part of the MLP acts as a set of keys that recognise specific patterns, while the second provides the values retrieved in response. When a specific pattern activates in the input — for instance, a semantic marker for “capital of a country” — the matching key triggers, and the associated value (the stored knowledge) is added to the flow.

A key matches, and its value joins the flowThree keys, each a pattern detector: capital of a country, programming syntax, and past tense of a verb. Each is paired with a stored value: Paris, for x in xs:, and went. The input carries the capital-of-a-country pattern, so that key fires, the wire carries the match to Paris, and Paris is added to the flow.keys: patterns in the inputvalues: stored knowledgecapital of a countryprogramming syntaxpast tense of a verbParisfor x in xs:wentthe flow after this layer:+ Paris added
A key fires when its pattern appears in the input, and its stored value is added to the flow.

Why the MLP is so large

This architecture explains why the MLP accounts for the vast majority of a model’s parameters. In a typical transformer, the MLP layers hold the bulk of the weights, with attention taking a smaller fraction and the remainder allocated to embeddings. The reasoning is structural: storing millions of independent facts — names, dates, relations, and language syntax — requires immense capacity. Routing context is relatively light; storing knowledge is heavy.

Where the weights sitOne bar shows the weight shares, drawn illustratively and not from a measured model: the MLP is the widest segment, attention is smaller, and embeddings take the rest. Below it, a table: attention routes context and stores no facts; the MLP stores knowledge such as names, dates and syntax; embeddings turn tokens into vectors.shares drawn illustratively, not a measured modelMLP: the bulkattentionembeddingspartwhat it doeswhat it storesattentionroutes contextno factsMLPstores knowledgenames, dates, syntaxembeddingstokens to vectorsthe rest of the weights
Most of a block's weights sit in its MLP layers; attention is a smaller share, and the embeddings take the rest.

Why this matters

This separation of concerns has direct practical consequences. When fine-tuning a model to ingest new facts, the MLP layers must be targeted alongside attention — otherwise, the new knowledge has nowhere to settle. Similarly, when compressing or pruning an architecture, altering the MLP directly degrades the model’s memory.

A new fact needs the MLP to settleOne fine-tuning example: the capital of Australia is Canberra. Tuned on attention alone, the MLP stays frozen and the fact has nowhere to settle; asked again, the model gives its old answer. Tuned on attention and the MLP, the fact settles into the MLP, and asked again, the model answers Canberra.new fact: the capital of Australia is Canberratune attention onlytune attention and MLPattentionattentionMLPfrozenMLPAustralia: Canberranowhere to settleasked again: Canberra
Fine-tuning that changes only attention leaves the new fact nowhere to settle; the MLP has to change too.

Putting it together

The core dynamic is straightforward: attention says “look at this”, and the MLP says “here is what I know about it”. A model’s knowledge is distributed across millions of MLP weights, not bound within the attention mechanism. Recognising this separation clarifies both how to approach fine-tuning and why models are so large — most of that parameter count is simply static memory.