A language model, at heart, just guesses the next word. Here's how that simple guess becomes an agent once you add tools, memory, and planning — and where you actually need one.
The Felesh team6 min
A simple next-word predictor
At the core of every large language model sits a deceptively simple mechanism: given the text so far, predict what comes next. The model reads text token by token, predicts a single token, appends it to the sequence, and starts over. That is it. It functions as a highly scaled autocomplete.
This simple job carries a surprising consequence. To correctly complete the sentence “If we heat water to 100 degrees…”, the model must implicitly know that water boils. Because it has processed millions of similar sentences across multiple domains, the objective of predicting the next word forces it to absorb broad knowledge about the world.
The model scores every candidate for the next token, appends the winner, and reads the longer text again.
How it is trained
Training happens in three stages. First, pre-training: the model processes an enormous volume of text — books, web pages, encyclopaedias, and code — and, without direct supervision, absorbs language structures, general knowledge, and patterns of reasoning. Second, supervised tuning: using thousands of high-quality question-and-answer pairs, the model learns to respond to instructions rather than aimlessly continue the text. Third, learning from human feedback: human reviewers rank various outputs, teaching the model what constitutes a useful and aligned answer.
At large scales, abilities often emerge that were never explicitly taught — such as learning from a few examples in context, or reasoning step by step. This phenomenon is known as emergence.
Each stage teaches what the one before could not: language, answering a question, and which answer people prefer.
Six things a language model alone doesn’t do well
Despite its capabilities, a bare language model has six structural limits. One, hallucination: it sometimes confidently states plausible but false information — like a student making a firm guess instead of admitting ignorance. Two, a knowledge cutoff: the model’s training data ends at a fixed date, functioning like an encyclopaedia printed in a specific year. Three, an inability to act: the model only produces text; it sends no email and saves no file — a brain with no hands. Four, poor arithmetic precision: it approximates numbers rather than strictly computing them. Five, no long-term memory: every conversation initiates from a blank slate. Six, prompt sensitivity: a minor variation in phrasing can drastically alter the output.
A bare model fails in six structural ways, from an invented source to a different answer when the question is reworded.
From model to agent
This is where the concept of an agent becomes relevant. An agent situates the language model inside a broader system. The architecture follows a straightforward formula: an agent equals a language model, plus tools, memory, and planning. A model alone acts as an advisor generating text; an agent acts as an operator that takes a goal, formulates steps, executes them, and corrects course when an action fails.
An agent takes a goal, plans steps, runs them with tools, and when a step fails it changes the plan, not the goal.
However, not every problem requires a full agent. A practical rule of thumb applies: a large share of tasks can be solved with a well-crafted prompt and basic information retrieval. Another portion requires a deterministic chain of tools. Only a small fraction genuinely requires a fully autonomous agent capable of complex planning. Add complexity only when the task demands it.
Most tasks stop at the first rung; step down to a chain of tools or a full agent only when the task forces you.
