Academy

The augmented LLM

A raw prompt in a playground creates an illusion. In production the working unit is an augmented model — wrapped with memory, tools, and context.

The Felesh team 3 min

You test a prompt in a web playground. The model parses your request, applies the logic perfectly, and returns a flawless response. You paste that exact string into a production application, and the illusion breaks. The model hallucinates an outdated product catalogue. It forgets the user’s previous message. It outputs conversational text instead of the strict schema your database expects.

Three production failures, each with its missing pieceRow one: the model hallucinates an old catalogue; the call carried only the prompt, and the live catalogue was not retrieved first. Row two: it forgets the user’s last message; the call carried only this message, not the pruned history of earlier turns. Row three: it replies in chat text, not the schema; the call carried no output contract and no schema the parser enforces.what brokewhat the call carriedwhat it neededhallucinates anold cataloguethe prompt, alonelive cataloguefetched firstforgets the user’slast messageonly this messagepruned historyof old turnsreplies in chat,not the schemano output contracta schema theparser checks
Each production failure has one cause: the call did not carry the piece that would have prevented it.

The web interface masked the reality of the underlying infrastructure. A base model endpoint is entirely stateless, isolated, and blind. It has no memory of the last turn, no connection to your live data stores, and no tools to verify facts.

A bare prompt is not a system. At this stage of deployment, the working unit is the augmented LLM: the model paired with external retrieval, memory management, and function calling. You take a raw inference engine and wrap it in deterministic scaffolding. You supply state by injecting carefully pruned conversation history. You supply grounding by retrieving relevant internal data immediately before the call. You supply constraints by enforcing rigid output structures.

A call assembled from three inputsThree inputs join one model call in turn: the last three turns, pruned, ending with the customer asking to move the delivery to Saturday; the record PRC-40912, out for delivery; and a schema the reply must meet, with intent, parcel_id and date. The reply comes back matching it: intent reschedule, parcel PRC-40912, date 2026-10-17.inputs, assembled firstone call, one checked replystatelast 3 turns, prunedmove it to Saturday?groundingPRC-40912: en routeschemaintent, parcel_id, datethe modelone request, one replyreply, schema-validintent: rescheduleparcel_id: PRC-40912date: 2026-10-17
The call is assembled from three pieces before the model sees it: state, grounding and a schema for the reply.

There is no autonomy at this rung. The system operates strictly within a request–response cycle governed by your application code. You are still driving the orchestration. The majority of the engineering value here does not come from attempting autonomous loops, but from disciplined context engineering — systematically managing exactly what information occupies the model’s finite context window at the moment of evaluation.

The window holds a selection, not the historyA window with room for a few segments: rules, turns 1 to 37, turns 38 to 40, and a new turn. The app takes turns 1 to 37 out of the window and keeps them stored outside it, then retrieves the record PRC-40912, out for delivery, into the same place. The window ends with rules, the record, turns 38 to 40 and the new turn.the window the model reads at the callrulesturns 1–37older historyPRC-40912 statusout for deliveryturns 38–40new turnkept by the app, outside the windowturns 1–37, prunedPRC-40912: out for delivery
The app swaps old turns out of the window and retrieves the one record the question needs, so the window holds a selection.

This stage establishes the baseline for everything that follows. Before handing over control flow, you must first reliably extract structured, predictable work from a stateless endpoint.