Academy

Fine-tune, RAG, or prompt: which one, and what each costs

There are three ways to adapt a model to your need, and the wrong choice can get expensive. The difference is in what problem each one actually solves.

The Felesh team 8 min

When a language model needs to adapt to a specific use case, you have three routes: prompt engineering, context retrieval (RAG), or fine-tuning. These are frequently confused, but each solves a distinct problem. The wrong choice wastes compute and yields poor results. Here is how to draw the line.

Three places to change one model callOne model call about parcel PX-3380 holds three parts. The instructions say: two sentences, status first; editing them is prompting. The documents say: fees, 5 days free, then 40 a day; filling them is RAG. The question is: PX-3380 is held, what do I owe. All three go into the model, whose weights are what fine-tuning edits. The reply: held at customs, 7 days, you owe 80.one call to the modelinstructionsprompt edits thisTwo sentences, status first.documentsRAG fills thisfees: 5 days free, then 40 a dayquestionPX-3380 is held. What do I owe?the modelfine-tuning editsthese weightsreply: Held at customs, 7 days. You owe 80.
One model call has three places to change it: the instructions, the documents beside the question, and the weights.

Prompt: the cheapest starting point

Prompt engineering means guiding the model with instructions alone, leaving the underlying weights untouched. This is the cheapest and fastest route. No training is required, and iterations take minutes. For many tasks, clear instructions are enough. Start here; only reach for heavier methods when prompting genuinely falls short.

RAG: when the model needs new knowledge

If the model lacks access to specific information — private documents, or data generated after its training run — the answer is usually RAG. This involves retrieving relevant text and placing it in the model’s context window at inference time. The advantage is that knowledge can be updated continuously without retraining the base model. RAG is the answer to “the model doesn’t know something”, not “the model doesn’t know how to do something.”

Retrieval hands the model the line it lacksThe question is: parcel PX-3380 is held, day 7, what do I owe. The file fees.md holds four lines; the one that matters, held parcels 5 days free then 40 a day, is picked out and travels into the model context, where a dashed slot said the fee rule was missing. Before, the model alone answered that it does not have the fee table; now it answers 2 days over at 40 a day, you owe 80. Edit fees.md tomorrow and the next answer changes, with no retraining.“Parcel PX-3380 is held, day 7. What do I owe?”fees.mdpickup: within 30 daysheld: 5 days free,then 40 a dayreturns: within 14 dayscustoms: broker feecontext windowQ: PX-3380, day 7the fee rule is missingheld: 5 days free,then 40 a dayfrom fees.mdmodel alone: “I don’t have the fee table.”answer: 2 days over, 40 a day: you owe 80edit fees.md and the next answer changes
RAG finds the one line the model lacks and sets it beside the question, so the answer is right today and changes the day the file does.

Fine-tuning: when the model has to change its behaviour

Fine-tuning applies elsewhere: when the problem isn’t missing knowledge, but behaviour. If you need the model to adopt a specific tone, consistently output a strict format, or internalise patterns that are hard to articulate in a system prompt, fine-tuning is the right route. The cost is inherently higher, requiring datasets, compute, and maintenance. Fine-tuning answers “the model doesn’t do something the way I want”, not “the model doesn’t know something.”

Training pairs change the weights, and the format sticksThree training pairs show an input and the output wanted: PX-3380 held, day 7 gives state=held next=pay_fee; PX-1204 out for delivery gives state=moving next=wait; PX-9051 delivered Monday gives state=done next=none. They train the model, so its weights change. Before tuning, a new input, PX-7712 held, day 9, gets friendly prose that varies each time. After tuning it gets state=held next=pay_fee, the same shape every time. The behaviour now lives in the weights, not in a longer prompt.training pairs: an input, then the output you wantPX-3380 held, day 7state=held next=pay_feePX-1204 out for deliverystate=moving next=waitPX-9051 delivered Monstate=done next=nonethe modelweightsnew input: PX-7712 held, day 9Hi! Your parcel looks like it’sheld at customs. Storage is freefor a while, then a fee applies…before tuning: prose that variesstate=held next=pay_feeafter tuning: same shape every timethe behaviour now lives in the weights, not in a longer prompt
Fine-tuning shows the model examples of the behaviour until the format becomes its default: the same new input, now in the same shape every time.

Fine-tuning is itself a ladder

Fine-tuning is not monolithic; it spans a spectrum from light to heavy. The lightest step involves parameter-efficient methods (PEFT) — most notably LoRA and its lower-memory variant, QLoRA. These techniques freeze the original model weights and train only a few small, inserted layers. Compute and memory costs drop sharply, training accelerates, and you can run multiple specialised versions on top of a single base model.

At the far end of the ladder is full fine-tuning, where every weight in the network is updated. It offers maximum flexibility but remains the most demanding option. It requires heavy compute, larger datasets, and strict maintenance, while introducing the risk of “catastrophic forgetting” — overwriting the model’s foundational reasoning abilities.

LoRA against full fine-tuningTwo columns. LoRA and QLoRA: the base weights stay frozen and only a small adapter is trained; one base can carry a fleet of adapters, such as tone-v3 and format-v1; cheap to train, more to track. Full fine-tuning: every weight is updated, with heavy compute and more data, and the risk of catastrophic forgetting, where the base reasoning is overwritten; the most compute, data and upkeep. Start at the lightest rung and go to full only if LoRA misses the target quality.LoRA, QLoRAbase frozen, adapter trainedadapterbase weights: frozentrained: only the adapterone base, a fleet of adapterstone-v3format-v1cheap to train, more to trackfull fine-tuningevery weight updatedall weights changeheavy compute, more datacatastrophic forgetting:reasoning overwrittenmost compute, data and upkeepstart light; go full only if LoRA falls short
LoRA freezes the base and trains a small adapter; full fine-tuning rewrites every weight, which costs more and can overwrite what the model already knew.

The core logic remains consistent: if you require fine-tuning, start at the lightest rung. Only consider full fine-tuning when methods like LoRA fail to reach your target quality.

A simple rule for choosing

The decision reduces to a few clear questions. Does a better instruction solve the problem? Prompt. Does the model need information it lacks? RAG. Does the model need to durably change its behaviour or style? Fine-tuning. Often, the right architecture is composite: a fine-tuned model retrieving new knowledge through RAG, guided by a precise prompt.

From the symptom to the routeA table of three symptoms, each with the question that decides it and the route. Six paragraphs for a status check: would a better instruction fix it? Prompt. Quotes last year’s storage fee: does the model lack information? RAG. Right facts but a different format every time: must its behaviour change for good? Fine-tune. Below, the often-right answer is all three together: a fine-tuned model, retrieval and a precise prompt.customer seesaskrouteSix paragraphs fora status checkWould a betterinstruction fix it?promptQuotes last year’sstorage feeDoes the model lackinformation?RAGFacts right, formatvaries every timeMust its behaviourchange for good?fine-tuneoften the answer is all threefine-tuned model+retrieval (RAG)+precise prompt
Each symptom ends in a question that picks the route, and the answer is often all three together.

The cost of each

These routes form an ascending cost ladder. Prompting is the cheapest. RAG sits in the middle, incurring the operational overhead of a retrieval infrastructure. Fine-tuning is the most expensive. Even within fine-tuning, LoRA sits at the lower end and full fine-tuning at the top. Both demand curated data, training cycles, and lifecycle maintenance, but full fine-tuning is far heavier. A common engineering mistake is fine-tuning a model to solve a problem that RAG or a better prompt could have addressed. Before paying the structural cost, verify the requirement.

Where each route hides a cost

The “start cheap, escalate only when needed” heuristic is sound, but every tier introduces friction, and honesty requires naming it. Prompting is cheap to write but brittle at scale. A prompt that works today drifts as models or inputs shift, and long instruction stacks quietly consume context windows and compute budgets. RAG provides updatable knowledge but demands a secondary system to build and operate — chunking, indexing, embedding, and adding latency to every call. It also fails in its own way: retrieve the wrong passage, and the model answers confidently from flawed context. LoRA, the recommended fine-tuning rung, is cheap to train but not free to run. You must manage adapter versions, route requests correctly, and accept a quality ceiling below full fine-tuning for profound behavioural shifts. Trading a single general model for a fleet of specialised ones means more assets to evaluate, version, and keep in sync. None of this argues against the ladder; it simply means climbing it with open eyes.

Where each route hides its costA table of three routes. Prompt: cheap to write; it drifts as the model or inputs change, and long instructions eat context and budget. RAG: updatable knowledge; a retrieval system to build and run, with chunking, indexing and latency on every call, and a failure of its own: a wrong passage in gives a confident wrong answer out. LoRA: cheap to train; adapter versions to track and route, and a quality ceiling below full fine-tuning. None of this argues against the ladder.promptcheap to writedrifts as the model or inputs changelong instructions eat context and budgetRAGfresh knowledgea retrieval system to build and runchunking, indexing, latency on every callwrong passage, confident errorLoRAcheap to trainadapter versions to track and routea quality ceiling below full fine-tuningnone of this argues against the ladder: climb it with open eyes
Every rung hides a cost somewhere; the sharpest is retrieval handing the model the wrong passage, which it answers from with full confidence.

Putting it together

These three routes are not rivals; they are discrete tools for distinct problems. Prompting guides immediate behaviour, RAG provides external knowledge, and fine-tuning internalises durable behaviour. The key to building reliable systems is isolating the exact problem you face — and deploying the cheapest tool that solves it.