Academy

Off-the-shelf, or adapt your own?

The demo worked. Your production domain did not. Before you commit to training your own model, understand the adaptation ladder and why most teams stop far lower than they think they need to.

The Felesh team 6 min

The model nails the demo. You feed it a clean prompt, and the output is plausible. It formats the response correctly. Then you deploy it against your actual production domain—your internal acronyms, your legacy document structures, the edge cases unique to your operation—and the illusion breaks. Latency climbs as you stuff the context window with routing rules. The strict schema degrades into conversational text when the prompt gets too complex.

The natural reflex for engineering teams is immediate: train a custom model. In traditional software, if an upstream library fails your specific constraints, you fork it or write your own. Applied to large language models, that reflex is usually wrong. Sometimes, however, it is exactly right. The difficulty is distinguishing between the two states before committing months of engineering time.

The adaptation ladder

You do not jump straight to training. You climb an adaptation ladder. Each rung buys tighter fit and more control over the model’s behaviour. Each rung also exacts a heavier toll in operational burden, compute cost, and infrastructure complexity.

1. The off-the-shelf frontier model You call an external API. The model is general-purpose. Maintenance is zero; latency and cost are strictly transactional.

2. Prompt and context engineering You shape the input. You provide few-shot examples or rewrite instructions to bound the model’s behaviour. It is cheap to iterate, but you pay for those instructional tokens on every single inference pass.

3. Retrieval-augmented generation (RAG) You ground the model in your own data at runtime. You maintain a vector store and a retrieval pipeline. It solves knowledge gaps without altering the underlying model weights. You trade model-training complexity for data-engineering complexity.

4. Parameter-efficient fine-tuning (PEFT/LoRA/QLoRA) You freeze the base model and adjust a small fraction of the weights via an adapter. The model learns your specific format, tone, or domain syntax. It requires dedicated compute and clean training data, but the resulting adapter is lightweight and can often be trained on a single GPU.

5. Full fine-tuning You update all the weights in a pre-trained model. It deeply embeds domain knowledge and reasoning patterns. The compute cost is high, the dataset requirements are stringent, and the risk of catastrophic forgetting—where the model loses its baseline competence—is real.

6. Custom pre-training You train from scratch on your own tokens. Control is absolute. You own the architecture. The cost is measured in millions of dollars and dedicated cluster time.

What each rung costs youSix rungs, one row each, with what each costs up front and what it keeps costing. The off-the-shelf model is metered per call with zero upkeep. Prompt and context are paid on every call. RAG needs a vector store to build and a pipeline to run. PEFT and LoRA need data and one GPU, then an adapter to version. Full fine-tuning needs compute and strict data and carries the risk of forgetting. Custom pre-training costs millions and cluster time, and you own the serving. Most enterprise tasks stop at the third row.rungyou pay inongoingoff-the-shelfper call, meteredzero upkeepprompt + contextinstruction tokenspaid every callRAGvector store to builda pipeline to runPEFT / LoRAdataset, one GPUadapter to versionfull fine-tuningcompute, strict datarisk of forgettingcustom pre-trainingmillions, cluster timeyou own servingmost enterprise tasks stop at the third row
Each rung buys a tighter fit and charges a heavier toll, and most enterprise work is finished by the third.

Climb only until it fits

You climb this ladder only until the task fits. You stop at the lowest possible rung.

Climbing the ladder until the answer is rightA depot note reads: CRX hold on lot 14. The first rung, an off-the-shelf model, reads CRX as the Honda CRX car, which is wrong. The second rung adds examples and reads it as cross-dock hold, but those examples are sent on every call. The third rung looks up the glossary, finds the CRX line and cites it. That answers the note, so the climb stops at rung three; rungs four to six add cost without fit.the ladderthe depot noteoff-the-shelf modelprompt and examplesRAG: glossary lookupPEFT adapterfull fine-tuningcustom pre-training“CRX hold on lot 14”the model reads it asCRX = Honda CRX, a carwrong: the note is about lot 14rung 2 adds examplesCRX = cross-dock holdexamples go out on every callrung 3 looks up the glossaryCRX = cross-dock holdcites: glossary, line 3stop here: rung 3 answers itrungs 4 to 6 add cost, not fitrung 1 read it as a car
Climb the ladder one rung at a time and keep the first rung that answers the depot note correctly.

Most enterprise use cases are resolved safely at the third rung. A frontier model combined with rigorous prompt engineering and a reliable RAG pipeline handles the vast majority of reasoning and summarisation tasks. RAG teaches the model what to read; it brings the data to the model.

Adapting the model itself—moving to rung four or beyond—earns its keep only when the gap is durable and expensive to bridge any other way. Fine-tuning teaches the model how to act. You adapt the weights when RAG fails to capture a specific stylistic cadence, when you need strict adherence to a complex output schema without burning thousands of tokens on instructions, or when latency requirements dictate using a smaller, specialised model rather than a large generalist.

Where adaptation loses

Adapting a model is not just a training cost; it is a maintenance commitment.

A fine-tuned or custom model begins going stale the moment training finishes. While a team spends three months curating a dataset and training a specialised 8-billion-parameter model, the baseline capabilities of off-the-shelf frontier models will often leap ahead. You risk shipping a specialised asset that performs worse than the newest general API.

A three-month fine-tune against the newest APIAn illustrative chart. A fine-tune of an 8B model fills its bar over three months. The newest API line starts just under the flat line of your adapter, climbs, and passes the adapter early in the training. At the ship marker, after three months, the API is well above the adapter, which stays fixed at the level it shipped with.while you train, the API keeps movingthe API passes itships hereyour 8B adapterfixed at shipnewest APIstartship, after 3 monthsfine-tuning the 8B model: three monthsillustrative, not measuredcapability
A model trained for three months ships behind the newest API, which keeps climbing while the training runs.

Once you own the weights, you own the serving infrastructure. You must build and maintain pipelines for versioning, evaluation, and deployment. You manage GPU provisioning and handle scaling for your own inference endpoints. For a small team, a low-sensitivity task, or an early architectural experiment, an off-the-shelf frontier call is the correct choice.

Cost profiles shift dramatically as you climb. Published community findings suggest that while API calls scale linearly with usage, fine-tuning introduces steep upfront compute costs and ongoing dedicated hosting fees. Industry estimates indicate that an open-weight model might eventually lower inference costs, but the break-even point requires sustained, high-volume throughput to justify the dedicated infrastructure.

Monthly cost against call volumeAn illustrative chart of monthly cost against call volume. The API line rises with every call from the origin. The self-hosted line starts higher, because of GPU hosting and upkeep, and rises slowly. The two lines cross at the break-even point. Below that volume the API is cheaper; at sustained high volume your own adapter on your own GPU is cheaper.monthly cost against call volumeillustrative shape, not measuredbreak-evenAPI: per callyour GPU and upkeepfew callssustained volume
API cost grows with every call, while your own adapter starts high and only pays off past a break-even volume.

Before choosing a rung, profile exactly what your task needs: Profile the task, don’t rank the models.