A VP of Engineering pings me: the model needs fine-tuning. I ask the only question that matters. How do you know?
The room goes quiet. There is a demo that felt wrong, a board member who wants a defensible moat, and a team that absorbed the idea that fine-tuning is how serious AI companies operate. What there is not is evidence that the failure is a capability gap.
That gap between the request and the evidence is where quarters go to die.
The request arrives before the work that justifies it
I am not the only one seeing this. The practitioners who scope fine-tuning engagements for a living keep landing on the same conclusion. Most teams asking about fine-tuning should fix their prompts first, build a real RAG pipeline, and write evals, in that order. And the timing is the whole problem: the question ‘should we fine-tune?’ almost always arrives before the prerequisite work is done, and the honest answer is usually ‘not yet.’
The pattern repeats across engagements. Teams spend four weeks preparing data, blow their GPU budget on a training run, and then realize their ‘improved’ model hallucinates more than the base model did. One consultancy described watching teams burn $80K on fine-tuning projects that never made it to production, twice in a single year, with capable models and a broken diagnosis underneath.
The most honest line I have read on this: teams spend $100K fine-tuning a model on their internal docs, when what they really needed was a vector database and a decent prompt. The model already knew the facts. It just was not retrieving them.
That is the question most teams never asked. Is this a capability problem at all?
The diagnostic order that saves the quarter
From the Forward Deployed seat, I run a fixed sequence before anyone provisions a GPU. It takes an afternoon. It has saved clients entire data-labeling programs.
Step one is the eval. The single biggest mistake to avoid is fine-tuning before you have a written eval. If you cannot tell whether a checkpoint is better than the previous one, you do not have a fine-tuning problem, you have an evaluation problem. Fix that first. Most teams stall here, and the stall is the finding. If you cannot define better in a way a machine can score, you were going to train forever anyway.
Step two is retrieval. If the model does not know facts specific to your domain, your refund policies, your product docs, your case history, the answer is RAG. Fine-tuning does not reliably inject facts. The model can learn patterns, but it will not memorize specific policy numbers or case file details. The research backs the ordering. Ovadia et al. compared unsupervised fine-tuning with RAG on knowledge-intensive tasks, and RAG consistently came out ahead, both for facts the model had seen in pretraining and entirely new ones. Models struggle to learn new facts from fine-tuning unless they see many variations of the same fact.
Step three is the prompt. A strong prompt with a few examples, tested on a held-out set, closes a shocking share of what teams were ready to fine-tune for. If you never ran that test, you have no signal, only a hunch.
Step four is where fine-tuning finally earns its seat. Fine-tuning is for form, and you use it to shape behavior, style, structured output, and refusal patterns, not to inject knowledge that changes weekly. The cleanest framing I have seen puts it on a grid: is the failure knowledge-bound or behavior-bound, and is the underlying signal stable or volatile, and knowledge-bound plus volatile is the common case for enterprise apps, where docs, tickets, products, and prices change.
The right sequence is boring and it works: Prompt, then RAG, then Fine-tune, then Distill, with the highest-ROI fine-tuning being a thin LoRA adapter on a strong base model, paired with retrieval rather than replacing it.
Where the CODN actually sits
Leaders invoke the Cost of Doing Nothing to justify the training run. Flip it. The CODN of shipping a fine-tune you did not need is real and it compounds. You carry a model you cannot audit, a knowledge layer that goes stale the moment a policy changes, and an engineering team that now owns a retraining treadmill instead of a document update.
The genuine CODN is skipping the diagnosis. That is the cost that hides, because it shows up a quarter later as a model nobody trusts and a budget nobody can explain.
Fine-tuning is a precision instrument. In 2027 the teams that win will be the ones who earned the right to pick it up, having proven the prompt, the retrieval, and the eval could not carry the load first. Diagnose like the quarter depends on it. It does.