The question comes up in almost every discovery call, usually framed as a budget decision. It is easier to answer as a question about what you are trying to change.
One rule that covers most cases
Knowledge belongs in context. Behaviour belongs in weights. If the model needs facts it could not have known, retrieve them. If it knows enough but responds in the wrong shape, tone or format, fine-tuning is the cheaper fix.
Teams reach for fine-tuning to teach a model their documentation and are surprised when it invents details. Training on documents teaches style and vocabulary far more reliably than it teaches facts.
What retrieval costs you
Retrieval adds an index to keep fresh, a search quality problem to own and latency on every request. In exchange, updating knowledge means updating a document, and every answer can point at its source.
What fine-tuning costs you
Fine-tuning needs examples, and good examples are expensive to produce. A few hundred carefully written pairs beat ten thousand scraped ones, and the work of writing them usually falls on the people who have the least free time.
You also inherit a versioning problem. Every retrained model needs the same evaluation run, and rolling back is slower than editing a prompt.
Where they meet
Mature systems often use both. Retrieval supplies the facts, a small fine-tuned model enforces the output format and house style, and a larger model handles the rare cases the small one refuses.
Start with retrieval and prompting. Fine-tune once you have production traffic showing a pattern the prompt cannot hold, and once you have the evaluation set to prove the new model is actually better.
