Most teams that want AI to work with their own information end up choosing between three approaches: prompting, retrieval-augmented generation, and fine-tuning. They are not competitors. They solve different problems, they cost different amounts of money and time, and they stack cleanly on top of each other. The practical skill is knowing which one answers which question, and in what order to try them. In most cases the answer is not one of the three. It is a specific sequence of all three.
What Each Approach Actually Means
Prompting is the simplest of the three and the one everyone starts with. You write instructions, include the information you need in the request, and send it to the model. Nothing about the model changes. The knowledge lives in the prompt, and so does the style. If you have a hundred pages of material, you cannot paste it all in, and if the material changes every week, pasting it in becomes a chore.
Retrieval-augmented generation, usually shortened to RAG, adds a search step. Instead of putting your documents into the prompt by hand, you store them in a system that finds the relevant passages at question time. The model reads the question, the system searches your documents, and the relevant excerpts go into the prompt alongside the question. The model's knowledge stays frozen, but its context stays current because the search happens fresh every time.
Fine-tuning takes a different route. You train the model on examples of the work you want it to do, and the lessons settle into the model's weights. The model itself changes. It learns your tone, your format, and your domain patterns without you having to explain them each time. You no longer need to include examples in the prompt because the examples are baked in.
What Each One Costs
Prompting costs tokens and attention. Every request carries the full context, and long contexts get expensive fast. A few thousand calls a month is trivial. A few million adds up, and the cost scales with how much text you send each time. The engineering cost is near zero.
RAG costs infrastructure and upkeep. You pay for embeddings, which turn text into numbers a search engine can compare. You pay for a vector database or a search index to store them. You pay for the orchestration that decides what to retrieve, and you pay to keep the index in sync when documents change. It is the cheapest option when your corpus is large and changes often, and the most expensive when your corpus is small and static.
Fine-tuning costs time and compute up front. You need a dataset of examples, usually hundreds or thousands of them, and you need to clean it. Training itself is often cheaper than people expect. A small adapter on an eight-billion-parameter model can train in a couple of hours on a single rented GPU for a few dollars. The real cost is the human time: building the dataset, running the training, evaluating the result, and repeating. Fine-tuning also locks in behavior, which means every change to your format is another training run.
When Prompting Is Enough
Prompting wins when the task is short, self-contained, and different every time. Rewriting a paragraph in a friendlier tone, summarizing a single meeting, classifying a support ticket, translating a sentence — none of these need your private data at all, and if they do, a few examples in the prompt usually cover it. This is called few-shot prompting, and it is surprisingly capable.
Prompting also wins when you are still exploring. You can test an idea in an afternoon. If it works, you can decide later whether to make it permanent with retrieval or training. Never fine-tune your way out of a problem that a better prompt would solve in ten minutes.
When Retrieval Is the Right Answer
Retrieval wins when the knowledge is large, detailed, or changes often. A 400-page employee handbook, a product catalog of ten thousand items, a support archive of old tickets — none of these fit in a prompt, and none of them should. RAG lets the model answer from the actual source instead of guessing, and it lets you cite the passage it used. That citation matters in regulated work, where "the model said so" is not a defense.
Retrieval also wins when correctness matters more than style. A customer asking about their specific plan needs the exact policy text, not a fluent paraphrase.
When Fine-Tuning Earns Its Keep
Fine-tuning wins when the task is stable, repeated, and stylistic. If you extract the same twelve fields from every invoice and you want them in the same JSON shape every time, a fine-tuned small model will do it faster and cheaper than a large model reading a long prompt full of rules. If your brand voice is specific and every output must match it, training teaches that voice once instead of describing it in every prompt.
Fine-tuning also helps when latency matters. A small fine-tuned model with no retrieval step answers in one hop, and on cheap hardware. That is why extraction and classification at volume are the classic fine-tuning use cases.
How the Three Combine
The strongest systems use all three. Prompting sets the behavior and handles the request. Retrieval supplies the facts. Fine-tuning shapes the output and carries the routine work. A support assistant might use retrieval to find the policy, a fine-tuned small model to draft the reply in house style, and a prompt to enforce tone and safety rules.
The order of operations matters. Start with prompting because it is fastest to test. Add retrieval when you run out of context or when answers drift from your source material. Fine-tune last, once you know the task well enough to write down what good looks like. Teams that fine-tune first usually pay for training on a problem they did not yet understand.
A Practical Decision Path
Ask three questions. Does the knowledge change often? If yes, retrieval. Is the format and style stable and repeated? If yes, consider fine-tuning. Is the task short and different every time? If yes, prompting is enough. Then measure on your own examples, because your examples are the only benchmark that counts. Most businesses that follow this path end up with a small fine-tuned model for the routine work, a retrieval layer for the reference material, and a prompt that ties them together.
If you want help deciding which approach fits your data and your budget, talk to us at Crowe Endeavors and we will test the options on your own examples before you commit.