# Choosing the right approach: prompt, RAG, fine-tune, or agent
A startup spent four months and $80,000 fine-tuningfine-tuningFine-tuning adapts a pre-trained model to a specific task or domain by continuing training on a smaller, targeted dataset, improving accuracy and style for that use case.View full definition → a model to answer customer questions about their product. Then a competitor shipped the same feature in a weekend by pasting their help docs into a prompt. The competitor won. The lesson: picking the wrong approach is the most expensive mistake in an AI project, and it is almost always avoidable.
There are four common ways to build with large language models (LLMs, the AI behind ChatGPT, Claude, and Gemini). Each solves a different problem. Most people reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → for the complicated one when the simple one would have worked.
Let's fix that.
Prompting. You just write good instructions and send them to the model. No setup, no code required.
RAG (Retrieval-Augmented Generation). You give the model your own documents to read before it answers. "Retrieval" means it looks things up; "augmented" means it uses what it found to write a better reply.
Fine-tuning. You train the model on many examples so it absorbs a style, format, or specialized behavior permanently.
Agents. You let the model take actions: search the web, send an email, query a database, run other tools, in a loop, until a task is done.
Here is the order you should consider them: prompt first, then RAG, then agent, and fine-tune almost last. The simplest approach that works is the right one.
Anchor everything in one real scenario: you run a mid-size law firm and want AI to help draft and answer questions about contracts.
Walk it down the tree.
Try this first. Always.
Modern models (GPT-5, Claude 4.5, Gemini 2.5) are extremely capable out of the box. If your task relies on general knowledge or reasoning, a prompt may be all you need.
For the law firm: "Explain the difference between an indemnity and a warranty clause in plain English" needs zero setup. Just ask.
from openai import OpenAI
client = OpenAI()
response = client.responses.create(
model="gpt-5",
input="Explain the difference between an indemnity "
"and a warranty clause in a commercial contract, "
"in plain English, for a non-lawyer."
)
print(response.output_text)If a prompt answers your question reliably, stop here. You are done. Do not build anything else.
For a deeper guide to writing prompts that work, the Anthropic prompt engineering docs are free and excellent.
The model does not know your contracts, your clients, or your internal policies. If the answer lives in your documents, you need RAG.
For the law firm: "What's the termination notice period in the Henderson account contract?" The model has never seen that contract. So you give it to them.
How RAG works in three steps:
1. Break your documents into chunks and store them in a searchable database (a "vector databasevector databaseA vector database stores data as high-dimensional numeric vectors (embeddings) and retrieves items by similarity rather than exact matches, powering semantic search and AI applications.View full definition →" that finds text by meaning, not just keywords).
2. When a question comes in, retrieve the most relevant chunks.
3. Paste those chunks into the prompt so the model answers from your real data.
A minimal version is just smart pasting:
relevant_clause = search_contracts("Henderson termination notice")
prompt = f"""Using ONLY the contract text below, answer the question.
If the answer isn't present, say so.
CONTRACT TEXT:
{relevant_clause}
QUESTION: What is the termination notice period?"""
response = client.responses.create(model="gpt-5", input=prompt)
print(response.output_text)RAG is the workhorse of business AI in 2026. Customer support, internal knowledge bases, document Q&A: most of it is RAG. It keeps answers grounded in your real, current data and reduces made-up answers (called "hallucinations").
Shortcut for small document sets: today's models have huge context windows (Gemini 2.5 handles over a million tokenstokensA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition →, roughly a few thousand pages). If you only have a handful of documents, you can skip the vector databasevector databaseA vector database stores data as high-dimensional numeric vectors (embeddings) and retrieves items by similarity rather than exact matches, powering semantic search and AI applications.View full definition → and paste everything in directly. Build the complex RAG pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition → only when your document pile is too big to fit.
If the task involves doing things in the real world (multiple steps, using tools, making decisions along the way), you need an agent.
For the law firm: "Review this new contract, flag any clause that conflicts with our standard terms, check whether the counterparty has past disputes in our system, and draft a summary email to the partner." That is not one answer. It is a sequence of actions with tool use.
An agent loops: think, act, observe the result, think again, until the job is finered.
response = client.responses.create(
model="gpt-5",
input="Review the attached contract. Flag clauses that "
"conflict with our standard terms, look up the "
"counterparty in our disputes database, and draft "
"a summary email to the partner.",
tools=[
{"type": "file_search"}, # read the contract
{"type": "function", "name": "query_disputes_db"},
{"type": "function", "name": "draft_email"}
]
)Agents are powerful and the hot topic of 2026, but they are harder to control and can fail in surprising ways. Give an agent the smallest set of tools it needs, and keep a human approving anything risky (like sending that email). Start agents narrow.
Knowledge check
1. According to the lesson, what is the recommended order for considering the four approaches?
2. In the opening story, why did the competitor beat the startup that spent $80,000 fine-tuning?
3. In RAG, what does the 'augmented' part of 'Retrieval-Augmented Generation' specifically refer to?
4. Select ALL correct answers about what an Agent approach enables an LLM to do, according to the lesson.
Sélectionnez toutes les réponses correctes.
5. Select ALL correct answers about when prompting alone is likely sufficient for a task.
Sélectionnez toutes les réponses correctes.
This is where fine-tuningfine-tuningFine-tuning adapts a pre-trained model to a specific task or domain by continuing training on a smaller, targeted dataset, improving accuracy and style for that use case.View full definition → comes in, and it is last for a reason.
Fine-tuningFine-tuningFine-tuning adapts a pre-trained model to a specific task or domain by continuing training on a smaller, targeted dataset, improving accuracy and style for that use case.View full definition → means showing the model hundreds or thousands of examples until a behavior becomes second nature. It is the right tool when:
For the law firm: if you have 2,000 past contracts written in your firm's distinct voice and want every draft to match it precisely, fine-tuningfine-tuningFine-tuning adapts a pre-trained model to a specific task or domain by continuing training on a smaller, targeted dataset, improving accuracy and style for that use case.View full definition → can bake that style in.
The critical distinction people get wrong: fine-tuningfine-tuningFine-tuning adapts a pre-trained model to a specific task or domain by continuing training on a smaller, targeted dataset, improving accuracy and style for that use case.View full definition → teaches *style and behavior*, not *facts*. It is bad at knowledge. If you fine-tune to add information about the Henderson contract, the model will confidently invent details. For facts, use RAG. For style, consider fine-tuningfine-tuningFine-tuning adapts a pre-trained model to a specific task or domain by continuing training on a smaller, targeted dataset, improving accuracy and style for that use case.View full definition →.
# Fine-tuning starts with example pairs, not live questions
{"messages": [
{"role": "user", "content": "Draft a confidentiality clause."},
{"role": "assistant", "content": "<your firm's exact house style here>"}
]}
# You provide hundreds of these, then train a custom model.Fine-tuningFine-tuningFine-tuning adapts a pre-trained model to a specific task or domain by continuing training on a smaller, targeted dataset, improving accuracy and style for that use case.View full definition → costs the most in time, data, and money. By 2026, fewer projects need it than people assume, because base models and RAG keep getting better. ReachReachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → for it only when the simpler tools have truly failed.
These are not rivals. The best systems stack them.
Our law firm's final tool might be: an agent that uses RAG to read contracts, runs on a fine-tuned model for house style, all driven by carefully written prompts.
But they did not build that on day one. They started with a prompt. Then added RAG when it needed their documents. Then an agent when it needed to act. They only considered fine-tuningfine-tuningFine-tuning adapts a pre-trained model to a specific task or domain by continuing training on a smaller, targeted dataset, improving accuracy and style for that use case.View full definition → after everything else was solid.
That sequence is the whole point. Each step adds cost and complexity, so you earn your way up the ladder.
1. Always start with a prompt. It is free, instant, and surprisingly capable. Most ideas should be tested here before you build anything.
2. Use RAG for facts, fine-tuning for style. Mixing these up is the most common and most expensive error. Your documents go in RAG; your tone goes in fine-tuningfine-tuningFine-tuning adapts a pre-trained model to a specific task or domain by continuing training on a smaller, targeted dataset, improving accuracy and style for that use case.View full definition →.
3. Earn your way up the ladder. Prompt, then RAG, then agent, and fine-tune almost last. Each step adds cost, so only climb when the simpler one provably fails.
4. For small document sets, skip RAG entirely. Today's large context windows let you paste a few documents straight into the prompt. Build the pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition → only when the pile gets too big.
5. Combine approaches in real systems, but add them one at a time. Validate each layer before stacking the next.