Skip to content

Prompt engineering, RAG, or fine-tuning?

I see people ask whether they should use prompt engineering, RAG, or fine-tuning as if these are three versions of the same tool.

They solve different problems.

The easiest way I know to choose is to ask what the model is missing:

  • Does it need clearer instructions?
  • Does it need information it cannot currently see?
  • Does it need to repeat a behavior more consistently?

Your answer points toward prompting, RAG, or fine-tuning.

Start with prompt engineering

A prompt is the instruction and information you send to a language model for one request. Prompt engineering means writing that input so the model understands the task.

I start here because it is the fastest thing to change and test.

Prompting is useful when the model already has the information but does the task incorrectly. You might need it to:

  • return a particular JSON format;
  • follow a review checklist;
  • write in a defined tone;
  • cite only the sources you supplied;
  • say it does not know when information is missing.

A prompt cannot retrieve a private document that was never sent to the model. It also cannot hold an unlimited number of files. Every model has a limit on how much text it can read in one request.

Use RAG when information is missing

RAG stands for Retrieval-Augmented Generation. Your application searches for useful information, gives that information to the model, and then asks for an answer.

I use RAG when the answer depends on current product documentation, private company policies, source code, support records, plan details, inventory, or another live system.

RAG does not require a vector database. Exact error codes may need text search. Table values may need SQL. Questions written with different words may need vector search. The retrieval method should fit the information.

RAG also makes it possible to show Citations, which are links back to the files or records supporting the answer.

Use fine-tuning for repeated behavior

Fine-tuning means continuing to train an existing model with examples of the input and output you want.

The training changes the model's parameters. Parameters are the internal values the model learned during training. You do not edit them like database rows.

Fine-tuning can help when you have enough good examples and need a behavior to become more consistent across many requests. Examples include classification, a specialized output pattern, or a repeated transformation.

I would not use fine-tuning as the first way to add changing facts. Updating one policy or price inside a trained model is difficult to verify, and the model cannot easily show where that fact came from.

A practical way to choose

What is going wrong? I would start with Why
The output format is wrong Prompting The model needs clearer instructions.
The answer depends on today's policy RAG The model needs current outside information.
Exact error codes are missed Text search inside RAG Exact words should be searched directly.
A repeated task needs many examples in every prompt Fine-tuning evaluation Training may make the behavior more consistent.
Answers need current facts and a house style RAG plus prompting One supplies information and the other guides the writing.

You can combine them

A customer-support assistant might retrieve the current policy with RAG, use a prompt that requires Citations, and use a fine-tuned classifier to route the request to the right team.

I still test each part separately. If retrieval found the wrong policy, I fix retrieval. If the right policy was supplied but the output format is wrong, I fix the prompt. If a repeated behavior remains inconsistent across many tested examples, I consider fine-tuning.

Adding all three at once makes failures much harder to understand.

Learn how I evaluate retrieval and answers

Sources