There are three ways to make an AI work the way you want. Most people only ever need the first two. Fine-tuning sounds impressive, but it is the right tool far less often than people think.
Three ways to teach an AI
Imagine hiring a very smart new employee.
- Prompting: tell them what you want. You give clear instructions each time. Everything in Parts 1 to 3 is prompting. A custom assistant's instructions (Part 4) are saved prompting.
- Retrieval: hand them the binder. You give them your files to look things up in. They don't memorize the files; they search them when a question comes in. The technical name is RAG, short for "retrieval-augmented generation." Uploading files to a custom assistant is retrieval.
- Fine-tuning: send them to training. You show them hundreds or thousands of examples until a style or habit becomes second nature. This changes the model itself, so it acts that way without being told.
Side by side
| Prompting | Retrieval (files) | Fine-tuning | |
|---|---|---|---|
| Effort to start | Minutes | An hour or two | Days to weeks |
| Needs code? | No | No, in chat apps | Usually yes |
| Cost | Included in your plan | Included in your plan | Pay to train, and often more per use |
| Good for | Tone, format, the task itself | Facts, documents, information that changes | A consistent style or behavior at large volume |
| Bad for | Very large amounts of reference info | Changing how the AI writes or thinks | Adding facts. It learns patterns, not reliable facts. |
| Updating it | Edit the text | Swap the file | Train again |
The most common mistake is fine-tuning to teach facts. If you want the AI to know your prices or your policies, upload the document. Fine-tuning is for how it responds, not what it knows.
Fine-tuning in plain terms
Fine-tuning means giving a model a large set of example conversations, each with a question and the ideal answer, and letting it practice until it imitates them.
What it takes:
- Examples. Usually hundreds at minimum, ideally thousands, all in a consistent format. Collecting and cleaning them is most of the work.
- A provider that allows it. Some model makers offer fine-tuning on some of their models. Open-weight models (see Part 0) can be fine-tuned by anyone with the right computer or a rented one.
- Testing. A fine-tuned model can get better at one thing and worse at others. You compare it against the original before using it.
When it is worth it:
- You run the same narrow task thousands of times a day, and a small fine-tuned model can replace an expensive large one.
- You need a very specific style that instructions and examples in the prompt cannot hold.
Why it is less common now: today's models follow instructions and examples so well that prompting plus retrieval gets most people most of the way. Try those first. Fine-tune only when you can point to a specific failure they cannot fix.
Which one do you need?
Work down this list and stop at the first "yes."
- Is the answer bad because the AI misunderstood the task? Improve the prompt. Add context, an example, and a format.
- Is the answer bad because the AI doesn't know your information? Use retrieval. Upload the files to a custom assistant.
- Is the answer right but the style is never quite right, even with examples? Add more examples to the instructions first. Still wrong? Consider fine-tuning.
- Is the task huge, repetitive, and expensive to run? Fine-tuning a small model may cut costs. This is a project for a developer.
Part 7 shows what happens when you push these tools to their limit: building an AI that sounds like you.
Found this helpful? Raya.ai is built on the ideas in this series. Learn more at raya.ai (opens in a new tab), or download the app on iOS (opens in a new tab) or Android (opens in a new tab).