Skip to content

What is RAG (Retrieval Augmented Generation)?

Do you want to create an AI chatbot? But How?

If you’re thinking about fine-tuning a LLM with your personal knowledge, you might want to think again.

The Problems with Fine-Tuning

Fine-tuning means training an existing model with new, specific data. It might sound like a good idea, but there are some big issues:

  • Not Enough Data: Most people don’t have lots of high-quality data to teach the model properly.
  • Hallucination: Fine-tuned models can give wrong or strange answers, even after training.
  • Costing: Even with enough data, fine-tuning is time-consuming and expensive.

Meet RAG: A better way

Here's what you need instead, RAG a.k.a Retrieval Augmented Generation.

Instead of training the model, RAG helps the chatbot give better answers by retrieving the correct information and then passing it to the LLM. This greatly reduces hallucination as the LLM is only using relevant information to answer the question.

Think of RAG as an open book exam. LLM can look up information from the book in real time to answer question.

Fine-tuning a model however is like an closed book exam. LLM has to rely on its training data to answer questions.

RAG vs Fine-Tuning

Note

Click on the image to zoom in.

Advantages of RAG

  • Up to Date: Chatbot can always fetch the latest information through RAG. You wont need to retrain the model every single time your business information changes.
  • Less Hallucination: Because LLM is only using relevant information to answer the question, it is less likely to hallucinate.
  • Integration: You can implement AI with your existing systems. (eg. Teams, Slack, Calendar, etc.)

How Big Companies are Using RAG

  • Grab: Grab uses RAG in their tool called Mystique to create personalized messages for each user. RAG helps Mystique find and use the right information from Grab’s data so the messages match each person’s preferences and past activities.
  • Shortwave: Shortwave email assistant uses RAG to select the most relevant data source then fetch relevant information to compose the best response.
  • Pinterest: Pinterest built a Text-to-SQL to generate SQL Queries for their database.

Note

RAG is not a replacement for fine-tuning. It is a supplement to fine-tuning.

Want to Learn More About RAG?