RAG vs. Fine-Tuning vs. Prompt Engineering

• Retrieval-Augmented Generation:
• Fine Tuning:
  • Pros:
    • Higher quality answers with respect to prompt engineering.
    • Smaller input size during inference, since we don’t need to include the context and additional instructions in the prompt.
  • Cons:
    • It can be expensive
    • The number of parameters of the model may not be sufficient to capture the entire knowledge we want to teach to it.
    • Fine tuning is not additive, meaning it may be replacing some already existing knowledge with new information.
• Prompt Engineering: