{"id":9900,"date":"2024-05-10T13:49:41","date_gmt":"2024-05-10T21:49:41","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=9900"},"modified":"2025-04-29T12:41:27","modified_gmt":"2025-04-29T12:41:27","slug":"advanced-rag-algorithms-optimize-retrieval","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/advanced-rag-algorithms-optimize-retrieval\/","title":{"rendered":"The 4 Advanced RAG Algorithms You Must Know to Implement"},"content":{"rendered":"\n

Welcome to Lesson 5<\/strong> of 12<\/strong> in our free course series, LLM Twin: Building Your Production-Ready AI Replica<\/strong>. You\u2019ll learn how to use LLMs, vector DVs, and LLMOps best practices to design, train, and deploy a production ready \u201cLLM twin\u201d of yourself. This AI character will write like you, incorporating your style, personality, and voice into an LLM. For a full overview of course objectives and prerequisites, start with Lesson 1<\/a>.<\/em><\/p>\n\n\n\n

Lessons<\/strong><\/p>\n\n\n\n

    \n
  1. An End-to-End Framework for Production-Ready LLM Systems by Building Your LLM Twin<\/a><\/li>\n\n\n\n
  2. Your Content is Gold: I Turned 3 Years of Blog Posts into an LLM Training<\/a><\/li>\n\n\n\n
  3. I Replaced 1000 Lines of Polling Code with 50 Lines of CDC Magic<\/a><\/li>\n\n\n\n
  4. SOTA Python Streaming Pipelines for Fine-tuning LLMs and RAG \u2014 in Real-Time!<\/a><\/li>\n\n\n\n
  5. The 4 Advanced RAG Algorithms You Must Know to Implement<\/a><\/li>\n\n\n\n
  6. Turning Raw Data Into Fine-Tuning Datasets<\/a><\/li>\n\n\n\n
  7. 8B Parameters, 1 GPU, No Problems: The Ultimate LLM Fine-tuning Pipeline<\/a><\/li>\n\n\n\n
  8. The Engineer\u2019s Framework for LLM & RAG Evaluation<\/a><\/li>\n\n\n\n
  9. Beyond Proof of Concept: Building RAG Systems That Scale<\/a><\/li>\n\n\n\n
  10. The Ultimate Prompt Monitoring Pipeline<\/a><\/li>\n\n\n\n
  11. [Bonus] Build a scalable RAG ingestion pipeline using 74.3% less code<\/a><\/li>\n\n\n\n
  12. [Bonus] Build Multi-Index Advanced RAG Apps<\/a><\/li>\n<\/ol>\n\n\n\n
    \n\n\n\n

    In Lesson 5<\/strong>, we will focus on building an advanced retrieval module used for RAG.<\/p>\n\n\n\n

    We will show you how to implement 4 retrieval<\/strong> and post-retrieval advanced optimization techniques<\/strong> to improve<\/strong> the accuracy<\/strong> of your RAG retrieval step<\/strong>.<\/p>\n\n\n\n

    In this lesson, we will focus only on the retrieval part of the RAG system.<\/p>\n\n\n\n

    In Lesson 4<\/strong><\/a>, we showed you how to clean, chunk, embed, and load social media data to a Qdrant vector DB<\/a> (the ingestion part of RAG).<\/p>\n\n\n\n

    In future lessons, we will integrate this retrieval module into the inference pipeline for a full-fledged RAG system.<\/p>\n\n\n\n

    \"flow
    Retrieval Python Module Architecture<\/figcaption><\/figure>\n\n\n\n

    We assume <\/strong>you <\/strong>are already familiar <\/strong>with <\/strong>what <\/strong>a naive RAG looks like.<\/strong> If not<\/strong>, check out this article<\/a><\/strong> from Decoding ML<\/a>.<\/p>\n\n\n\n

    Table of Contents<\/h2>\n\n\n\n
      \n
    1. Overview of advanced RAG optimization techniques<\/a><\/li>\n\n\n\n
    2. Advanced RAG techniques applied to the LLM twin<\/a><\/li>\n\n\n\n
    3. Retrieval optimization (1): Query expansion<\/a><\/li>\n\n\n\n
    4. Retrieval optimization (2): Self query<\/a><\/li>\n\n\n\n
    5. Retrieval optimization (3): Hybrid & filtered vector search<\/a><\/li>\n\n\n\n
    6. Implement the advanced retrieval Python class<\/a><\/li>\n\n\n\n
    7. Post-retrieval optimization: Rerank using GPT-4<\/a><\/li>\n\n\n\n
    8. Running the RAG retrieval module<\/a><\/li>\n<\/ol>\n\n\n\n

      \ud83d\udd17 <\/em>Check out <\/em><\/strong>the code on GitHub<\/em><\/a> [1] and support us with a \u2b50\ufe0f<\/em><\/p>\n\n\n\n

      1. Overview of advanced RAG optimization techniques<\/h2>\n\n\n\n

      A production RAG system is split into 3 main components<\/strong>:<\/p>\n\n\n\n