Transcript: Building a RAG Pipeline from Scratch with RedisVL ｜ Step-by-Step Tutorial

Version 1: Source Transcript

This document represents the raw lecture transcript recorded from the video lecture. For architectural study notes and system design documentation, see Version 2.

• Video URL: https://www.youtube.com/watch?v=cCTKmmGO4CY
• Video ID: cCTKmmGO4CY
• Word Count: 1,519 words

Full Transcript

have you ever asked chat GPT a question and gotten a completely madeup answer that's because llms as powerful as they are still have limitations today I'm going to explain what rag is how reddis helps and show a quick demo of rag in action we will pre-process data store embeddings and reddits retrieve relevant documents and use an llm to generate accurate answers to understand why rag is important let's first look at some key challenges with llms and why they need retrieval to improve accuracy llms rely on stale training data meaning they can't access real time or proprietary information they're costly to scale requiring more resources as models grow if you send all your data for every request they are prone to hallucinations so without real-time data retrieval llms can generate incorrect or misleading information these challenges make it difficult to use llms for real world applications that need accurate upto-date and efficient responses now that we've covered llm limitations let's talk about how retrieval augmented generation or rag for short solves these challenges rag enhances llm responses by retrieving real world Data before generating an answer making them more accurate upto-date and context aware it works in Three Steps step one embedding documents are broken into chunks and converted into Vector embeddings that capture meaning step two retrieval when a user asks a question the system finds the most relevant document ch chunks in step three augmented generation the llm then generates a response grounded in facts not just its training data by combining retrieval with generation rag ures factual reliable AI responses making it ideal for document search customer support and financial analysis now why are we using reddis for this rag pipeline reddis is a great solution because we're fast redus provides unparalleled data access with its approach to storage we're faster than any Benchmark competitor we're flexible developers can use reddits with over 30 top AI tools to give you whatever you need to deploy your AI app quickly because we work with more tools and systems customers can just get started instead of worrying about making sure each integration is working properly and lastly we're trusted we are battle proven by the biggest companies in the world and work at the scale they need we can provide 300 times higher queries per second to give you the performance and production that you expect if you'd like to follow along with the demo scan the QR code on the screen to access the tutorial this will allow you to explore the code and try things out as we go now let's connect this rag architecture to the tutorial in this session we'll build a working AI assistant that can answer queries about a recent Nike earnings call it retrieves relevant context from Nike's 10K filings and generates accurate fact-based responses using an llm here's how each component maps to our work the documents represent the 10K filings we pre-process in chunk the retrieval step is implemented using redus as a vector database to perform similarity searches based on semantic embeddings of the document chunks for augmented generation we integrate with open ai's GPT model feeding it both the retrieved context and the user's question to generate responses finally the tutorial will demonstrate how we achieve the endtoend flow from question to answer using these tools and techniques great let's jump into the demo now so step one is setting up the environment since we're running this in Google collab I'll pull the necessary resources from GitHub including financial documents and helper scripts next I'll install the required dependencies like linkchain reddis redis VL open Ai and sentence Transformers this will help us with document processing Vector storage and llm integration finally we'll set up Reda stack as our Vector database to store and retrieve embeddings efficiently once our environment is ready we can move to the pre-processing stage here we load the financial documents like the 10K filings which are PDFs and split them into smaller more manageable chunks this is important because language models and Vector search perform better with smaller self-contained pieces of text for this we use Link Chain a library designed to handle unstructured text by the end of the step our documents are broken into chunks each representing a section of the original file as you can see here we created 20111 chunks now that we have the document chunks the next step is to convert them into Vector embeddings think of embeddings as numerical representations of text that capture the meaning or semantics of the content we use a model from sentence Transformers called all mini LM L6 V2 to create these embeddings each chunk of text is represented by a high-dimensional vector these embeddings are critical because they enable us to compare the user's query with the stored chunks based on meaning rather than exact text matching after generating the embeddings we can store them in redus Long with the original text and metadata to do this we create a schema for our rdus index the schema includes the fields for the chunk ID the original text content and the vector embeddings redus uses the schema to enable efficient Vector similarity searches by the end of the step our document chunks and their embeddings are indexed and ready to be queried with our data stored in index and reddis we can now perform Vector similarity searches when a user asks a question we convert their query into an embeded just like how we did with the document chunks redus then Compares this query embedding to the stor embeddings and retrieves the chunks that are the most similar this process allows us to identify the most relevant pieces of information to answer the user's question now that we've run our query let's take a quick look at this output you can see that redis V has returned three document chunks each identified by a chunk ID in this case we have 88 80 and 87 alongside each Chunk we also see a vector distance score which tells us how similar each retrieved chunk is to our query the lower the distance the more relevant the chunk is to our input in this case chunk 88 is the closest match followed by chunk 80 and chunk 87 red SP offers additional ways to refine search results such as hybrid search sorting filtering and range queries these techniques help improve result ranking restrict searches based on specific criteria and expand retrieval Scope when needed there are examples of these in the notebook but for today we'll skip over this part to keep the focus on building the core rag pipeline great and now we can finally do rag once we have refined and filter the results we move to the generation step this is where open ai's GPT model comes into play we take the user's query and the retriev chunks and feed them into the GPT model the model generates a response based on both the context provided by the chunks and the query itself this step is the generation part of retrieval augmented generation where we combine retrieve knowledge with the reasoning capabilities of an llm great so now we can test out the pipeline to test the pipeline we can use a set of example questions for each question we convert the query into an embedding retrieve relevant document chunks using redus pass the refined results and the query to the GPT model to generate an answer this testing phase validates that the pipeline works as expected from pre-processing to answer generation it also showcases how the system can handle different types of questions so let's view the results after testing the pipeline with sample questions we can finally review the results to see how well the system performed let's just the pipeline with a new question I will type it in what is Nike's gross margin for newest FAL here amazing and it tells us that Nike's gross margin for the latest fiscal year was 43.5% as you can see the system retrieves the relevant document chunks and provides an accurate response based on Nike's financial data this demonstrates how our rag pipeline effectively grounds the model's answers in real data validating its ability to pull factual context aware information and it's that easy you now have a working rag pipeline amazing and that wraps up our tutorial today we covered how rag enhances llms by combining retrieval and generation for more urate responses in this demo we pre-processed documents into manageable chunks generated Vector embeddings to capture meaning stored and retrieved data efficiently using reddis and used open ai's GPT model to generate fact-based Answers by the end of this tutorial you've seen how redv makes building scalable rag pipelines easy powering applications like document search customer support and knowledge assistance check out more rag examples with our redis AI devhub or talk with our experts and find finally try the demo yourself using this QR code thank you so much for listening

Timestamped Segments

• [00:00] have you ever asked chat GPT a question and gotten a completely madeup answer that's because llms as powerful as they are still have limitations today I'm going to explain what rag is how reddis helps and show a quick demo of rag in action we will pre-process data store embeddings and reddits retrieve relevant documents and use an llm to generate accurate answers to understand why rag is important let's first look at some key challenges with llms and why they need retrieval to improve accuracy llms rely on stale training data meaning they can't access
• [00:35] real time or proprietary information they're costly to scale requiring more resources as models grow if you send all your data for every request they are prone to hallucinations so without real-time data retrieval llms can generate incorrect or misleading information these challenges make it difficult to use llms for real world applications that need accurate upto-date and efficient responses now that we've covered llm limitations let's talk about how retrieval augmented generation or rag for short solves these challenges rag enhances llm responses by
• [01:09] retrieving real world Data before generating an answer making them more accurate upto-date and context aware it works in Three Steps step one embedding documents are broken into chunks and converted into Vector embeddings that capture meaning step two retrieval when a user asks a question the system finds the most relevant document ch chunks in step three augmented generation the llm then generates a response grounded in facts not just its training data by combining retrieval with generation rag ures factual reliable AI responses
• [01:43] making it ideal for document search customer support and financial analysis now why are we using reddis for this rag pipeline reddis is a great solution because we're fast redus provides unparalleled data access with its approach to storage we're faster than any Benchmark competitor we're flexible developers can use reddits with over 30 top AI tools to give you whatever you need to deploy your AI app quickly because we work with more tools and systems customers can just get started instead of worrying about making sure each integration is working
• [02:16] properly and lastly we're trusted we are battle proven by the biggest companies in the world and work at the scale they need we can provide 300 times higher queries per second to give you the performance and production that you expect if you'd like to follow along with the demo scan the QR code on the screen to access the tutorial this will allow you to explore the code and try things out as we go now let's connect this rag architecture to the tutorial in this session we'll build a working AI assistant that can answer queries about a recent Nike earnings call it retrieves relevant context from Nike's 10K filings
• [02:51] and generates accurate fact-based responses using an llm here's how each component maps to our work the documents represent the 10K filings we pre-process in chunk the retrieval step is implemented using redus as a vector database to perform similarity searches based on semantic embeddings of the document chunks for augmented generation we integrate with open ai's GPT model feeding it both the retrieved context and the user's question to generate responses finally the tutorial will demonstrate how we achieve the endtoend flow from question to answer using these
• [03:25] tools and techniques great let's jump into the demo now so step one is setting up the environment since we're running this in Google collab I'll pull the necessary resources from GitHub including financial documents and helper scripts next I'll install the required dependencies like linkchain reddis redis VL open Ai and sentence Transformers this will help us with document processing Vector storage and llm integration finally we'll set up Reda stack as our Vector database to store and retrieve embeddings efficiently once our environment is ready we can move to
• [03:58] the pre-processing stage here we load the financial documents like the 10K filings which are PDFs and split them into smaller more manageable chunks this is important because language models and Vector search perform better with smaller self-contained pieces of text for this we use Link Chain a library designed to handle unstructured text by the end of the step our documents are broken into chunks each representing a section of the original file as you can see here we created 20111 chunks now that we have the document chunks the next step is to convert them into Vector
• [04:30] embeddings think of embeddings as numerical representations of text that capture the meaning or semantics of the content we use a model from sentence Transformers called all mini LM L6 V2 to create these embeddings each chunk of text is represented by a high-dimensional vector these embeddings are critical because they enable us to compare the user's query with the stored chunks based on meaning rather than exact text matching after generating the embeddings we can store them in redus Long with the original text and metadata to do this we create a schema for our rdus index the
• [05:05] schema includes the fields for the chunk ID the original text content and the vector embeddings redus uses the schema to enable efficient Vector similarity searches by the end of the step our document chunks and their embeddings are indexed and ready to be queried with our data stored in index and reddis we can now perform Vector similarity searches when a user asks a question we convert their query into an embeded just like how we did with the document chunks redus then Compares this query embedding to the stor embeddings and retrieves the chunks that are the most
• [05:37] similar this process allows us to identify the most relevant pieces of information to answer the user's question now that we've run our query let's take a quick look at this output you can see that redis V has returned three document chunks each identified by a chunk ID in this case we have 88 80 and 87 alongside each Chunk we also see a vector distance score which tells us how similar each retrieved chunk is to our query the lower the distance the more relevant the chunk is to our input in this case chunk 88 is the closest match followed by chunk 80 and chunk 87
• [06:13] red SP offers additional ways to refine search results such as hybrid search sorting filtering and range queries these techniques help improve result ranking restrict searches based on specific criteria and expand retrieval Scope when needed there are examples of these in the notebook but for today we'll skip over this part to keep the focus on building the core rag pipeline great and now we can finally do rag once we have refined and filter the results we move to the generation step this is where open ai's GPT model comes into play we take the user's query and the
• [06:46] retriev chunks and feed them into the GPT model the model generates a response based on both the context provided by the chunks and the query itself this step is the generation part of retrieval augmented generation where we combine retrieve knowledge with the reasoning capabilities of an llm great so now we can test out the pipeline to test the pipeline we can use a set of example questions for each question we convert the query into an embedding retrieve relevant document chunks using redus pass the refined results and the query to the GPT model
• [07:18] to generate an answer this testing phase validates that the pipeline works as expected from pre-processing to answer generation it also showcases how the system can handle different types of questions so let's view the results after testing the pipeline with sample questions we can finally review the results to see how well the system performed let's just the pipeline with a new question I will type it in what is Nike's gross margin for newest FAL here amazing and it tells us that Nike's
• [07:57] gross margin for the latest fiscal year was 43.5% as you can see the system retrieves the relevant document chunks and provides an accurate response based on Nike's financial data this demonstrates how our rag pipeline effectively grounds the model's answers in real data validating its ability to pull factual context aware information and it's that easy you now have a working rag pipeline amazing and that wraps up our tutorial today we covered how rag enhances llms by combining retrieval and generation for more urate
• [08:30] responses in this demo we pre-processed documents into manageable chunks generated Vector embeddings to capture meaning stored and retrieved data efficiently using reddis and used open ai's GPT model to generate fact-based Answers by the end of this tutorial you've seen how redv makes building scalable rag pipelines easy powering applications like document search customer support and knowledge assistance check out more rag examples with our redis AI devhub or talk with our experts and find finally try the demo yourself using this QR code thank
• [09:02] you so much for listening