Transcript: Long-Term Memory with LangGraph

Version 1: Source Transcript

This document represents the raw lecture transcript recorded from the video lecture. For architectural study notes and system design documentation, see Version 2.

• Video URL: https://www.youtube.com/watch?v=fsENEq4F55Q
• Video ID: fsENEq4F55Q
• Word Count: 3,241 words

Full Transcript

Imagine with me you are having a conversation with an AI agent on a Monday and then during that conversation you have provided some very important piece of information that's going to be extremely relevant to the remaining of that conversation until you accomplish whatever the outcome you are trying to accomplish right but for some reason that conversation finishes on Monday and then you resume it on a Wednesday or perhaps next week and then all of the sudden when you start that new conversation the AI agent is able to remember everything that you have said before. It's been 84 years. It's okay. Just try to remember anything, anything at all. This is the example of what we call long-term memory applied to the concept of agent architectures. Right? So, we're going to use the same example here and show how you can actually implement this using the integration between Langraph and Radis. So you can reuse long-term memory across different conversation or threads and more importantly make sure that this piece of information is reused with the right user at the right time. Let's get to it. Let's get started by taking care of the dependencies that are required for this application to run. So the first one is going to be obviously the ranch database and then for this uh what you are going to use is this uh docker compose file that I provided with the sample code here in this example. So in this docker compose uh you are going to be able to instantiate a radius database using the eth version of the uh of radis and then this is handy because you don't need to install anything and if you have docker installed on your machine you can uh simply spin up by executing docker compose. So all you got to do is to docker compose up this project and then when you see this message ready to accept connections TCP that means that the rad database is open for business right so uh let's move forward and then the also the dependencies for the langraph application and just jupyter notebook is are are also important I'm going to actually start the execution here of them so there are three that you need to take care of the first one is the lang chain open aai we're going to use openai here as our LLM for this example and obviously since we're building a langraph application we need langraph as a dependency and we need a dependency that provides the integration between radius and langraph right so this is going to be fulfilled by the langraph checkpoint radius right I'm going to explain the concept of checkpoint in a minute right but before we move forward to the actual implementation let let me explain with the example that we're going to build like I'm going to uh jump to the to the end here so you can understand the logic that we're going to build. So essentially, we're going to start a set of conversations with the LLM and uniquely identify the threads or conversations that we're going to use. But more importantly, we're going to also identify the user, right? So think about the identification of the user as a unique ID that it human being uh talking with the uh the agent is going to be provided with. Right? This as you can see here is a string. So you can use a custom one, you can use a generated identifier, you're free to use whatever approach suits you. But uh the most important part here is that as we provide the same user ID, the information that pertains to that user ID needs to be stored within that code, right? And then uh every time we need to actually kind of uh deviate from that conversation with uh the same user ID but another conversation. So as you can see here the second uh interaction is going to be the same user ID with another thread right so same user but different conversation let's say for example that the user start the conversation in one day and then one day after he pops up in the chat again and started either start a new conversation or resume the conversation it doesn't matter right what matters here in the example what needs to happen is that all the pieces of information that pertains to that user the LLM needs to remember right technically it's not the LM lamb that remembers is the langraph application that pulls the data from the database where the information is stored in this case radis and then it provides as a context to the llm. So but this is the example that we're going to build. Uh let's uh walk you through step by step. Um as I mentioned before we're going to use openai here as our LLM. And because of this you need to create a open AAI API key beforehand. Right? So I'm going to provide here in the description of this video the instructions about how you can create one but as you create it make available as a environment variable right so you can as you can so you can use here in this Jupyter notebook and this is the part where you need to provide a vector store right so recorder why you need a vector store here uh so the the the integration between uh radius and langraph allows you to store different pieces of information one of them is plain data uh such as JSON document uh documents or hashes or you can store embeddings. Right? So this is more for you to uh to have a flexible way to actually store information and this information is going to be the cross thread or uh multicversations uh pieces of information that a user may provide during that interaction with the uh the agent. Right? So uh in this case we're going to use this capability of storing information on radius and for this we need a index right so index is a basic construct for you to create at radius so all the information that is stored there becomes searchable right so this is backed by something called red square engine right so the red square engine is going to be the uh the system behind the scenes that's given a prompt or input it's going to be able to find information that you were looking for right So uh what matters is that you need to define this index as we are defining here and then you are going to connect with red is that at this point is running behind the scenes and we're going to use this construct here called red store and then you can use the front connection string uh function provide the red URI provide the index and instantiate a red store right so the most important part here is that you invocate this function called setup right so what setup does it's essentially go to radis and then creates all the uh all the indexes that are going to be required for here. So if you go here on indexes you're going to see that you have like store you have store vectors uh which is essentially the indexes that are going to be used behind the scenes. When you store like plain data you're going to use store and when you store embeddings or vectors you're going to use store vectors right. Uh that's that's the consequence of you invoke that function setup. Right now we've created the rad store and then let's talk about the actual implementation of our graph. Right? So the graph here is going I'm going to execute so you can see the visualization of the graph and then I'm going to walk you through later um what it does. So the graph here is going to start call the model and then it's going to end. This is a very simple graph for demonstration purposes but it's a very powerful one right because uh what's going to do is not only to uh be able to provide the models that we're using but also we're going to be able to use the concept of vector store along with checkpoints right so uh in here we're creating the graph so builder compiles as you can see here we're providing as a parameter the checkpointer that we've created and red store right so where is this rad store is coming from we've just created here the rad store right so that's the connection with the radius that again is going to be used to store the cross thread uh pieces of information right and Ricardo what is the need of the checkpointer here so checkpointer is for you when you want to uh store the information within a single thread or the same conversation that's currently being handled by uh langraph flow uh in the in the video that that I've created before that talks about short-term memory with langraph and radis you can see the need for uh checkpointers but just remember that checkpoints are a way for you to as the langraph application executes all the states that pertains to that is a particular execution are saved in a data store like radis using checkpoints right so it is almost like we are using here both both short-term memory checkpointers and long-term memory which is the uh red store that we're providing here. Right? So um the checkpointer is created here uh by this building block called red saver and then we're also using the front connection string to instantiate the checkpointer. We also executed here the setup port. This setup port also create some indices on radius which is the indices that you can see here. So that's the checkpoint rights the checkpoints and the checkpoints blobs. So those three additional indices were created giving the usage of checkpointers. Right? So uh also the actual graph that we've created as I mentioned before is a simple graph. So we created the state graph here and then we added a node called call model that's essentially points to this call model function here and then we added a edge which is the start where the edge is the beginning of the the your graph. So the call model is where the actually the magic happens. So the call models we've we've provided the to the lang graph the ability to interact with the pieces of information is stored on red. So uh you can ask Ricardo but since we've have the uh the checkp pointers already instantiated and the vector store already instantiated do we need to actually kind of provide the st information uh yes if you want to customize uh how the the the messages and the pieces of information are going to be retrieved from the vector source. So in here we're uh we're using like some construct like where is the name space that we're using here to actually interact with the piece information that is stored there. So we've using the keys like memories and then we're uh perusing like the arrays like messages which is the states that is stored on this and then we're using the store search uh as a function to actually iterate with this. So and in this core module we're actually providing the langraph the ability to how to search data at radice right so with name spaces with key which key prefixes which entities needs to be evaluated for you to um locate a specific type of information right so now that we have our graph defined let's start using it right so I've created this uh utility function here called print string that basically is going to walk through a stream and then it's basically for for us to not able to be able to manipulate like if it's a simple message just print a message or if it's a JSON uh one we're going to actually uh print pretty right so that's so you can see more visually uh in our console right so let's start our conversation by uh providing the config which is the is going to be thread ID number one and then the user number one and the message is going to be hi my name is Bob Right. So that is going to be useful for us to make sure that this piece of information is stored on radius. Right? So as you can see the AI message or LLM responded, hello Bob, how can I assist you today? Right? And then this is going to be the equivalent right of a different conversation. Let's say for example that you have different window popups open, right? The first one is one conversation identified by thread ID one. And this one as you can see here uses the thread ID two. Right? So two different conversations but from the same user. And then the question that the user the human is going to ask is going to be what is my name? And guess what? The LM or more importantly the langraph agent needs to be able to okay here's the thread ID number two. So a brand new thread ID, brand new conversation but same user. I'm going to use this integration that was provided by the call model before to actually inspect the vector store and try to locate information about user number one and it's going to be able to find as you can see here. So the question is what is my name? said yeah your name is Bob I remember that because you mentioned this before right. So this is what makes the langraph uh integration with reds particularly useful because this is the type of experience that users want to have in the real world that uh makes them feel a little bit more comfortable talking with the AI right because uh it it gives us that feeling that okay it's I'm interacting and as I interacting is remembering things but this is what we call long-term memory right because uh like I said before this is a second conversation that imagine that maybe took place uh two days, three days, a week later than the previous conversation. And even though that happens, uh the agent was able to remember that uh piece of information. So that's how that's why we we we say that it's called long-term memory because it uh the persistency and the durability of that information takes more than the single thread or the single conversation to expire, right? And then to validate if this is actually also true because we have to also negate the the the the validation here right because we can say okay let's start a new conversation thread id three and then another user right and then let's ask the same question what is my name so presumably the uh lingraph state and the vector store associated to this is not going to have any piece of information of a user ID number two so what needs to happen is that I don't know who you more unfortunately and there you have it like the the answer for the AI was I'm sorry I don't have access to the personal information about you so including your name if you would like you can tell me your name and how I'd like me to for you to be addressed so if if I provided my name here we we're going to be 100% sure that it's going to remember because all the plumbing and integration with RAD is already took place so we're going to need we're going to be able to store that information but this is important because that proves proves that uh pieces of information associated with a given user are not going to be swapped or exchanged with another user. Right? Think about this as the concept of sessions on an HTTP environment, right? You don't want your shopping cart of uh to be shared or reused by another uh user, right? Or payment data or credit card information, right? So the this is the importance of being able to associate pieces of data with different threads right for specific users but being able to reuse that piece of information in multiple threads and multiple conversations right so just to wrap up here uh optionally uh the SDK from red store gives you the ability to also uh inspect that information that is stored there right so you can use this red store search API to like walk through the memories from the conversations, right? And also optionally, if you have access to radius insight and you can simply uh use your your ID to actually store this. If you go to store for example, you're going to see here that the value of that conversation username is Bob is stored here as a JSON document. Right? So also uh and that happened out of the box. We also store that same piece of information right memories we call here memories one right and as you can see here instead of actually the JSON document we stored the embedding or the set of vectors that it is the equivalent of the same information so what that means is that you can also perform similarity search to answer the question for that user ID is his name Bob so that's that's a a different way for you to retrieve the same piece of information this is more on the arena of being flexible in how you retrieve the data is not necessarily a requirement but it it is nice for this integration for radisd and langraph to provides you the ability to store even plain data and uh vector data so you can search whatever approach you want to pursue that is great right now you have the ability to make sure that your AI agent not only remember things that were discussed before but use them with the right thread and with the right user All right, this is very important not only for security purposes but for the user experience that uh the human being are having with your AI agent. If you like the approach that we have used here, I highly encourage you to take a look on the set of examples that are available on the uh GitHub reposer that I'm going to post here on the description of this video. It contains different other approaches that you can leverage between this integration of radius and langraph like other scenarios where memory management are relevant and it's going to give you the ability to customize how you actually uh pull data store data and retrieve data at the right moment at the right time right but different scenarios and examples and use cases right if you like this video make sure to give it a like and if you enjoy the content from this channel I would highly encourage you to also subscribe it so you never miss the next video that we're going to publish. I appreciate the time that you have spent here today with me and as always, see you next time.

Timestamped Segments

• [00:00] Imagine with me you are having a conversation with an AI agent on a Monday and then during that conversation you have provided some very important piece of information that's going to be extremely relevant to the remaining of that conversation until you accomplish whatever the outcome you are trying to accomplish right but for some reason that conversation finishes on Monday and then you resume it on a Wednesday or perhaps next week and then all of the sudden when you start that new conversation the AI agent is able to remember everything that you have said
• [00:32] before. It's been 84 years. It's okay. Just try to remember anything, anything at all. This is the example of what we call long-term memory applied to the concept of agent architectures. Right? So, we're going to use the same example here and show how you can actually implement this using the integration between Langraph and Radis. So you can reuse long-term memory across different conversation or threads and more importantly make sure
• [01:05] that this piece of information is reused with the right user at the right time. Let's get to it. Let's get started by taking care of the dependencies that are required for this application to run. So the first one is going to be obviously the ranch database and then for this uh what you are going to use is this uh docker compose file that I provided with the sample code here in this example. So in this docker compose uh you are going to be able to instantiate a radius database using the eth version of the uh of radis
• [01:39] and then this is handy because you don't need to install anything and if you have docker installed on your machine you can uh simply spin up by executing docker compose. So all you got to do is to docker compose up this project and then when you see this message ready to accept connections TCP that means that the rad database is open for business right so uh let's move forward and then the also the dependencies for the langraph application and just jupyter notebook is are are also important I'm going to actually start the execution
• [02:11] here of them so there are three that you need to take care of the first one is the lang chain open aai we're going to use openai here as our LLM for this example and obviously since we're building a langraph application we need langraph as a dependency and we need a dependency that provides the integration between radius and langraph right so this is going to be fulfilled by the langraph checkpoint radius right I'm going to explain the concept of checkpoint in a minute right but before we move forward to the actual implementation let let me explain with the example that we're going to build
• [02:44] like I'm going to uh jump to the to the end here so you can understand the logic that we're going to build. So essentially, we're going to start a set of conversations with the LLM and uniquely identify the threads or conversations that we're going to use. But more importantly, we're going to also identify the user, right? So think about the identification of the user as a unique ID that it human being uh talking with the uh the agent is going to be provided with. Right? This as you can see here is a string. So you can use
• [03:17] a custom one, you can use a generated identifier, you're free to use whatever approach suits you. But uh the most important part here is that as we provide the same user ID, the information that pertains to that user ID needs to be stored within that code, right? And then uh every time we need to actually kind of uh deviate from that conversation with uh the same user ID but another conversation. So as you can see here the second uh interaction is going to be the same user ID with another thread right so same user but different conversation
• [03:52] let's say for example that the user start the conversation in one day and then one day after he pops up in the chat again and started either start a new conversation or resume the conversation it doesn't matter right what matters here in the example what needs to happen is that all the pieces of information that pertains to that user the LLM needs to remember right technically it's not the LM lamb that remembers is the langraph application that pulls the data from the database where the information is stored in this case radis and then it provides as a context to the llm. So but this is the
• [04:25] example that we're going to build. Uh let's uh walk you through step by step. Um as I mentioned before we're going to use openai here as our LLM. And because of this you need to create a open AAI API key beforehand. Right? So I'm going to provide here in the description of this video the instructions about how you can create one but as you create it make available as a environment variable right so you can as you can so you can use here in this Jupyter notebook and this is the part where you need to provide a vector store right so recorder
• [04:58] why you need a vector store here uh so the the the integration between uh radius and langraph allows you to store different pieces of information one of them is plain data uh such as JSON document uh documents or hashes or you can store embeddings. Right? So this is more for you to uh to have a flexible way to actually store information and this information is going to be the cross thread or uh multicversations uh pieces of information that a user may provide
• [05:31] during that interaction with the uh the agent. Right? So uh in this case we're going to use this capability of storing information on radius and for this we need a index right so index is a basic construct for you to create at radius so all the information that is stored there becomes searchable right so this is backed by something called red square engine right so the red square engine is going to be the uh the system behind the scenes that's given a prompt or input it's going to be able to find information that you were looking for
• [06:03] right So uh what matters is that you need to define this index as we are defining here and then you are going to connect with red is that at this point is running behind the scenes and we're going to use this construct here called red store and then you can use the front connection string uh function provide the red URI provide the index and instantiate a red store right so the most important part here is that you invocate this function called setup right so what setup does it's essentially go to radis and then creates
• [06:36] all the uh all the indexes that are going to be required for here. So if you go here on indexes you're going to see that you have like store you have store vectors uh which is essentially the indexes that are going to be used behind the scenes. When you store like plain data you're going to use store and when you store embeddings or vectors you're going to use store vectors right.
• [06:59] Uh that's that's the consequence of you invoke that function setup. Right now we've created the rad store and then let's talk about the actual implementation of our graph. Right? So the graph here is going I'm going to execute so you can see the visualization of the graph and then I'm going to walk you through later um what it does. So the graph here is going to start call the model and then it's going to end.
• [07:23] This is a very simple graph for demonstration purposes but it's a very powerful one right because uh what's going to do is not only to uh be able to provide the models that we're using but also we're going to be able to use the concept of vector store along with checkpoints right so uh in here we're creating the graph so builder compiles as you can see here we're providing as a parameter the checkpointer that we've created and red store right so where is this rad store is coming from we've just
• [07:57] created here the rad store right so that's the connection with the radius that again is going to be used to store the cross thread uh pieces of information right and Ricardo what is the need of the checkpointer here so checkpointer is for you when you want to uh store the information within a single thread or the same conversation that's currently being handled by uh langraph flow uh in the in the video that that I've created before that talks about short-term memory with langraph and radis you can see the need for uh
• [08:32] checkpointers but just remember that checkpoints are a way for you to as the langraph application executes all the states that pertains to that is a particular execution are saved in a data store like radis using checkpoints right so it is almost like we are using here both both short-term memory checkpointers and long-term memory which is the uh red store that we're providing here. Right? So um the checkpointer is created here uh by this building block called red saver and then we're also using the front
• [09:06] connection string to instantiate the checkpointer. We also executed here the setup port. This setup port also create some indices on radius which is the indices that you can see here. So that's the checkpoint rights the checkpoints and the checkpoints blobs. So those three additional indices were created giving the usage of checkpointers.
• [09:28] Right? So uh also the actual graph that we've created as I mentioned before is a simple graph. So we created the state graph here and then we added a node called call model that's essentially points to this call model function here and then we added a edge which is the start where the edge is the beginning of the the your graph. So the call model is where the actually the magic happens. So the call models we've we've provided the to the lang graph the ability to interact with the pieces of information
• [10:03] is stored on red. So uh you can ask Ricardo but since we've have the uh the checkp pointers already instantiated and the vector store already instantiated do we need to actually kind of provide the st information uh yes if you want to customize uh how the the the messages and the pieces of information are going to be retrieved from the vector source.
• [10:23] So in here we're uh we're using like some construct like where is the name space that we're using here to actually interact with the piece information that is stored there. So we've using the keys like memories and then we're uh perusing like the arrays like messages which is the states that is stored on this and then we're using the store search uh as a function to actually iterate with this. So and in this core module we're actually providing the langraph the ability to how to search data at radice right so with name spaces with key which
• [11:01] key prefixes which entities needs to be evaluated for you to um locate a specific type of information right so now that we have our graph defined let's start using it right so I've created this uh utility function here called print string that basically is going to walk through a stream and then it's basically for for us to not able to be able to manipulate like if it's a simple message just print a message or if it's a JSON uh one we're going to actually uh print pretty right so that's so you can
• [11:33] see more visually uh in our console right so let's start our conversation by uh providing the config which is the is going to be thread ID number one and then the user number one and the message is going to be hi my name is Bob Right. So that is going to be useful for us to make sure that this piece of information is stored on radius. Right? So as you can see the AI message or LLM responded, hello Bob, how can I assist you today?
• [12:02] Right? And then this is going to be the equivalent right of a different conversation. Let's say for example that you have different window popups open, right? The first one is one conversation identified by thread ID one. And this one as you can see here uses the thread ID two. Right? So two different conversations but from the same user.
• [12:26] And then the question that the user the human is going to ask is going to be what is my name? And guess what? The LM or more importantly the langraph agent needs to be able to okay here's the thread ID number two. So a brand new thread ID, brand new conversation but same user. I'm going to use this integration that was provided by the call model before to actually inspect the vector store and try to locate information about user number one and it's going to be able to find as you can
• [12:59] see here. So the question is what is my name? said yeah your name is Bob I remember that because you mentioned this before right. So this is what makes the langraph uh integration with reds particularly useful because this is the type of experience that users want to have in the real world that uh makes them feel a little bit more comfortable talking with the AI right because uh it it gives us that feeling that okay it's I'm interacting and as I interacting is remembering things but this is what we
• [13:31] call long-term memory right because uh like I said before this is a second conversation that imagine that maybe took place uh two days, three days, a week later than the previous conversation. And even though that happens, uh the agent was able to remember that uh piece of information.
• [13:50] So that's how that's why we we we say that it's called long-term memory because it uh the persistency and the durability of that information takes more than the single thread or the single conversation to expire, right? And then to validate if this is actually also true because we have to also negate the the the the validation here right because we can say okay let's start a new conversation thread id three and then another user right and then let's ask the same question what is my name so presumably the uh lingraph state and the
• [14:27] vector store associated to this is not going to have any piece of information of a user ID number two so what needs to happen is that I don't know who you more unfortunately and there you have it like the the answer for the AI was I'm sorry I don't have access to the personal information about you so including your name if you would like you can tell me your name and how I'd like me to for you to be addressed so if if I provided my name here we we're going to be 100% sure that it's going to remember because all the plumbing and integration with RAD is already took place so we're going to
• [15:00] need we're going to be able to store that information but this is important because that proves proves that uh pieces of information associated with a given user are not going to be swapped or exchanged with another user. Right? Think about this as the concept of sessions on an HTTP environment, right?
• [15:18] You don't want your shopping cart of uh to be shared or reused by another uh user, right? Or payment data or credit card information, right? So the this is the importance of being able to associate pieces of data with different threads right for specific users but being able to reuse that piece of information in multiple threads and multiple conversations right so just to wrap up here uh optionally uh the SDK from red store gives you the ability to also uh inspect that information that is
• [15:54] stored there right so you can use this red store search API to like walk through the memories from the conversations, right? And also optionally, if you have access to radius insight and you can simply uh use your your ID to actually store this. If you go to store for example, you're going to see here that the value of that conversation username is Bob is stored here as a JSON document. Right?
• [16:23] So also uh and that happened out of the box. We also store that same piece of information right memories we call here memories one right and as you can see here instead of actually the JSON document we stored the embedding or the set of vectors that it is the equivalent of the same information so what that means is that you can also perform similarity search to answer the question for that user ID is his name Bob so that's that's a a different way for you to retrieve the same piece of
• [16:55] information this is more on the arena of being flexible in how you retrieve the data is not necessarily a requirement but it it is nice for this integration for radisd and langraph to provides you the ability to store even plain data and uh vector data so you can search whatever approach you want to pursue that is great right now you have the ability to make sure that your AI agent not only remember things that were discussed before but use them with the right thread and with the right user All right, this is very important not only
• [17:28] for security purposes but for the user experience that uh the human being are having with your AI agent. If you like the approach that we have used here, I highly encourage you to take a look on the set of examples that are available on the uh GitHub reposer that I'm going to post here on the description of this video. It contains different other approaches that you can leverage between this integration of radius and langraph like other scenarios where memory management are relevant and it's going to give you the ability to customize how
• [18:00] you actually uh pull data store data and retrieve data at the right moment at the right time right but different scenarios and examples and use cases right if you like this video make sure to give it a like and if you enjoy the content from this channel I would highly encourage you to also subscribe it so you never miss the next video that we're going to publish. I appreciate the time that you have spent here today with me and as always, see you next time.