Transcript: Short-Term Memory with LangGraph

Version 1: Source Transcript

This document represents the raw lecture transcript recorded from the video lecture. For architectural study notes and system design documentation, see Version 2.

• Video URL: https://www.youtube.com/watch?v=k3FUWWEwgfc
• Video ID: k3FUWWEwgfc
• Word Count: 2,472 words

Full Transcript

I'm sure that we can all agree with this. There is nothing worse than you are having a conversation with an AI agent and the AI agent keep asking you follow-up questions about things that you've just said seconds before. Hi, I'm Tom. Henry Marlin. Doug. Lucy. Hi. Oh, those are cool flip-flops. Where'd you get them? You like those? It's an interesting story. I was over on the north door the other day and Hi, I'm Tom. Huh? Henry. Hi, Marlin. If this happens, that means that the AI agent wasn't designed to remember things that pertain to a given context or conversation or more specifically a thread between the user and the AI agent. In this video, we're going to show you how to actually design this capability using the integration between Lingraph and Radis. Don't worry, you're not going to suffer any short-term memory loss. Let's get to it. All right. So let's start our implementation by taking care of the dependencies that we're going to need for this example. The first dependency and the most obvious one is going to be the radis database. Right? So in the repulsor that I'm using here, I've created this docker compos file that uh gives you the ability to spin up a radis database uh using the 8 version of the radis database. And then that's the preferable way for you to use this example. Uh optionally you can create a database on radscloud a free one. So you don't have to pay anything to run this example. Uh but for this example we're going to use docker uh and spin up the database locally. So uh all you have to do is to open up a console and then execute docker compose up. Right? So what this is going to do is basically spin up the rad's database locally. And as you can see here, when you see this message on the log, ready to accept connections TCP, that means that the rest database is open for business. So that is the go-to for you to continue the example. Right? So I'm going to minimize the console here. The database is up and running. And then the next step is taking care of the dependencies of our Jupyter notebook. Right? So I'm going to actually execute this part here while I explain it. But there are three dependencies that you need to take care of. The first one is going to be the link chain open AAI. We're going to use OpenAI to actually uh use our LLM for answering the questions. So the integration with Langchen is going to be necessary. Obviously since we're building an application with Langraph, we need to install Lingraph and also you're going to need Lingraph checkpoint radius. So what this is uh this is the actual integration that Radius provides to the Langraph framework and it provides the ability to not only use checkpointers. So I'm going to explain checkp pointers in a minute but um check uh also the ability to use a database and a vector store. So all of this integration is provided out of the box with this dependency here. So we're going to in this example explore the concept of checkpointters which is the ability to uh store information within a single thread or a single context. Right? So, uh, since we're going to use OpenAI, uh, you're going to need to create a OpenAI API key beforehand, right? So, in this case here, I'm reading my OpenAI API key as a environment variable. So, I encourage you to do this beforehand. I'm going to put here in the description of this video the link about how you can create your own API key from OpenAI. So, uh, you can set up with this. And then the next step is going to be the graph implementation. So as you may know lang graph is all about creating graphs right. So uh the concept of a graph is very powerful because it provides you the building blocks of the states and the execution of each states and the connected graphs between them. So every time you move forward between one step to another that is a checkpoint right that provides a state and the state needs to be saved somewhere. That's where the concept of checkp pointers is going to be useful. Right? So, uh our code starts with the instantiation of our model. So, we're going to create a model that is basically a chat with OpenAI. So, this is going to be the LLM that we're going to use for this example. We're going to also use a tool that we've created here manually using Python. And this is basically this function called get weather. Um this is a very simplistic function that basically given a location provided as a parameter we're going to provide the responses this if it's New York is we're going to say that it's always cloudy and if it's San Francisco it's going to be always sunny right we know that this is not always true but bear with me here in this example right so uh we're going to provide the stools as an array here as you can see and then we're going to actually start the implementation of our integration with radis so the Radis UI is uses the uh syntax that you can connect with your Radis database. Since we're using a radius database running locally here, this is the syntax that you need to use, right? And then B the building block that you're going to use for this is going to be called red saver. Right? So red saver as you can see here is part of the uh checkpoint radius link graph integration. So this is the SDK that we're going to use. And then from the connection string which is the ra U uri we're going to instantiate a checkpointer right so checkpo pointers just for you to know is a concept that is uh is kind of a native for lang graph like ling graph needs checkp pointers as I mentioned before to handle the transitions between its states of the graph right so this is has nothing to do conceptually with radius what we're going to do here is to replace the built-in checkp pointers with a custom one right provided by radice, right? And then we're going to use this checkpoint. As you can see here, we are associating the this checkpointers with this variable here called checkpointer. And when we create a graph here below, we're going to provide a model, provide the array of tools. There's only one, but it could be more than one. And as a checkpointer, we're going to use this checkpointter that points to radius. Right? So, let me execute this code um beforehand and then let's see how this graph looks like. So as you can see here this graph is very simplistic. So that that's start there's the agent. the agent's going to be like multiple may provide multiple invocation to the tools and then once it responds it goes to the end right so that's a very simplistic graph but it's going to be very useful for us to prove the point of the communication with radius right uh as a heads up uh there there is this invocation call once we create a checkpoint from radius to this function called setup right so this is very important never forget to actually execute this part here because uh behind the scenes what this does is to create the required indices uh on radis for you to materialize all the data that needs to be saved. If we actually inspect that the database right now um using radius insight we're going to see uh some keys uh actually I take that back there will be no keys stored because there's no data but the indices yeah that's the one that I would like to showcase the indices will be created already this is the result of that invocation setup right so it's created checkpoint rights checkpoints and checkpoints blobs right so uh there's no data yet because there's no execution of the graph at this point, right? So, uh let's move forward and then continue the execution of our uh example. And in this example, we're going to basically do this, right? We're going to instantiate a new thread. You can think about a thread as a conversation between one user and the uh LLM, right? So, that's a thread, right? That's a single occurrence of a conversation. And that thread is going to be named. So, this is going to be thread number one, right? So this is important because the concept of thread ID that's what is going to uniquely identify that conversation or thread for langraph and more importantly this is information is going to be saved on radis. So that's how radis is going to be able to oh okay it's part of the same conversation. So here is this set of information that is stored uh along with that conversation right. So, and then we're going to say what's the weather on New York City, right? So, that's a very simple question. And then, uh, behind the scenes, what's going to happen is that the LLM is going to use the tools that we're going that we provided to the graph and it's going to respond. Yeah, it's always cloudy, right? But subsequentially, we're going to also know what's known for, but mind that in this question here, we're not mention explicitly the uh the word New York City or New York or NY as an acron. So that's how we're going to force the llm not the llm actually lang graph to remember this piece of information that at this point is going to be stored on radis but uh before I actually execute this code let me explain only what this print string uh function does this is a utility function that I've created here for demonstration purposes so we can actually navigate the uh the result that comes from the stream uh of the graph right so I'm going to actually define this function and then let's start executing at our first interaction with the LLM. So, as I mentioned before, the question is, what is the weather on New York City? This is for thread one, right? So, as you can see here on the follow-up log, there's uh all the the questions and the invocation of the tools and then the name of the function that proves that uh the graph was able to identify given the context of the usage what tools to provide to uh fulfill that question, right? and then is going to answer that is it might be cloudy in New York City. So that that is the response that was provided by the tools that was leveraged by uh langraph before we continue with the our verification if the langraph is actually being able to remember let's take a look on radius to see if there are any keys stored right now. So uh I'm going to refresh the number of keys and as you can see now we have a couple keys already uh stored. So if you see here on checkpoint, you're going to see a couple of pieces of information already stored. So that is my original question here. And as you can see, the concept of thread ID is a very vital piece of information. That's how uh radius and ultimately langraph is able to kind of locate the information that is associated with the current conversation or uh for this for the sake of the argument the conversation that is being handled right now because remember uh LLM applications are inherently stateless right so they they're not able to maintain state so that's why it's important for you to actually provide a backend uh database such as radius that is capable of storing information uh in a fast and a scalable way. Right? So that's that's one of important things about the architecture. So now let's prove the point of remembering. So we're going to we're going to ask what is known for. So hopefully it's going to be able to detect that we're talking about New York and then put some information that only relevant to New York. So this is here's the answer, right? So yeah, New York. So it was able to remember that we're talking about New York. It's known for education and research. So there's a prestigious uh institutions like Columbia University and New York University. So the proof that we're talking about New York, right? Because I know for sure that those universities are not available in San Francisco. The iconic skyline, so know for its impressive skyline future and sk uh skyscrapers, the public transportation, the subways that we all know uh about New York and sports, right? So there's just a few highlights of makes New York City right unique and a vibrant place. So uh as you can see here all of this was able because uh we've stored all the transitions and the state of this current conversation or thread right uh in a backend store which is in this case is radius and this is was possible giving the integration that radius provides uh with langraph through this building block called checkpoints right so the tlddrs that every time you need to state to store the state between a lingraph execution and a backend store like radius. You need to instantiate a checkpointer and this checkpointer needs to be created with the SDK from langraph which is provided here for you out of the box with this building block called rad saver. How's about that? Was so easy to make sure that your AI agent remember things given the context of a given conversation using this built-in integration between radius and langraph. I hope you have enjoyed this approach and if you are interesting in see more scenarios of how to handle memory within the context of lingraph applications I will put here on the description of this video a link of a GitHub reposer that provides several examples of how you can further explore this integration between radius and langraph. Also, if you like this video, please don't forget to give it a like. And then if the context of this channel is also important to you and provides you useful information, make sure to subscribe so you never miss the next video that we're going to publish. I hope you have a great day and see you next time. [Music]

Timestamped Segments

• [00:00] I'm sure that we can all agree with this. There is nothing worse than you are having a conversation with an AI agent and the AI agent keep asking you follow-up questions about things that you've just said seconds before. Hi, I'm Tom. Henry Marlin. Doug. Lucy. Hi. Oh, those are cool flip-flops. Where'd you get them? You like those? It's an interesting story. I was over on the north door the other day and Hi, I'm Tom. Huh? Henry. Hi, Marlin. If this happens, that means that the AI agent wasn't designed to remember things that pertain to a given context or
• [00:34] conversation or more specifically a thread between the user and the AI agent. In this video, we're going to show you how to actually design this capability using the integration between Lingraph and Radis. Don't worry, you're not going to suffer any short-term memory loss. Let's get to it.
• [00:55] All right. So let's start our implementation by taking care of the dependencies that we're going to need for this example. The first dependency and the most obvious one is going to be the radis database. Right? So in the repulsor that I'm using here, I've created this docker compos file that uh gives you the ability to spin up a radis database uh using the 8 version of the radis database. And then that's the preferable way for you to use this example. Uh optionally you can create a database on radscloud a free one. So you don't have to pay anything to run this
• [01:27] example. Uh but for this example we're going to use docker uh and spin up the database locally. So uh all you have to do is to open up a console and then execute docker compose up. Right? So what this is going to do is basically spin up the rad's database locally. And as you can see here, when you see this message on the log, ready to accept connections TCP, that means that the rest database is open for business. So that is the go-to for you to continue the example. Right? So I'm going to minimize the console here. The database
• [02:00] is up and running. And then the next step is taking care of the dependencies of our Jupyter notebook. Right? So I'm going to actually execute this part here while I explain it. But there are three dependencies that you need to take care of. The first one is going to be the link chain open AAI. We're going to use OpenAI to actually uh use our LLM for answering the questions. So the integration with Langchen is going to be necessary. Obviously since we're building an application with Langraph, we need to install Lingraph and also you're going to need Lingraph checkpoint radius. So what this is uh this is the
• [02:33] actual integration that Radius provides to the Langraph framework and it provides the ability to not only use checkpointers. So I'm going to explain checkp pointers in a minute but um check uh also the ability to use a database and a vector store. So all of this integration is provided out of the box with this dependency here. So we're going to in this example explore the concept of checkpointters which is the ability to uh store information within a single thread or a single context.
• [03:04] Right? So, uh, since we're going to use OpenAI, uh, you're going to need to create a OpenAI API key beforehand, right? So, in this case here, I'm reading my OpenAI API key as a environment variable. So, I encourage you to do this beforehand. I'm going to put here in the description of this video the link about how you can create your own API key from OpenAI. So, uh, you can set up with this. And then the next step is going to be the graph implementation. So as you may know lang graph is all about creating graphs right. So uh the concept of a graph is
• [03:38] very powerful because it provides you the building blocks of the states and the execution of each states and the connected graphs between them. So every time you move forward between one step to another that is a checkpoint right that provides a state and the state needs to be saved somewhere. That's where the concept of checkp pointers is going to be useful. Right? So, uh our code starts with the instantiation of our model. So, we're going to create a model that is basically a chat with OpenAI. So, this is going to be the LLM
• [04:11] that we're going to use for this example. We're going to also use a tool that we've created here manually using Python. And this is basically this function called get weather. Um this is a very simplistic function that basically given a location provided as a parameter we're going to provide the responses this if it's New York is we're going to say that it's always cloudy and if it's San Francisco it's going to be always sunny right we know that this is not always true but bear with me here in this example right so uh we're going to provide the stools as an array here as
• [04:43] you can see and then we're going to actually start the implementation of our integration with radis so the Radis UI is uses the uh syntax that you can connect with your Radis database. Since we're using a radius database running locally here, this is the syntax that you need to use, right? And then B the building block that you're going to use for this is going to be called red saver. Right? So red saver as you can see here is part of the uh checkpoint radius link graph integration. So this is the SDK that we're going to use. And
• [05:17] then from the connection string which is the ra U uri we're going to instantiate a checkpointer right so checkpo pointers just for you to know is a concept that is uh is kind of a native for lang graph like ling graph needs checkp pointers as I mentioned before to handle the transitions between its states of the graph right so this is has nothing to do conceptually with radius what we're going to do here is to replace the built-in checkp pointers with a custom one right provided by radice, right? And then we're going to use this checkpoint.
• [05:50] As you can see here, we are associating the this checkpointers with this variable here called checkpointer. And when we create a graph here below, we're going to provide a model, provide the array of tools. There's only one, but it could be more than one. And as a checkpointer, we're going to use this checkpointter that points to radius.
• [06:09] Right? So, let me execute this code um beforehand and then let's see how this graph looks like. So as you can see here this graph is very simplistic. So that that's start there's the agent. the agent's going to be like multiple may provide multiple invocation to the tools and then once it responds it goes to the end right so that's a very simplistic graph but it's going to be very useful for us to prove the point of the communication with radius right uh as a heads up uh there there is this invocation call once we create a
• [06:42] checkpoint from radius to this function called setup right so this is very important never forget to actually execute this part here because uh behind the scenes what this does is to create the required indices uh on radis for you to materialize all the data that needs to be saved. If we actually inspect that the database right now um using radius insight we're going to see uh some keys uh actually I take that back there will be no keys stored because there's no
• [07:13] data but the indices yeah that's the one that I would like to showcase the indices will be created already this is the result of that invocation setup right so it's created checkpoint rights checkpoints and checkpoints blobs right so uh there's no data yet because there's no execution of the graph at this point, right? So, uh let's move forward and then continue the execution of our uh example. And in this example, we're going to basically do this, right?
• [07:42] We're going to instantiate a new thread. You can think about a thread as a conversation between one user and the uh LLM, right? So, that's a thread, right? That's a single occurrence of a conversation. And that thread is going to be named. So, this is going to be thread number one, right? So this is important because the concept of thread ID that's what is going to uniquely identify that conversation or thread for langraph and more importantly this is information is going to be saved on radis. So that's how radis is going to
• [08:16] be able to oh okay it's part of the same conversation. So here is this set of information that is stored uh along with that conversation right. So, and then we're going to say what's the weather on New York City, right? So, that's a very simple question. And then, uh, behind the scenes, what's going to happen is that the LLM is going to use the tools that we're going that we provided to the graph and it's going to respond. Yeah, it's always cloudy, right? But subsequentially, we're going to also know what's known for, but mind that in this question here, we're not mention explicitly the uh the word New York City
• [08:50] or New York or NY as an acron. So that's how we're going to force the llm not the llm actually lang graph to remember this piece of information that at this point is going to be stored on radis but uh before I actually execute this code let me explain only what this print string uh function does this is a utility function that I've created here for demonstration purposes so we can actually navigate the uh the result that comes from the stream uh of the graph right so I'm going to actually define this function and then let's start
• [09:22] executing at our first interaction with the LLM. So, as I mentioned before, the question is, what is the weather on New York City? This is for thread one, right? So, as you can see here on the follow-up log, there's uh all the the questions and the invocation of the tools and then the name of the function that proves that uh the graph was able to identify given the context of the usage what tools to provide to uh fulfill that question, right? and then is going to answer that is it might be cloudy in New York City. So that that is
• [09:56] the response that was provided by the tools that was leveraged by uh langraph before we continue with the our verification if the langraph is actually being able to remember let's take a look on radius to see if there are any keys stored right now. So uh I'm going to refresh the number of keys and as you can see now we have a couple keys already uh stored. So if you see here on checkpoint, you're going to see a couple of pieces of information already stored.
• [10:25] So that is my original question here. And as you can see, the concept of thread ID is a very vital piece of information. That's how uh radius and ultimately langraph is able to kind of locate the information that is associated with the current conversation or uh for this for the sake of the argument the conversation that is being handled right now because remember uh LLM applications are inherently stateless right so they they're not able to maintain state so that's why it's important for you to actually provide a
• [10:58] backend uh database such as radius that is capable of storing information uh in a fast and a scalable way. Right? So that's that's one of important things about the architecture. So now let's prove the point of remembering. So we're going to we're going to ask what is known for. So hopefully it's going to be able to detect that we're talking about New York and then put some information that only relevant to New York. So this is here's the answer, right? So yeah, New York. So it was able to remember that we're talking about New York. It's
• [11:31] known for education and research. So there's a prestigious uh institutions like Columbia University and New York University. So the proof that we're talking about New York, right? Because I know for sure that those universities are not available in San Francisco. The iconic skyline, so know for its impressive skyline future and sk uh skyscrapers, the public transportation, the subways that we all know uh about New York and sports, right? So there's just a few highlights of makes New York City right unique and a vibrant place.
• [12:02] So uh as you can see here all of this was able because uh we've stored all the transitions and the state of this current conversation or thread right uh in a backend store which is in this case is radius and this is was possible giving the integration that radius provides uh with langraph through this building block called checkpoints right so the tlddrs that every time you need to state to store the state between a lingraph execution and a backend store
• [12:36] like radius. You need to instantiate a checkpointer and this checkpointer needs to be created with the SDK from langraph which is provided here for you out of the box with this building block called rad saver. How's about that? Was so easy to make sure that your AI agent remember things given the context of a given conversation using this built-in integration between radius and langraph.
• [13:00] I hope you have enjoyed this approach and if you are interesting in see more scenarios of how to handle memory within the context of lingraph applications I will put here on the description of this video a link of a GitHub reposer that provides several examples of how you can further explore this integration between radius and langraph. Also, if you like this video, please don't forget to give it a like.
• [13:27] And then if the context of this channel is also important to you and provides you useful information, make sure to subscribe so you never miss the next video that we're going to publish. I hope you have a great day and see you next time. [Music]