Skip to main content

Astra DB Chat Memory

For longer-term persistence across chat sessions, you can swap out the default in-memory chatHistory that backs chat memory classes like BufferMemory for Astra DB.

Setup​

You need to install the Astra DB TS client:

npm install @datastax/astra-db-ts
tip

See this section for general instructions on installing integration packages.

npm install @langchain/openai @langchain/community

Configuration and Initalization​

There are two ways to inialize your AstraDBChatMessageHistory

If you already have an instance of the AstraDB client defined you can connect to your collection and initialize an instance of the ChatMessageHistory using the constuctor.

const client = (client = new AstraDB(
  process.env.ASTRA_DB_APPLICATION_TOKEN,
  process.env.ASTRA_DB_ENDPOINT,
  process.env.ASTRA_DB_NAMESPACE
));

const collection = await client.collection("YOUR_COLLECTION_NAME");

const chatHistory = new AstraDBChatMessageHistory({
  collection,
  sessionId: "YOUR_SESSION_ID",
});

If you don't already have an instance of an AstraDB client you can use the initialize method.

const chatHistory = await AstraDBChatMessageHistory.initialize({
  token: process.env.ASTRA_DB_APPLICATION_TOKEN ?? "token",
  endpoint: process.env.ASTRA_DB_ENDPOINT ?? "endpoint",
  namespace: process.env.ASTRA_DB_NAMESPACE,
  collectionName: "YOUR_COLLECTION_NAME",
  sessionId: "YOUR_SESSION_ID",
});

Usage​

Tip

Your collection must already exist

import { RunnableWithMessageHistory } from "@langchain/core/runnables";
import {
  ChatPromptTemplate,
  MessagesPlaceholder,
} from "@langchain/core/prompts";
import { StringOutputParser } from "@langchain/core/output_parsers";
import { ChatOpenAI } from "@langchain/openai";
import { AstraDBChatMessageHistory } from "@langchain/community/stores/message/astradb";

const model = new ChatOpenAI({
  model: "gpt-3.5-turbo",
  temperature: 0,
});

const prompt = ChatPromptTemplate.fromMessages([
  [
    "system",
    "You are a helpful assistant. Answer all questions to the best of your ability.",
  ],
  new MessagesPlaceholder("chat_history"),
  ["human", "{input}"],
]);

const chain = prompt.pipe(model).pipe(new StringOutputParser());

const chainWithHistory = new RunnableWithMessageHistory({
  runnable: chain,
  inputMessagesKey: "input",
  historyMessagesKey: "chat_history",
  getMessageHistory: async (sessionId) => {
    const chatHistory = await AstraDBChatMessageHistory.initialize({
      token: process.env.ASTRA_DB_APPLICATION_TOKEN as string,
      endpoint: process.env.ASTRA_DB_ENDPOINT as string,
      namespace: process.env.ASTRA_DB_NAMESPACE,
      collectionName: "YOUR_COLLECTION_NAME",
      sessionId,
    });
    return chatHistory;
  },
});

const res1 = await chainWithHistory.invoke(
  {
    input: "Hi! I'm Jim.",
  },
  { configurable: { sessionId: "langchain-test-session" } }
);
console.log({ res1 });
/*
{
  res1: {
    text: "Hello Jim! It's nice to meet you. My name is AI. How may I assist you today?"
  }
}
*/

const res2 = await chainWithHistory.invoke(
  { input: "What did I just say my name was?" },
  { configurable: { sessionId: "langchain-test-session" } }
);
console.log({ res2 });

/*
{
  res2: {
    text: "You said your name was Jim."
  }
}
*/

API Reference:

Help us out by providing feedback on this documentation page: