Google

Functionality related to Google Cloud Platform

Chat models

Gemini Models

Access Gemini models such as gemini-1.5-pro and gemini-1.5-flex through the ChatGoogleGenerativeAI, or if using VertexAI, via the ChatVertexAI class.

GenAI

tip

See this section for general instructions on installing integration packages.

npm

npm install @langchain/google-genai @langchain/core

Yarn

yarn add @langchain/google-genai @langchain/core

pnpm

pnpm add @langchain/google-genai @langchain/core

Configure your API key.

export GOOGLE_API_KEY=your-api-key

import { ChatGoogleGenerativeAI } from "@langchain/google-genai";

const model = new ChatGoogleGenerativeAI({
  model: "gemini-pro",
  maxOutputTokens: 2048,
});

// Batch and stream are also supported
const res = await model.invoke([
  [
    "human",
    "What would be a good company name for a company that makes colorful socks?",
  ],
]);

Gemini vision models support image inputs when providing a single human message. For example:

const visionModel = new ChatGoogleGenerativeAI({
  model: "gemini-pro-vision",
  maxOutputTokens: 2048,
});
const image = fs.readFileSync("./hotdog.jpg").toString("base64");
const input2 = [
  new HumanMessage({
    content: [
      {
        type: "text",
        text: "Describe the following image.",
      },
      {
        type: "image_url",
        image_url: `data:image/png;base64,${image}`,
      },
    ],
  }),
];

const res = await visionModel.invoke(input2);

tip

Click here for the @langchain/google-genai specific integration docs

VertexAI

tip

See this section for general instructions on installing integration packages.

npm

npm install @langchain/google-vertexai @langchain/core

Yarn

yarn add @langchain/google-vertexai @langchain/core

pnpm

pnpm add @langchain/google-vertexai @langchain/core

Then, you'll need to add your service account credentials, either directly as a GOOGLE_VERTEX_AI_WEB_CREDENTIALS environment variable:

GOOGLE_VERTEX_AI_WEB_CREDENTIALS={"type":"service_account","project_id":"YOUR_PROJECT-12345",...}

or as a file path:

GOOGLE_VERTEX_AI_WEB_CREDENTIALS_FILE=/path/to/your/credentials.json

import { ChatVertexAI } from "@langchain/google-vertexai";
// Or, if using the web entrypoint:
// import { ChatVertexAI } from "@langchain/google-vertexai-web";

const model = new ChatVertexAI({
  model: "gemini-1.0-pro",
  maxOutputTokens: 2048,
});

// Batch and stream are also supported
const res = await model.invoke([
  [
    "human",
    "What would be a good company name for a company that makes colorful socks?",
  ],
]);

Gemini vision models support image inputs when providing a single human message. For example:

const visionModel = new ChatVertexAI({
  model: "gemini-pro-vision",
  maxOutputTokens: 2048,
});
const image = fs.readFileSync("./hotdog.png").toString("base64");
const input2 = [
  new HumanMessage({
    content: [
      {
        type: "text",
        text: "Describe the following image.",
      },
      {
        type: "image_url",
        image_url: `data:image/png;base64,${image}`,
      },
    ],
  }),
];

const res = await visionModel.invoke(input2);

tip

Click here for the @langchain/google-vertexai specific integration docs

The value of image_url must be a base64 encoded image (e.g., data:image/png;base64,abcd124).

Non-Gemini Models

See above for setting up authentication through Vertex AI to use these models.

Anthropic Claude models are also available through the Vertex AI platform. See here for more information about enabling access to the models and the model names to use.

PaLM models are no longer supported.

Vector Store

Vertex AI Vector Search

Vertex AI Vector Search, formerly known as Vertex AI Matching Engine, provides the industry's leading high-scale low latency vector database. These vector databases are commonly referred to as vector similarity-matching or an approximate nearest neighbor (ANN) service.

import { MatchingEngine } from "langchain/vectorstores/googlevertexai";

Tools

Google Search

• Set up a Custom Search Engine, following these instructions
• Get an API Key and Custom Search Engine ID from the previous step, and set them as environment variables GOOGLE_API_KEY and GOOGLE_CSE_ID respectively

There exists a GoogleCustomSearch utility which wraps this API. To import this utility:

import { GoogleCustomSearch } from "langchain/tools";

We can easily load this wrapper as a Tool (to use with an Agent). We can do this with:

const tools = [new GoogleCustomSearch({})];
// Pass this variable into your agent.