Skip to main content

Parallel tool use

In the Chains with multiple tools guide we saw how to build function-calling chains that select between multiple tools. Some models, like the OpenAI models released in Fall 2023, also support parallel function calling. This allows you to invoke multiple functions (or the same function multiple times) in a single model call.

Our previous chain from the multiple tools guides actually already supports this, we just need to use an OpenAI model capable of parallel function calling.

Setup​

We’ll use OpenAI for this guide, and will need to install its partner package:

tip

See this section for general instructions on installing integration packages.

npm install @langchain/openai

You'll need to sign up for an OpenAI key and set it as an environment variable named OPENAI_API_KEY.

We'll also use the popular validation library Zod to define our tool schemas. It's already a dependency of langchain, but you can install it explicitly like this too:

npm install zod

Tools​

Tools​

Recall the tools we set up earlier:

import { z } from "zod";
import { DynamicStructuredTool } from "@langchain/core/tools";

const addTool = new DynamicStructuredTool({
  name: "add",
  description: "Add two integers together.",
  schema: z.object({
    firstInt: z.number(),
    secondInt: z.number(),
  }),
  func: async ({ firstInt, secondInt }) => {
    return (firstInt + secondInt).toString();
  },
});

const multiplyTool = new DynamicStructuredTool({
  name: "multiply",
  description: "Multiply two integers together.",
  schema: z.object({
    firstInt: z.number(),
    secondInt: z.number(),
  }),
  func: async ({ firstInt, secondInt }) => {
    return (firstInt * secondInt).toString();
  },
});

const exponentiateTool = new DynamicStructuredTool({
  name: "exponentiate",
  description: "Exponentiate the base to the exponent power.",
  schema: z.object({
    base: z.number(),
    exponent: z.number(),
  }),
  func: async ({ base, exponent }) => {
    return (base ** exponent).toString();
  },
});

Chain​

import { ChatOpenAI } from "@langchain/openai";
import { convertToOpenAITool } from "@langchain/core/utils/function_calling";
import { JsonOutputToolsParser } from "langchain/output_parsers";
import {
  RunnableLambda,
  RunnablePassthrough,
  RunnableSequence,
} from "@langchain/core/runnables";

const model = new ChatOpenAI({
  model: "gpt-3.5-turbo-1106",
});

const tools = [multiplyTool, exponentiateTool, addTool];

const toolMap: Record<string, any> = {
  multiply: multiplyTool,
  exponentiate: exponentiateTool,
  add: addTool,
};

const modelWithTools = model.bind({
  tools: tools.map(convertToOpenAITool),
});

// Function for dynamically constructing the end of the chain based on the model-selected tool.
const callSelectedTool = RunnableLambda.from(
  (toolInvocation: Record<string, any>) => {
    const selectedTool = toolMap[toolInvocation.type];
    if (!selectedTool) {
      throw new Error(
        `No matching tool available for requested type "${toolInvocation.type}".`
      );
    }
    const toolCallChain = RunnableSequence.from([
      (toolInvocation) => toolInvocation.args,
      selectedTool,
    ]);
    // We use `RunnablePassthrough.assign` here to return the intermediate `toolInvocation` params
    // as well, but you can omit if you only care about the answer.
    return RunnablePassthrough.assign({
      output: toolCallChain,
    });
  }
);

const chain = RunnableSequence.from([
  modelWithTools,
  new JsonOutputToolsParser(),
  // .map() allows us to apply a function for each item in a list of inputs.
  // Required because the model can call multiple tools at once.
  callSelectedTool.map(),
]);
info

You can see a LangSmith trace of this example here

await chain.invoke(
  "What's 23 times 7, and what's five times 18 and add a million plus a billion and cube thirty-seven"
);
[
  {
    type: 'multiply',
    args: { firstInt: 23, secondInt: 7 },
    output: '161'
  },
  {
    type: 'multiply',
    args: { firstInt: 5, secondInt: 18 },
    output: '90'
  },
  {
    type: 'add',
    args: { firstInt: 1000000, secondInt: 1000000000 },
    output: '1001000000'
  },
  {
    type: 'exponentiate',
    args: { base: 37, exponent: 3 },
    output: '50653'
  }
]

Help us out by providing feedback on this documentation page: