Skip to main content

IBM watsonx.ai

This will help you get started with IBM text completion models (LLMs) using LangChain. For detailed documentation on IBM watsonx.ai features and configuration options, please refer to the IBM watsonx.ai.

Overview​

Integration details​

ClassPackageLocalSerializablePY supportPackage downloadsPackage latest
IBM watsonx.ai@langchain/community❌✅✅NPM - DownloadsNPM - Version

Setup​

To access IBM WatsonxAI models you’ll need to create an IBM watsonx.ai account, get an API key or any other type of credentials, and install the @langchain/community integration package.

Credentials​

Head to IBM Cloud to sign up to IBM watsonx.ai and generate an API key or provide any other authentication form as presented below.

IAM authentication​

export WATSONX_AI_AUTH_TYPE=iam
export WATSONX_AI_APIKEY=<YOUR-APIKEY>

Bearer token authentication​

export WATSONX_AI_AUTH_TYPE=bearertoken
export WATSONX_AI_BEARER_TOKEN=<YOUR-BEARER-TOKEN>

CP4D authentication​

export WATSONX_AI_AUTH_TYPE=cp4d
export WATSONX_AI_USERNAME=<YOUR_USERNAME>
export WATSONX_AI_PASSWORD=<YOUR_PASSWORD>
export WATSONX_AI_URL=<URL>

Once these are placed in your environment variables and object is initialized authentication will proceed automatically.

Authentication can also be accomplished by passing these values as parameters to a new instance.

IAM authentication​

import { WatsonxLLM } from "@langchain/community/llms/ibm";

const props = {
  version: "YYYY-MM-DD",
  serviceUrl: "<SERVICE_URL>",
  projectId: "<PROJECT_ID>",
  watsonxAIAuthType: "iam",
  watsonxAIApikey: "<YOUR-APIKEY>",
};
const instance = new WatsonxLLM(props);

Bearer token authentication​

import { WatsonxLLM } from "@langchain/community/llms/ibm";

const props = {
  version: "YYYY-MM-DD",
  serviceUrl: "<SERVICE_URL>",
  projectId: "<PROJECT_ID>",
  watsonxAIAuthType: "bearertoken",
  watsonxAIBearerToken: "<YOUR-BEARERTOKEN>",
};
const instance = new WatsonxLLM(props);

CP4D authentication​

import { WatsonxLLM } from "@langchain/community/llms/ibm";

const props = {
  version: "YYYY-MM-DD",
  serviceUrl: "<SERVICE_URL>",
  projectId: "<PROJECT_ID>",
  watsonxAIAuthType: "cp4d",
  watsonxAIUsername: "<YOUR-USERNAME>",
  watsonxAIPassword: "<YOUR-PASSWORD>",
  watsonxAIUrl: "<url>",
};
const instance = new WatsonxLLM(props);

If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below:

# export LANGCHAIN_TRACING_V2="true"
# export LANGCHAIN_API_KEY="your-api-key"

Installation​

The LangChain IBM watsonx.ai integration lives in the @langchain/community package:

tip

See this section for general instructions on installing integration packages.

yarn add @langchain/community @langchain/core

Instantiation​

Now we can instantiate our model object and generate chat completions:

import { WatsonxLLM } from "@langchain/community/llms/ibm";

const props = {
  decoding_method: "sample",
  max_new_tokens: 100,
  min_new_tokens: 1,
  temperature: 0.5,
  top_k: 50,
  top_p: 1,
};
const instance = new WatsonxLLM({
  version: "YYYY-MM-DD",
  serviceUrl: process.env.API_URL,
  projectId: "<PROJECT_ID>",
  spaceId: "<SPACE_ID>",
  idOrName: "<DEPLOYMENT_ID>",
  model: "<MODEL_ID>",
  ...props,
});

Note:

  • You must provide spaceId, projectId or idOrName(deployment id) in order to proceed.
  • Depending on the region of your provisioned service instance, use correct serviceUrl.
  • You need to specify the model you want to use for inferencing through model_id.

Invocation and generation​

const result = await instance.invoke("Print hello world.");
console.log(result);

const results = await instance.generate([
  "Print hello world.",
  "Print bye, bye world!",
]);
console.log(results);

print('Hello world.')<|endoftext|>
{
  generations: [ [ [Object] ], [ [Object] ] ],
  llmOutput: { tokenUsage: { generated_token_count: 28, input_token_count: 10 } }
}

Chaining​

We can chain our completion model with a prompt template like so:

import { PromptTemplate } from "@langchain/core/prompts";

const prompt = PromptTemplate.fromTemplate(
  "How to say {input} in {output_language}:\n"
);

const chain = prompt.pipe(instance);
await chain.invoke({
  output_language: "German",
  input: "I love programming.",
});
Ich liebe Programmieren.

To express that you are passionate about programming in German,

Props overwriting​

Passed props at initialization will last for the whole life cycle of the object, however you may overwrite them for a single method’s call by passing second argument as below

const result2 = await instance.invoke("Print hello world.", {
  parameters: {
    max_new_tokens: 20,
  },
});
console.log(result2);

print('Hello world.')<|endoftext|>

Tokenization​

This package has it’s custom getNumTokens implementation which returns exact amount of tokens that would be used.

const tokens = await instance.getNumTokens("Print hello world.");
console.log(tokens);
4

API reference​

For detailed documentation of all IBM watsonx.ai features and configurations head to the API reference: API docs

Was this page helpful?


You can also leave detailed feedback on GitHub.