Together AI

caution

You are currently on a page documenting the use of Together AI models as text completion models. Many popular models available on Together AI are chat completion models.

You may be looking for this page instead.

Here's an example of calling a Together AI model as an LLM:

import { TogetherAI } from "@langchain/community/llms/togetherai";
import { PromptTemplate } from "@langchain/core/prompts";

const model = new TogetherAI({
  model: "mistralai/Mixtral-8x7B-Instruct-v0.1",
});
const prompt = PromptTemplate.fromTemplate(`System: You are a helpful assistant.
User: {input}.
Assistant:`);
const chain = prompt.pipe(model);
const response = await chain.invoke({
  input: `Tell me a joke about bears`,
});
console.log("response", response);
/**
response Sure, here's a bear joke for you: Why do bears hate shoes so much? Because they like to run around in their bear feet!
 */

API Reference:

• TogetherAI from @langchain/community/llms/togetherai
• PromptTemplate from @langchain/core/prompts

info

You can see a LangSmith trace of this example here

You can run other models through Together by changing the modelName parameter.

You can find a full list of models on Together's website.

Streaming

Together AI also supports streaming, this example demonstrates how to use this feature.

import { TogetherAI } from "@langchain/community/llms/togetherai";
import { ChatPromptTemplate } from "@langchain/core/prompts";

const model = new TogetherAI({
  model: "mistralai/Mixtral-8x7B-Instruct-v0.1",
  streaming: true,
});
const prompt = ChatPromptTemplate.fromMessages([
  ["ai", "You are a helpful assistant."],
  [
    "human",
    `Tell me a joke about bears.
Assistant:`,
  ],
]);
const chain = prompt.pipe(model);
const result = await chain.stream({});
let fullText = "";
for await (const item of result) {
  console.log("stream item:", item);
  fullText += item;
}
console.log(fullText);
/**
stream item:  Sure
stream item: ,
stream item:  here
stream item: '
stream item: s
stream item:  a
stream item:  light
stream item: -
stream item: heart
stream item: ed
stream item:  bear
stream item:  joke
stream item:  for
stream item:  you
stream item: :
stream item:

stream item:

stream item: Why
stream item:  do
stream item:  bears
stream item:  hate
stream item:  shoes
stream item:  so
stream item:  much
stream item: ?
stream item:

stream item:

stream item: Because
stream item:  they
stream item:  like
stream item:  to
stream item:  run
stream item:  around
stream item:  in
stream item:  their
stream item:  bear
stream item:  feet
stream item: !
stream item: </s>
 Sure, here's a light-hearted bear joke for you:

Why do bears hate shoes so much?

Because they like to run around in their bear feet!</s>
 */

API Reference:

• TogetherAI from @langchain/community/llms/togetherai
• ChatPromptTemplate from @langchain/core/prompts

info

You can see a LangSmith trace of this example here