Skip to main content


Friendli enhances AI application performance and optimizes cost savings with scalable, efficient deployment options, tailored for high-demand AI workloads.

This tutorial guides you through integrating ChatFriendli for chat applications using LangChain. ChatFriendli offers a flexible approach to generating conversational AI responses, supporting both synchronous and asynchronous calls.


Ensure the @langchain/community is installed.

npm install @langchain/community

Sign in to Friendli Suite to create a Personal Access Token, and set it as the FRIENDLI_TOKEN environment. You can set team id as FRIENDLI_TEAM environment.

You can initialize a Friendli chat model with selecting the model you want to use. The default model is llama-2-13b-chat. You can check the available models at


import { ChatFriendli } from "@langchain/community/chat_models/friendli";

const model = new ChatFriendli({
model: "llama-2-13b-chat", // Default value
friendliToken: process.env.FRIENDLI_TOKEN,
friendliTeam: process.env.FRIENDLI_TEAM,
maxTokens: 800,
temperature: 0.9,
topP: 0.9,
frequencyPenalty: 0,
stop: [],

const response = await model.invoke(
"Draft a cover letter for a role in software engineering."


Dear [Hiring Manager],

I am excited to apply for the role of Software Engineer at [Company Name]. With my passion for innovation, creativity, and problem-solving, I am confident that I would be a valuable asset to your team.

As a highly motivated and detail-oriented individual, ...

const stream = await
"Draft a cover letter for a role in software engineering."

for await (const chunk of stream) {


API Reference:

Was this page helpful?

You can also leave detailed feedback on GitHub.