ChatCloudflareWorkersAI

Workers AI allows you to run machine learning models, on the Cloudflare network, from your own code. This will help you getting started with Cloudflare Workers AI chat models. For detailed documentation of all ChatCloudflareWorkersAI features and configurations head to the API reference.

Overview

Integration details

Class	Package	Local	Serializable	PY support	Downloads	Version
`ChatCloudflareWorkersAI`	`@langchain/cloudflare`	❌	✅	❌

Model features

See the links in the table headers below for guides on how to use specific features.

Tool calling	Structured output	JSON mode	Image input	Audio input	Video input	Token-level streaming	Token usage	Logprobs
❌	❌	❌	✅	❌	❌	✅	❌	❌

Setup

To access Cloudflare Workers AI models you’ll need to create a Cloudflare account, get an API key, and install the @langchain/cloudflare integration package.

Credentials

Head to this page to sign up to Cloudflare and generate an API key. Once you’ve done this, note your CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_API_TOKEN. Passing a binding within a Cloudflare Worker is not yet supported.

Installation

The LangChain ChatCloudflareWorkersAI integration lives in the @langchain/cloudflare package:

npm install @langchain/cloudflare @langchain/core

yarn add @langchain/cloudflare @langchain/core

pnpm add @langchain/cloudflare @langchain/core

Instantiation

Now we can instantiate our model object and generate chat completions:

// @lc-docs-hide-cell

// @ts-expect-error Deno is not recognized
const CLOUDFLARE_ACCOUNT_ID = Deno.env.get("CLOUDFLARE_ACCOUNT_ID");
// @ts-expect-error Deno is not recognized
const CLOUDFLARE_API_TOKEN = Deno.env.get("CLOUDFLARE_API_TOKEN");

import { ChatCloudflareWorkersAI } from "@langchain/cloudflare";

const llm = new ChatCloudflareWorkersAI({
  model: "@cf/meta/llama-2-7b-chat-int8", // Default value
  cloudflareAccountId: CLOUDFLARE_ACCOUNT_ID,
  cloudflareApiToken: CLOUDFLARE_API_TOKEN,
  // Pass a custom base URL to use Cloudflare AI Gateway
  // baseUrl: `https://gateway.ai.cloudflare.com/v1/{YOUR_ACCOUNT_ID}/{GATEWAY_NAME}/workers-ai/`,
});

Invocation

const aiMsg = await llm.invoke([
  [
    "system",
    "You are a helpful assistant that translates English to French. Translate the user sentence.",
  ],
  ["human", "I love programming."],
])
aiMsg

AIMessage {
  lc_serializable: true,
  lc_kwargs: {
    content: 'I can help with that! The translation of "I love programming" in French is:\n' +
      "\n" +
      `"J'adore le programmati`... 4 more characters,
    tool_calls: [],
    invalid_tool_calls: [],
    additional_kwargs: {},
    response_metadata: {}
  },
  lc_namespace: [ "langchain_core", "messages" ],
  content: 'I can help with that! The translation of "I love programming" in French is:\n' +
    "\n" +
    `"J'adore le programmati`... 4 more characters,
  name: undefined,
  additional_kwargs: {},
  response_metadata: {},
  tool_calls: [],
  invalid_tool_calls: []
}

console.log(aiMsg.content)

I can help with that! The translation of "I love programming" in French is:

"J'adore le programmation."

API reference

For detailed documentation of all ChatCloudflareWorkersAI features and configurations head to the API reference.

Edit the source of this page on GitHub.

Connect these docs programmatically to Claude, VSCode, and more via MCP for real-time answers.

Popular Providers

General integrations

RAG integrations

Overview

Integration details

Model features

Setup

Credentials

Installation

Instantiation

Invocation

API reference

​Overview

​Integration details

​Model features

​Setup

​Credentials

​Installation

​Instantiation

​Invocation

​API reference

Overview

Integration details

Model features

Setup

Credentials

Installation

Instantiation

Invocation

API reference