Adding LLM and AI Capabilities (TypeScript)

Overview

There are no Golem-specific AI libraries for TypeScript. Instead, use third-party npm packages that work with the fetch API — Golem's TypeScript runtime provides full fetch support via WASI HTTP, so most LLM client libraries that use fetch internally will work out of the box.

Recommended Libraries

OpenAI

The official openai npm package works in Golem:

npm install openai

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello!' }],
});

const text = response.choices[0]?.message?.content ?? '';

Anthropic

The official @anthropic-ai/sdk package works in Golem:

npm install @anthropic-ai/sdk

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY,
});

const response = await client.messages.create({
  model: 'claude-sonnet-4-20250514',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'Hello!' }],
});

Other Providers

Any npm library that uses fetch or node:http internally should work. This includes:

• Google AI (@google/generative-ai) — Gemini models
• Cohere (cohere-ai) — chat, embeddings, reranking
• Mistral (@mistralai/mistralai) — Mistral models
• Groq (groq-sdk) — fast inference

Calling Any LLM API Directly

You can also call any LLM provider's REST API directly using fetch:

const response = await fetch('https://api.openai.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
  },
  body: JSON.stringify({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: 'Hello!' }],
  }),
});

const data = await response.json();
const text = data.choices[0]?.message?.content ?? '';

Load the golem-make-http-request-ts skill for more details on making HTTP requests.

Setting API Keys

Store provider API keys as secrets using Golem's typed config system. Load the golem-add-secret-ts skill for full details. In brief, declare the key as a config field marked with s.secret(...):

import { z } from 'zod';
import { defineAgent, s } from '@golemcloud/golem-ts-sdk';

export const MyAgent = defineAgent({
  name: 'MyAgent',
  id: { name: z.string() },
  config: {
    apiKey: s.secret(z.string()),
  },
  methods: { /* ... */ },
});

Then manage it via the CLI:

golem secret create apiKey --secret-type string --secret-value "sk-..."

Access it inside a handler with this.config.apiKey.get() — a secret field surfaces as a lazy Secret<string> handle; call .get() to reveal the current value.

Complete Agent Example

import { z } from 'zod';
import { defineAgent, method, http, s } from '@golemcloud/golem-ts-sdk';
import OpenAI from 'openai';

export const ChatAgent = defineAgent({
  name: 'ChatAgent',
  id: { chatName: z.string() },
  http: http.mount('/chats/{chatName}'),
  config: {
    apiKey: s.secret(z.string()),
  },
  methods: {
    ask: method({ input: { question: z.string() }, returns: z.string(), http: http.post('/ask') }),
  },
});

export const ChatAgentImpl = ChatAgent.implement({
  // `init` receives a context with `id`, `config`, `principal`, `phantomId`.
  init: ({ id, config }) => {
    const client = new OpenAI({ apiKey: config.apiKey.get() });
    const messages: OpenAI.ChatCompletionMessageParam[] = [
      { role: 'system', content: `You are a helpful assistant for chat '${id.chatName}'` },
    ];
    return { client, messages };
  },
  methods: {
    async ask({ question }) {
      this.messages.push({ role: 'user', content: question });

      const response = await this.client.chat.completions.create({
        model: 'gpt-4o',
        messages: this.messages,
      });

      const reply = response.choices[0]?.message?.content ?? '';
      this.messages.push({ role: 'assistant', content: reply });
      return reply;
    },
  },
});

Note: Inside a method handler, this is bound to the state returned by init plus SDK helpers (this.config, this.getId(), this.getPrincipal()). Inside init, read config/id from the context argument instead: init: ({ id, config }) => ....

Key Constraints

• Use npm libraries that internally use fetch or node:http — these work in Golem's WASM runtime
• Libraries that depend on native C/C++ bindings (e.g., onnxruntime-node) will not work
• API keys should be stored as secrets using Golem's typed config system (load the golem-add-secret-ts skill)
• All HTTP requests made from agent code are automatically durably persisted by Golem