LLMService API Contract

The most common source of bugs in this codebase is calling LLMService incorrectly. This skill defines the exact API contract.

Service Definition

export class LLMService extends Context.Tag("LLMService")<
  LLMService,
  {
    readonly complete: (
      request: CompletionRequest,
    ) => Effect.Effect<CompletionResponse, LLMErrors>;
    readonly stream: (
      request: CompletionRequest,
    ) => Effect.Effect<Stream.Stream<StreamEvent, LLMErrors>, LLMErrors>;
    readonly completeStructured: <A>(
      request: StructuredCompletionRequest<A>,
    ) => Effect.Effect<A, LLMErrors>;
    readonly embed: (
      texts: readonly string[],
      model?: string,
    ) => Effect.Effect<readonly number[][], LLMErrors>;
    readonly countTokens: (
      messages: readonly LLMMessage[],
    ) => Effect.Effect<number, LLMErrors>;
    readonly getModelConfig: () => Effect.Effect<ModelConfig, never>;
  }
>() {}

CompletionRequest — What You Send

// Matches packages/llm-provider/src/types.ts — keep in sync
export type CompletionRequest = {
  readonly messages: readonly LLMMessage[];
  readonly model?: ModelConfig;
  readonly maxTokens?: number;
  readonly temperature?: number;
  readonly stopSequences?: readonly string[];
  readonly tools?: readonly ToolDefinition[];
  readonly systemPrompt?: string;
  readonly logprobs?: boolean;
  readonly topLogprobs?: number;
};

CORRECT usage:

const response =
  yield *
  llm.complete({
    messages: [{ role: "user", content: "What is quantum computing?" }],
    systemPrompt: "You are a helpful assistant.",
    maxTokens: 300,
    temperature: 0.7,
  });

WRONG — these will NOT compile:

// ❌ WRONG: No "prompt" field exists
llm.complete({ prompt: "Hello" });

// ❌ WRONG: No "input" field exists
llm.complete({ input: "Hello" });

// ❌ WRONG: messages must be an array of LLMMessage
llm.complete({ messages: "Hello" });

// ❌ WRONG: Do NOT wrap in Effect.tryPromise — complete() already returns Effect
yield* Effect.tryPromise({ try: () => llm.complete({ ... }), ... });

CompletionResponse — What You Get Back

interface CompletionResponse {
  readonly content: string; // ← The text response (NOT .text, NOT .result)
  readonly stopReason: StopReason; // "end_turn" | "max_tokens" | "stop_sequence" | "tool_use"
  readonly usage: TokenUsage; // Token accounting
  readonly model: string; // Model that was used
  readonly toolCalls?: readonly ToolCall[]; // If tools were invoked
}

interface TokenUsage {
  readonly inputTokens: number;
  readonly outputTokens: number;
  readonly totalTokens: number;
  readonly estimatedCost: number; // USD (NOT .cost, use .estimatedCost)
}

CORRECT field access:

const response = yield* llm.complete({ messages: [...] });
const text = response.content;                    // ✅ .content
const tokens = response.usage.totalTokens;        // ✅ .usage.totalTokens
const cost = response.usage.estimatedCost;        // ✅ .usage.estimatedCost
const reason = response.stopReason;               // ✅ .stopReason

WRONG field access:

// ❌ response.text — does not exist, use .content
// ❌ response.result — does not exist, use .content
// ❌ response.usage.cost — does not exist, use .usage.estimatedCost
// ❌ response.usage.confidence — does not exist
// ❌ response.output — does not exist, use .content

LLMMessage Types

type LLMMessage =
  | { readonly role: "system"; readonly content: string }
  | {
      readonly role: "user";
      readonly content: string | readonly ContentBlock[];
    }
  | {
      readonly role: "assistant";
      readonly content: string | readonly ContentBlock[];
    };

Error Handling

LLMService methods return Effect.Effect<T, LLMErrors> — they already handle errors using Effect. Do NOT wrap calls in Effect.tryPromise. Use Effect.mapError or Effect.catchTag to transform errors:

// ✅ CORRECT: pipe with mapError
const result = yield* llm.complete({ messages: [...] }).pipe(
  Effect.mapError((e) => new MyError({ message: `LLM failed: ${e.message}` })),
);

// ✅ CORRECT: catch specific error tags
const result = yield* llm.complete({ messages: [...] }).pipe(
  Effect.catchTag("LLMRateLimitError", (e) =>
    Effect.sleep(e.retryAfterMs).pipe(Effect.flatMap(() => llm.complete({ messages: [...] }))),
  ),
);

Embeddings (Tier 2 Memory Only)

// embed() is the SOLE embedding source for the entire framework
// Memory Tier 1 does NOT call embed()
// Memory Tier 2 calls embed() for sqlite-vec KNN search
const vectors = yield * llm.embed(["text to embed", "another text"]);
// Returns: readonly number[][] (one vector per input text)

Prompt Caching (Anthropic)

import { makeCacheable } from "@reactive-agents/llm-provider";

// Wrap static content in cacheable blocks for Anthropic prompt caching
const message: LLMMessage = {
  role: "user",
  content: [
    makeCacheable(staticSystemContext), // Cached across requests
    { type: "text", text: dynamicInput },
  ],
};

Model Presets

Available presets: "claude-haiku", "claude-sonnet", "claude-sonnet-4-5", "claude-opus", "gpt-4o-mini", "gpt-4o"

import { ModelPresets } from "@reactive-agents/llm-provider";
const config = ModelPresets["claude-sonnet"]; // { provider, model, costPer1MInput, ... }

Provider Fallback Chain

// FallbackChain in @reactive-agents/llm-provider
// Automatically falls back to secondary providers on error
.withFallbacks({
  providers: ["anthropic", "openai"],
})

Test Provider

The test provider is auto-selected when using .withTestScenario(). It returns deterministic responses from the TestTurn[] array.