Migrate to AgentControl

You're using a skill that will guide you through migrating an application from hardcoded LLM prompts to a full LaunchDarkly AgentControl implementation. Your job is to run the migration in five stages, stopping at each stage for the user to confirm:

1. Audit the code — read-only scan that produces a structured list of everything hardcoded (prompt, model, parameters, tools, app-scoped knobs).
2. Wrap the call — install the SDK, create the config in LaunchDarkly with a fallback that mirrors the hardcoded values, and rewrite the call site to fetch the config fresh on every request.
3. Move the tools — extract each tool's JSON schema, attach it to the config, and swap every call site that references the old tool list.
4. Add tracking — wire the per-request tracker (duration, tokens, success/error) around the provider call.
5. Attach evaluators — either offline evals via the Playground + Datasets, or online judges that score sampled traffic automatically.

⚠️ Three first-run failure modes to avoid.

1. Tracker in the wrong scope. For an agent with a loop, mint create_tracker() once per user turn in a setup_run entry node — not inside call_model. Per-iteration factory calls produce N runIds and trip the at-most-once guards. See agent-mode-frameworks.md § Custom StateGraph.
2. load_chat_model wrapper reuse. Templates like langchain-ai/react-agent ship a load_chat_model(f"{provider}/{name}") helper that wraps init_chat_model(...) and silently drops every variation parameter. Delete it (don't just avoid using it) and replace call sites with create_langchain_model(ai_config).
3. Fallthrough not flipped after /configs-create. A freshly-created config's fallthrough points at an auto-generated disabled variation, so the SDK returns enabled=False until /configs-targeting runs. Flip it before Stage 2 verification.

Coverage — which shapes are well-trodden vs require extrapolation

The skill is optimized for Python and Node.js / TypeScript; other languages are install-only. Within Python and Node the coverage tiers are:

Shape	Python	Node.js	Reference
One-shot completion (direct OpenAI / Anthropic / Bedrock / Gemini call)	✅ Worked example	✅ Worked example	before-after-examples.md, per-provider docs in built-in-metrics/references/
Chat loop via managed runner (ManagedModel)	✅ Tier 1 pattern	✅ Tier 1 pattern	built-in-metrics SKILL.md
LangChain single-call	✅ Worked example	✅ Worked example	langchain-tracking.md
LangGraph prebuilt agent (Python langchain.agents.create_agent, Node createReactAgent)	✅ Worked example	✅ Worked example	agent-mode-frameworks.md § LangGraph
LangGraph custom StateGraph with run-scoped tracker (setup_run + call_model + finalize)	✅ Deep worked example	⚠️ Mentioned — translate from Python	agent-mode-frameworks.md § Custom StateGraph
CrewAI Agent	✅ Worked example	— (not a Node framework)	agent-mode-frameworks.md § CrewAI
Strands Agent	✅ Worked example	⚠️ BedrockModel + OpenAIModel only (no Anthropic)	agent-mode-frameworks.md § Strands
Custom ReAct loop (hand-rolled, any framework or none)	✅ Worked example	⚠️ Apply framework-agnostic invariants; translate from Python	agent-mode-frameworks.md § Custom ReAct loop
Vercel AI SDK (generateText / streamText)	— (not a Python framework)	⚠️ Provider package exists; no worked example in skill	built-in-metrics provider-package matrix
Streaming (SSE / WebSocket)	⚠️ Delegated to built-in-metrics streaming doc	⚠️ Same — use trackStreamMetricsOf + manual TTFT	streaming-tracking.md
Multi-agent graph (supervisor + workers)	⚠️ Out of main scope; see reference	⚠️ Out of main scope; see reference	agent-graph-reference.md
Non-LangGraph agent frameworks (Pydantic AI, DSPy, AutoGen, Haystack, LlamaIndex agents, Semantic Kernel)	⚠️ Apply the three invariants; no framework-specific example	⚠️ Same	agent-mode-frameworks.md § Framework-agnostic invariants
Go, Ruby, .NET	ℹ️ Install commands only	ℹ️ Install commands only	phase-1-analysis-checklist.md § SDK routing table

Reading the key: ✅ = follow the skill verbatim; ⚠️ = the architecture applies but you'll translate idioms or cross-reference another skill; ℹ️ = skill doesn't go past the install step.

If the target app is in the ⚠️ column, start by reading agent-mode-frameworks.md § Framework-agnostic invariants — those three rules (one agent_config per turn, one tracker per turn, at-most-once methods fire once at turn end) apply regardless of framework, and every code snippet in this skill is an instantiation of them. Translate the Python example's shape onto the target framework's primitives.

Prerequisites

This skill requires the remotely hosted LaunchDarkly MCP server to be configured in your environment, and an application that already calls an LLM provider with hardcoded model, prompt, and parameter values.

Required environment:

• LD_SDK_KEY — server-side SDK key (starts with sdk-) from the target LaunchDarkly project

MCP tools used directly by this skill: none — every LaunchDarkly write happens in a focused sibling skill.

Check the SDK CHANGELOG before applying any pattern. The API surface described throughout this skill targets the SDK behavior at the time of the skill's last update; SDK releases can rename, remove, or split methods after that. Before you start, fetch the latest CHANGELOG for the SDK(s) you'll target and skim for anything that contradicts the pattern you're about to apply:

• Python: https://github.com/launchdarkly/python-server-sdk-ai/blob/main/packages/sdk/server-ai/CHANGELOG.md (and per-provider CHANGELOGs under packages/ai-providers/server-ai-{openai,langchain}/CHANGELOG.md)
• Node: https://github.com/launchdarkly/js-core/blob/main/packages/sdk/server-ai/CHANGELOG.md (and per-provider CHANGELOGs under packages/ai-providers/server-ai-{openai,langchain,vercel}/CHANGELOG.md)

If a CHANGELOG entry post-dates this skill and changes an API you're about to use, the CHANGELOG wins — and the skill should be updated.

Hand-off model. This skill does not auto-invoke other skills. At each stage that needs a LaunchDarkly write, this skill prepares the inputs (config key, mode, model, prompt, tool schemas, judge keys) and then tells the user to run the next slash-command themselves. After the user finishes that sibling skill, return to the next step here. Treat the "Delegate" lines below as next-step instructions, not auto-handoffs.

Sibling skills the user runs at each stage:

• projects — pre-Stage 2, only if no project exists yet
• configs-create — Stage 2 (creates the config and first variation)
• tools — Stage 3 (creates tool definitions and attaches them)
• configs-targeting — between Stage 2 and Stage 4 (promotes the new variation to fallthrough so the SDK actually serves it)
• online-evals — Stage 5 (attaches judges, creates custom judges)

Core Principles

1. Inspect before you mutate. Every stage begins with a read-only audit. Do not touch code until Step 1 is confirmed by the user.
2. Replace config, not business logic. The SDK call is a drop-in for the place where the model, parameters, and prompt are defined — not for the provider call itself. OpenAI/Anthropic/Bedrock calls stay where they are.
3. Fallback mirrors current behavior. The fallback passed to completion_config / agent_config must preserve the hardcoded values you removed, so the app is unchanged if LaunchDarkly is unreachable.
4. Stages are ordered. Wrap before you add tools. Add tools before you track. Track before you add evals. Skipping ahead produces configs without traffic, metrics without context, and judges with nothing to score.
5. Hand off to focused skills, manually. Each stage that needs a LaunchDarkly write tells the user to run a sibling slash-command (/configs-create, /tools, /configs-targeting, /online-evals) and waits for them to come back. This skill does not auto-invoke other skills.

Workflow

Minimum viable migration

Stages 1–4 (audit, wrap, tools, tracker) are independently shippable. A migration that stops after Stage 4 is complete, production-ready, and delivers the core value — externalized prompts and model config, targeting, variation A/B testing, and Monitoring-tab metrics. Stage 5 (evaluators) is a quality-of-life addition, not a gate. Do not block a Stage-4 rollout on evaluators; ship the run-scoped tracker path, verify metrics flow, then come back for Stage 5 when the team has time to curate a dataset.

That said, do not skip Stage 4. A migration without the tracker gives you externalized prompts but no visibility, which is most of the payoff left on the floor.

Step 1: Audit the codebase (Stage 1)

This is the first stage. It is read-only — no code writes, no LaunchDarkly resources created. The goal is to scan the repo and produce a structured manifest of every hardcoded value that needs to move, then hand the manifest back to the user for confirmation before any code is touched in Stage 2.

Use phase-1-analysis-checklist.md to scan:

1. Language and package manager — Python (pip/poetry/uv), TypeScript/JavaScript (npm/pnpm/yarn), Go, Ruby, .NET
2. LLM provider — OpenAI, Anthropic, Bedrock, Gemini, LangChain, LangGraph, CrewAI, Strands
3. Existing LaunchDarkly usage — any pre-existing LDClient or ldclient initialization to reuse
4. Hardcoded model configs — model name string literals, temperature / max_tokens / top_p, system prompts, instruction strings
5. Template placeholders in prompts — .format() calls, f-strings in prompt constants, JS/TS template literals, %(var)s, hand-rolled str.replace("__VAR__", ...). Flag each placeholder name and its runtime-value source; all get rewritten to Mustache {{ variable }} in Stage 2.
6. Externalized prompt files — scan YAML / JSON / TOML / Markdown / .prompt / .j2 files and prompt-template registries (langchain.hub.pull(...), LangSmith client.pull_prompt(...)) for prompts loaded at runtime. Common shapes: CrewAI agents.yaml / tasks.yaml, LangChain Promptfiles, k8s ConfigMap overlays, Pydantic Settings classes with prompt_* fields. Same Mustache rewrite (sub-step 5 of Stage 2) applies if the placeholder syntax differs. See phase-1-analysis-checklist.md § 4.
7. Hardcoded app-scoped knobs — search-result limits, retry budgets, tool-timeout overrides, feature toggles, any config-dataclass field that isn't a prompt or model parameter but still governs agent behavior. These belong in model.custom on the variation (not model.parameters, which is forwarded to the provider SDK and will crash on unknown kwargs).
8. Mode decision — completion mode (chat messages array) or agent mode (single instructions string). Completion mode is the default and the only mode that supports judges attached in the UI.

For each hardcoded target the audit finds, record:

• File path and line range
• Current value (model name, full prompt text, parameter dict)
• Target config field (model.name, model.parameters.temperature, messages[].content, instructions)
• Whether the surrounding call uses function calling / tools (drives Stage 3)
• Whether the surrounding call has retry logic (affects where Stage 4 tracker calls go)

This manifest is the contract for the next four stages.

Stage 1 output (return to user as a structured summary):

Language: Python 3.12
Package manager: uv
LLM provider: OpenAI
Existing LD SDK: none
Target mode: completion
Hardcoded targets:
  - src/chat.py:42   model="gpt-4o"
  - src/chat.py:43   temperature=0.7, max_tokens=2000
  - src/chat.py:45   system="You are a helpful assistant..."
Externalized prompt files: none (or e.g. "prompts/agents.yaml — CrewAI role/goal/backstory")
Prompt-template registries: none (or e.g. langchain.hub.pull("rlm/rag-prompt") at app.py:14)
Coverage totals: 3 hardcoded code targets · 0 externalized prompt files · 0 registry pulls
Proposed plan: single config key `chat-assistant`, mirror fallback, Stage 3 (tools) skipped (no function calling), Stage 4 (tracking) inline, Stage 5 (evals) attach built-in accuracy judge.

STOP. Present this summary, state the coverage totals out loud (e.g. "I found N hardcoded code targets and M externalized prompt files — does that match what you expected?"), and wait for the user to reply with one of four explicit forms:

• confirm — proceed to Stage 2.
• add: <files or paths> — re-run the audit with the new locations and present an updated summary.
• fix: <correction> — update a target in the list (provider, mode, prompt content, etc.) and ask again.
• stop — pause the migration here.

Do not interpret any other word — including skip, next, go, ok, proceed — as confirmation; ask the user to pick one of the four forms. This is the most important checkpoint in the workflow — if the audit is wrong, every stage after this will be wrong. The user should cross-check the hardcoded-targets list against what they know is in the code before giving the go-ahead.

Step 2: Wrap the call in the AI SDK (Stage 2)

This is the first stage that writes code. It has nine sub-steps.

1. Delete any hand-rolled model / tool wrappers the audit flagged. Do this before installing the new SDK so the replacement lands in a repo without confusing fallback imports. The two shapes the Stage 1 audit should have surfaced:

   • load_chat_model(f"{provider}/{name}") or any init_chat_model(...) wrapper. Ships with langchain-ai/react-agent and many derivative repos. Delete the function and its module; the replacement is create_langchain_model(ai_config) (installed in the next sub-step). Leaving the wrapper in place means the next edit in this repo will import the familiar helper and silently drop variation parameters.
   • Hand-rolled resolve_tools / TOOL_REGISTRY / ALL_TOOLS helpers that hard-code a static tool list. Delete them; ldai_langchain.langchain_helper.build_structured_tools(ai_config, TOOL_REGISTRY_DICT) is the canonical replacement and gets wired in Stage 3. If you leave the hand-rolled version, both shapes will live side-by-side and the next contributor will pick the familiar one.

   Commit the deletion separately from the SDK install if the repo's review process benefits from it — otherwise bundle with sub-step 2.

2. Install the AI SDK. Detect the package manager from Step 1, then install:

   • Python: launchdarkly-server-sdk + launchdarkly-server-sdk-ai>=0.20.0
   • Node.js/TypeScript: @launchdarkly/node-server-sdk + @launchdarkly/server-sdk-ai@^0.20.0
   • Go: github.com/launchdarkly/go-server-sdk/v7 + github.com/launchdarkly/go-server-sdk/ldai

   Tier-2 provider packages (install in Stage 4, only if you're using the matching provider):

   • OpenAI: launchdarkly-server-sdk-ai-openai>=0.4.0 (Python) / @launchdarkly/server-sdk-ai-openai@^0.5.5 (Node)
   • LangChain / LangGraph: launchdarkly-server-sdk-ai-langchain>=0.5.0 (Python) / @launchdarkly/server-sdk-ai-langchain@^0.5.5 (Node)
   • Vercel AI SDK (Node only): @launchdarkly/server-sdk-ai-vercel@^0.5.5
   • Anthropic, Gemini, Bedrock — no provider package published; use Tier-3 custom extractor (see built-in-metrics)

3. Initialize LDAIClient once at startup. Reuse any existing LDClient — do not create a second base client. Place the initialization in the same module that owns existing app config.

   Python:

   import os
   import ldclient
   from ldclient.config import Config
   from ldai.client import LDAIClient
   
   # Order matters: ldclient.get() raises if called before ldclient.set_config().
   # The set_config call is what initializes the singleton; .get() just returns it.
   sdk_key = os.environ.get("LD_SDK_KEY")
   if sdk_key:
       ldclient.set_config(Config(sdk_key))
   else:
       # Missing key: init in offline mode so the app still starts and the fallback
       # path runs on every call. Never raise at import time for a missing env var —
       # that turns a config gap into a boot failure.
       import logging
       logging.getLogger(__name__).warning(
           "LD_SDK_KEY not set; configs will use fallback values only."
       )
       ldclient.set_config(Config("", offline=True))
   
   ai_client = LDAIClient(ldclient.get())
   

   Node.js/TypeScript:

   import { init } from '@launchdarkly/node-server-sdk';
   import { initAi } from '@launchdarkly/server-sdk-ai';
   
   // The Node SDK does not have an explicit offline mode — a missing or invalid
   // key fails fast during waitForInitialization, and every agent_config /
   // completion_config call returns the fallback. Log a warning; do not throw.
   if (!process.env.LD_SDK_KEY) {
     console.warn('LD_SDK_KEY not set; configs will use fallback values only.');
   }
   const ldClient = init(process.env.LD_SDK_KEY ?? 'sdk-offline');
   await ldClient.waitForInitialization({ timeout: 10 }).catch(() => {
     // Swallow init failures in offline mode; fallback path runs.
   });
   const aiClient = initAi(ldClient);
   

4. Hand off to configs-create. Print the extracted model, prompt/instructions, parameters, and mode from the Stage 1 manifest, then tell the user: "Run /configs-create with these inputs, then come back here." Supply the config key you want the code to call (e.g. chat-assistant). Do not attempt to auto-invoke the sibling skill — wait for the user to finish it before continuing.

   After configs-create finishes, the user must also run /configs-targeting to promote the new variation to fallthrough. A freshly created variation returns enabled=False to every consumer until targeting is updated. Skip this and Stage 2 verification (sub-step 9 below) will silently take the fallback path on every request.

5. Rewrite template placeholders to Mustache syntax. If the hardcoded prompt interpolates runtime values with Python .format(), f-strings, JS template literals, or any other non-Mustache syntax (e.g. {system_time}, ${userName}, %(topic)s), rewrite every placeholder to {{ variable }} Mustache form. Do this in both the file you're about to send to /configs-create and the fallback string you'll write in sub-step 6. The AI SDK interpolates variables through a Mustache renderer on the LD-served path and the fallback path using the fourth-argument variables dict to completion_config(...) / completionConfig(...). Leaving a Python-style {system_time} literal in the fallback ships a silent regression when LaunchDarkly is unreachable — the renderer won't match the single-brace form and the literal {system_time} goes to the provider as part of the prompt.

   Before:

   SYSTEM_PROMPT = "You are a helpful assistant. The time is {system_time}."
   prompt = SYSTEM_PROMPT.format(system_time=datetime.now().isoformat())
   

   After (in source):

   SYSTEM_PROMPT = "You are a helpful assistant. The time is {{ system_time }}."
   # .format() is removed at the call site — the SDK interpolates via `variables`
   config = ai_client.completion_config(
       CONFIG_KEY,
       context,
       fallback,
       variables={"system_time": datetime.now().isoformat()},
   )
   

   Common shapes to rewrite:

   • Python "{var}" / "{var!s}" / "%(var)s" → "{{ var }}"
   • JS/TS `${var}` template literals inside prompt strings → "{{ var }}"
   • Any hand-rolled str.replace("__VAR__", value) scheme → "{{ var }}"

   See fallback-defaults-pattern.md § Template placeholders for the fallback-specific variant.

6. Build the fallback. Mirror the hardcoded values you extracted. Use AICompletionConfigDefault / AIAgentConfigDefault in Python, plain object literals in Node. See fallback-defaults-pattern.md for inline, file-backed, and bootstrap-generated patterns.

   Python fallback (completion mode):

   from ldai.client import AICompletionConfigDefault, ModelConfig, ProviderConfig, LDMessage
   
   fallback = AICompletionConfigDefault(
       enabled=True,
       model=ModelConfig(name="gpt-4o", parameters={"temperature": 0.7, "max_tokens": 2000}),
       provider=ProviderConfig(name="openai"),
       messages=[LDMessage(role="system", content="You are a helpful assistant...")],
   )
   

7. Replace the hardcoded call site. Swap the hardcoded model/prompt/params for a completion_config / completionConfig (or agent_config / agentConfig) call, then read the returned fields into the existing provider call. Keep the provider call intact.

   Python — before:

   response = openai_client.chat.completions.create(
       model="gpt-4o",
       temperature=0.7,
       max_tokens=2000,
       messages=[
           {"role": "system", "content": "You are a helpful assistant..."},
           {"role": "user", "content": user_input},
       ],
   )
   

   Python — after:

   context = Context.builder(user_id).set("email", user.email).build()
   config = ai_client.completion_config("chat-assistant", context, fallback)
   
   if not config.enabled:
       return disabled_response()
   
   params = config.model.parameters or {}
   response = openai_client.chat.completions.create(
       model=config.model.name,
       temperature=params.get("temperature"),
       max_tokens=params.get("max_tokens"),
       messages=[m.to_dict() for m in (config.messages or [])] + [
           {"role": "user", "content": user_input},
       ],
   )
   

   Python — after (agent mode) — for LangGraph, CrewAI, or any framework that takes a goal/instructions string:

   context = Context.builder(user_id).kind("user").build()
   config = ai_client.agent_config("support-agent", context, FALLBACK)
   
   if not config.enabled:
       return disabled_response()
   
   # config is a single AIAgentConfig object — NOT a (config, tracker) tuple.
   # Obtain the tracker once per execution via the factory: tracker = config.create_tracker()
   model_name = f"{config.provider.name}/{config.model.name}"
   instructions = config.instructions
   params = config.model.parameters or {}
   
   # Pass model_name + instructions into your framework's agent constructor.
   # Example: LangGraph prebuilt agent (Python — `from langchain.agents import create_agent`;
   # this replaces `langgraph.prebuilt.create_react_agent`, deprecated in LangGraph 1.0
   # and removed in 2.0. Same return shape; `prompt=` was renamed to `system_prompt=`.)
   # agent = create_agent(
   #     create_langchain_model(config),  # forwards every variation parameter
   #     TOOLS,                            # Stage 3 will replace this with a config.tools loader
   #     system_prompt=instructions,
   # )
   

   See before-after-examples.md for full Python OpenAI, Node Anthropic, and LangGraph agent-mode paired snippets.