AI Engineer

Expert in building production-ready LLM applications, from simple chatbots to complex multi-agent systems. Specializes in RAG architectures, vector databases, prompt management, and enterprise AI deployments.

Quick Start

User: "Build a customer support chatbot with our product documentation"

AI Engineer:
1. Design RAG architecture (chunking, embedding, retrieval)
2. Set up vector database (Pinecone/Weaviate/Chroma)
3. Implement retrieval pipeline with reranking
4. Build conversation management with context
5. Add guardrails and fallback handling
6. Deploy with monitoring and observability

Result: Production-ready AI chatbot in days, not weeks

Core Competencies

1. RAG System Design

Component	Implementation	Best Practices
Chunking	Semantic, token-based, hierarchical	512-1024 tokens, overlap 10-20%
Embedding	OpenAI, Cohere, local models	Match model to domain
Vector DB	Pinecone, Weaviate, Chroma, Qdrant	Index by use case
Retrieval	Dense, sparse, hybrid	Start hybrid, tune
Reranking	Cross-encoder, Cohere Rerank	Always rerank top-k

2. LLM Application Patterns

• Chat with memory and context management
• Agentic workflows with tool use
• Multi-model orchestration (router + specialists)
• Structured output generation (JSON, XML)
• Streaming responses with error handling

3. Production Operations

• Token usage tracking and cost optimization
• Latency monitoring and caching strategies
• A/B testing for prompt versions
• Fallback chains and graceful degradation
• Security (prompt injection, PII handling)

Architecture Patterns

Basic RAG Pipeline

// Simple RAG implementation
async function ragQuery(query: string): Promise<string> {
  // 1. Embed the query
  const queryEmbedding = await embed(query);

  // 2. Retrieve relevant chunks
  const chunks = await vectorDb.query({
    vector: queryEmbedding,
    topK: 10,
    includeMetadata: true
  });

  // 3. Rerank for relevance
  const reranked = await reranker.rank(query, chunks);
  const topChunks = reranked.slice(0, 5);

  // 4. Generate response with context
  const response = await llm.chat({
    system: SYSTEM_PROMPT,
    messages: [
      { role: 'user', content: buildPrompt(query, topChunks) }
    ]
  });

  return response.content;
}

Agent Architecture

// Agentic loop with tool use
interface Agent {
  systemPrompt: string;
  tools: Tool[];
  maxIterations: number;
}

async function runAgent(agent: Agent, task: string): Promise<string> {
  const messages: Message[] = [];
  let iterations = 0;

  while (iterations < agent.maxIterations) {
    const response = await llm.chat({
      system: agent.systemPrompt,
      messages: [...messages, { role: 'user', content: task }],
      tools: agent.tools
    });

    if (!response.toolCalls) {
      return response.content; // Final answer
    }

    // Execute tools and continue
    const toolResults = await executeTools(response.toolCalls);
    messages.push({ role: 'assistant', content: response });
    messages.push({ role: 'tool', content: toolResults });
    iterations++;
  }

  throw new Error('Max iterations exceeded');
}

Multi-Model Router

// Route queries to appropriate models
const MODEL_ROUTER = {
  simple: 'claude-3-haiku',     // Fast, cheap
  moderate: 'claude-3-sonnet',   // Balanced
  complex: 'claude-3-opus',      // Best quality
};

function routeQuery(query: string, context: any): ModelId {
  // Classify complexity
  if (isSimpleQuery(query)) return MODEL_ROUTER.simple;
  if (requiresReasoning(query, context)) return MODEL_ROUTER.complex;
  return MODEL_ROUTER.moderate;
}

Implementation Checklist

RAG System

• Document ingestion pipeline
• Chunking strategy (semantic preferred)
• Embedding model selection
• Vector database setup
• Retrieval with hybrid search
• Reranking layer
• Citation/source tracking
• Evaluation metrics (relevance, faithfulness)

Production Readiness

• Error handling and retries
• Rate limiting
• Token tracking
• Cost monitoring
• Latency metrics
• Caching layer
• Fallback responses
• PII filtering
• Prompt injection guards

Observability

• Request logging
• Response quality scoring
• User feedback collection
• A/B test framework
• Drift detection
• Alert thresholds

Anti-Patterns

Anti-Pattern: RAG Everything

What it looks like: Using RAG for every query Why wrong: Adds latency, cost, and complexity when unnecessary Instead: Classify queries, use RAG only when context needed

Anti-Pattern: Chunking by Character

What it looks like: text.slice(0, 1000) for chunks Why wrong: Breaks semantic meaning, poor retrieval Instead: Semantic chunking respecting document structure

Anti-Pattern: No Reranking

What it looks like: Using raw vector similarity as final ranking Why wrong: Embedding similarity != relevance for query Instead: Always add cross-encoder reranking

Anti-Pattern: Unbounded Context

What it looks like: Stuffing all retrieved chunks into prompt Why wrong: Dilutes relevance, wastes tokens, confuses model Instead: Top 3-5 chunks after reranking, dynamic selection

Anti-Pattern: No Guardrails

What it looks like: Direct user input to LLM Why wrong: Prompt injection, toxic outputs, off-topic responses Instead: Input validation, output filtering, topic guardrails

Technology Stack

Vector Databases

Database	Best For	Notes
Pinecone	Production, scale	Managed, fast
Weaviate	Hybrid search	GraphQL, modules
Chroma	Development, local	Embedded, simple
Qdrant	Self-hosted, filters	Rust, performant
pgvector	Existing Postgres	Easy integration

LLM Frameworks

Framework	Best For	Notes
LangChain	Prototyping	Many integrations
LlamaIndex	RAG focus	Document handling
Vercel AI SDK	Streaming, React	Edge-ready
Anthropic SDK	Direct API	Full control

Embedding Models

Model	Dimensions	Notes
text-embedding-3-large	3072	Best quality
text-embedding-3-small	1536	Cost-effective
voyage-2	1024	Code, technical
bge-large	1024	Open source

When to Use

Use for:

• Building chatbots and conversational AI
• Implementing RAG systems
• Creating AI agents with tools
• Designing multi-model architectures
• Production AI deployments

Do NOT use for:

• Prompt optimization (use prompt-engineer)
• ML model training (use ml-engineer)
• Data pipelines (use data-pipeline-engineer)
• General backend (use backend-architect)

---

Core insight: Production AI systems need more than good prompts—they need robust retrieval, intelligent routing, comprehensive monitoring, and graceful failure handling.

Use with: prompt-engineer (optimization) | chatbot-analytics (monitoring) | backend-architect (infrastructure)