Google Gemini API - Complete Guide

Version: Phase 2 Complete + Gemini 3 ✅ Package: @google/genai@1.27.0 (⚠️ NOT @google/generative-ai) Last Updated: 2025-11-19 (Gemini 3 preview release)

---

⚠️ CRITICAL SDK MIGRATION WARNING

DEPRECATED SDK: @google/generative-ai (sunset November 30, 2025) CURRENT SDK: @google/genai v1.27+

If you see code using @google/generative-ai, it's outdated!

This skill uses the correct current SDK and provides a complete migration guide.

---

Status

✅ Phase 1 Complete:

• ✅ Text Generation (basic + streaming)
• ✅ Multimodal Inputs (images, video, audio, PDFs)
• ✅ Function Calling (basic + parallel execution)
• ✅ System Instructions & Multi-turn Chat
• ✅ Thinking Mode Configuration
• ✅ Generation Parameters (temperature, top-p, top-k, stop sequences)
• ✅ Both Node.js SDK (@google/genai) and fetch approaches

✅ Phase 2 Complete:

• ✅ Context Caching (cost optimization with TTL-based caching)
• ✅ Code Execution (built-in Python interpreter and sandbox)
• ✅ Grounding with Google Search (real-time web information + citations)

📦 Separate Skills:

• Embeddings: See google-gemini-embeddings skill for text-embedding-004

---

Table of Contents

Phase 1 - Core Features:

1. Quick Start
2. Current Models (2025)
3. SDK vs Fetch Approaches
4. Text Generation
5. Streaming
6. Multimodal Inputs
7. Function Calling
8. System Instructions
9. Multi-turn Chat
10. Thinking Mode
11. Generation Configuration

Phase 2 - Advanced Features: 12. Context Caching 13. Code Execution 14. Grounding with Google Search

Common Reference: 15. Error Handling 16. Rate Limits 17. SDK Migration Guide 18. Production Best Practices

---

Quick Start

Installation

CORRECT SDK:

npm install @google/genai@1.27.0

❌ WRONG (DEPRECATED):

npm install @google/generative-ai  # DO NOT USE!

Environment Setup

export GEMINI_API_KEY="..."

Or create .env file:

GEMINI_API_KEY=...

First Text Generation (Node.js SDK)

import { GoogleGenAI } from '@google/genai';

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const response = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: 'Explain quantum computing in simple terms'
});

console.log(response.text);

First Text Generation (Fetch - Cloudflare Workers)

const response = await fetch(
  `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent`,
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-goog-api-key': env.GEMINI_API_KEY,
    },
    body: JSON.stringify({
      contents: [{ parts: [{ text: 'Explain quantum computing in simple terms' }] }]
    }),
  }
);

const data = await response.json();
console.log(data.candidates[0].content.parts[0].text);

---

Current Models (2025)

Gemini 3 Series (Preview - November 2025)

gemini-3-pro-preview

• Context: TBD (documentation pending)
• Status: 🆕 Preview release (November 18, 2025)
• Description: Google's newest and most intelligent AI model with state-of-the-art reasoning
• Best for: Most complex reasoning tasks, advanced multimodal understanding, benchmark-critical applications
• Features: Enhanced multimodal (text, image, video, audio, PDF), function calling, streaming
• Benchmark Performance: Outperforms Gemini 2.5 Pro on every major AI benchmark
• ⚠️ Preview: Use for evaluation. Consider gemini-2.5-pro for production until stable release

Gemini 2.5 Series (General Availability - Stable)

gemini-2.5-pro

• Context: 1,048,576 input tokens / 65,536 output tokens
• Description: State-of-the-art thinking model for complex reasoning
• Best for: Code, math, STEM, complex problem-solving
• Features: Thinking mode (default on), function calling, multimodal, streaming
• Knowledge cutoff: January 2025

gemini-2.5-flash

• Context: 1,048,576 input tokens / 65,536 output tokens
• Description: Best price-performance workhorse model
• Best for: Large-scale processing, low-latency, high-volume, agentic use cases
• Features: Thinking mode (default on), function calling, multimodal, streaming
• Knowledge cutoff: January 2025

gemini-2.5-flash-lite

• Context: 1,048,576 input tokens / 65,536 output tokens
• Description: Cost-optimized, fastest 2.5 model
• Best for: High throughput, cost-sensitive applications
• Features: Thinking mode (default on), function calling, multimodal, streaming
• Knowledge cutoff: January 2025

Model Feature Matrix

Feature	3-Pro (Preview)	2.5-Pro	2.5-Flash	2.5-Flash-Lite
Thinking Mode	TBD	✅ Default ON	✅ Default ON	✅ Default ON
Function Calling	✅	✅	✅	✅
Multimodal	✅ Enhanced	✅	✅	✅
Streaming	✅	✅	✅	✅
System Instructions	✅	✅	✅	✅
Context Window	TBD	1,048,576 in	1,048,576 in	1,048,576 in
Output Tokens	TBD	65,536 max	65,536 max	65,536 max
Status	Preview	Stable	Stable	Stable

⚠️ Context Window Correction

ACCURATE (Gemini 2.5): Gemini 2.5 models support 1,048,576 input tokens (NOT 2M!) OUTDATED: Only Gemini 1.5 Pro (previous generation) had 2M token context window GEMINI 3: Context window specifications pending official documentation

Common mistake: Claiming Gemini 2.5 has 2M tokens. It doesn't. This skill prevents this error.

---

SDK vs Fetch Approaches

Node.js SDK (@google/genai)

Pros:

• Type-safe with TypeScript
• Easier API (simpler syntax)
• Built-in chat helpers
• Automatic SSE parsing for streaming
• Better error handling

Cons:

• Requires Node.js or compatible runtime
• Larger bundle size
• May not work in all edge runtimes

Use when: Building Node.js apps, Next.js Server Actions/Components, or any environment with Node.js compatibility

Fetch-based (Direct REST API)

Pros:

• Works in any JavaScript environment (Cloudflare Workers, Deno, Bun, browsers)
• Minimal dependencies
• Smaller bundle size
• Full control over requests

Cons:

• More verbose syntax
• Manual SSE parsing for streaming
• No built-in chat helpers
• Manual error handling

Use when: Deploying to Cloudflare Workers, browser clients, or lightweight edge runtimes

---

Text Generation

Basic Text Generation (SDK)

import { GoogleGenAI } from '@google/genai';

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const response = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: 'Write a haiku about artificial intelligence'
});

console.log(response.text);

Basic Text Generation (Fetch)

const response = await fetch(
  `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent`,
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-goog-api-key': env.GEMINI_API_KEY,
    },
    body: JSON.stringify({
      contents: [
        {
          parts: [
            { text: 'Write a haiku about artificial intelligence' }
          ]
        }
      ]
    }),
  }
);

const data = await response.json();
console.log(data.candidates[0].content.parts[0].text);

Response Structure

{
  text: string,                  // Convenience accessor for text content
  candidates: [
    {
      content: {
        parts: [
          { text: string }       // Generated text
        ],
        role: string             // "model"
      },
      finishReason: string,      // "STOP" | "MAX_TOKENS" | "SAFETY" | "OTHER"
      index: number
    }
  ],
  usageMetadata: {
    promptTokenCount: number,
    candidatesTokenCount: number,
    totalTokenCount: number
  }
}

---

Streaming

Streaming with SDK (Async Iteration)

const response = await ai.models.generateContentStream({
  model: 'gemini-2.5-flash',
  contents: 'Write a 200-word story about time travel'
});

for await (const chunk of response) {
  process.stdout.write(chunk.text);
}

Streaming with Fetch (SSE Parsing)

const response = await fetch(
  `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:streamGenerateContent`,
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-goog-api-key': env.GEMINI_API_KEY,
    },
    body: JSON.stringify({
      contents: [{ parts: [{ text: 'Write a 200-word story about time travel' }] }]
    }),
  }
);

const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = '';

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split('\n');
  buffer = lines.pop() || '';

  for (const line of lines) {
    if (line.trim() === '' || line.startsWith('data: [DONE]')) continue;
    if (!line.startsWith('data: ')) continue;

    try {
      const data = JSON.parse(line.slice(6));
      const text = data.candidates[0]?.content?.parts[0]?.text;
      if (text) {
        process.stdout.write(text);
      }
    } catch (e) {
      // Skip invalid JSON
    }
  }
}

Key Points:

• Use streamGenerateContent endpoint (not generateContent)
• Parse Server-Sent Events (SSE) format: data: {json}\n\n
• Handle incomplete chunks in buffer
• Skip empty lines and [DONE] markers

---

Multimodal Inputs

Gemini 2.5 models support text + images + video + audio + PDFs in the same request.

Images (Vision)

SDK Approach

import { GoogleGenAI } from '@google/genai';
import fs from 'fs';

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

// From file
const imageData = fs.readFileSync('/path/to/image.jpg');
const base64Image = imageData.toString('base64');

const response = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: [
    {
      parts: [
        { text: 'What is in this image?' },
        {
          inlineData: {
            data: base64Image,
            mimeType: 'image/jpeg'
          }
        }
      ]
    }
  ]
});

console.log(response.text);

Fetch Approach

const imageData = fs.readFileSync('/path/to/image.jpg');
const base64Image = imageData.toString('base64');

const response = await fetch(
  `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent`,
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-goog-api-key': env.GEMINI_API_KEY,
    },
    body: JSON.stringify({
      contents: [
        {
          parts: [
            { text: 'What is in this image?' },
            {
              inlineData: {
                data: base64Image,
                mimeType: 'image/jpeg'
              }
            }
          ]
        }
      ]
    }),
  }
);

const data = await response.json();
console.log(data.candidates[0].content.parts[0].text);

Supported Image Formats:

• JPEG (.jpg, .jpeg)
• PNG (.png)
• WebP (.webp)
• HEIC (.heic)
• HEIF (.heif)

Max Image Size: 20MB per image

Video

// Video must be < 2 minutes for inline data
const videoData = fs.readFileSync('/path/to/video.mp4');
const base64Video = videoData.toString('base64');

const response = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: [
    {
      parts: [
        { text: 'Describe what happens in this video' },
        {
          inlineData: {
            data: base64Video,
            mimeType: 'video/mp4'
          }
        }
      ]
    }
  ]
});

console.log(response.text);

Supported Video Formats:

• MP4 (.mp4)
• MPEG (.mpeg)
• MOV (.mov)
• AVI (.avi)
• FLV (.flv)
• MPG (.mpg)
• WebM (.webm)
• WMV (.wmv)

Max Video Length (inline): 2 minutes Max Video Size: 2GB (use File API for larger files - Phase 2)

Audio

const audioData = fs.readFileSync('/path/to/audio.mp3');
const base64Audio = audioData.toString('base64');

const response = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: [
    {
      parts: [
        { text: 'Transcribe and summarize this audio' },
        {
          inlineData: {
            data: base64Audio,
            mimeType: 'audio/mp3'
          }
        }
      ]
    }
  ]
});

console.log(response.text);

Supported Audio Formats:

• MP3 (.mp3)
• WAV (.wav)
• FLAC (.flac)
• AAC (.aac)
• OGG (.ogg)
• OPUS (.opus)

Max Audio Size: 20MB

PDFs

const pdfData = fs.readFileSync('/path/to/document.pdf');
const base64Pdf = pdfData.toString('base64');

const response = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: [
    {
      parts: [
        { text: 'Summarize the key points in this PDF' },
        {
          inlineData: {
            data: base64Pdf,
            mimeType: 'application/pdf'
          }
        }
      ]
    }
  ]
});

console.log(response.text);

Max PDF Size: 30MB PDF Limitations: Text-based PDFs work best; scanned images may have lower accuracy

Multiple Inputs

You can combine multiple modalities in one request:

const response = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: [
    {
      parts: [
        { text: 'Compare these two images and describe the differences:' },
        { inlineData: { data: base64Image1, mimeType: 'image/jpeg' } },
        { inlineData: { data: base64Image2, mimeType: 'image/jpeg' } }
      ]
    }
  ]
});

---

Function Calling

Gemini supports function calling (tool use) to connect models with external APIs and systems.

Basic Function Calling (SDK)

import { GoogleGenAI, FunctionCallingConfigMode } from '@google/genai';

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

// Define function declarations
const getCurrentWeather = {
  name: 'get_current_weather',
  description: 'Get the current weather for a location',
  parametersJsonSchema: {
    type: 'object',
    properties: {
      location: {
        type: 'string',
        description: 'City name, e.g. San Francisco'
      },
      unit: {
        type: 'string',
        enum: ['celsius', 'fahrenheit']
      }
    },
    required: ['location']
  }
};

// Make request with tools
const response = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: 'What\'s the weather in Tokyo?',
  config: {
    tools: [
      { functionDeclarations: [getCurrentWeather] }
    ]
  }
});

// Check if model wants to call a function
const functionCall = response.candidates[0].content.parts[0].functionCall;

if (functionCall) {
  console.log('Function to call:', functionCall.name);
  console.log('Arguments:', functionCall.args);

  // Execute the function (your implementation)
  const weatherData = await fetchWeather(functionCall.args.location);

  // Send function result back to model
  const finalResponse = await ai.models.generateContent({
    model: 'gemini-2.5-flash',
    contents: [
      'What\'s the weather in Tokyo?',
      response.candidates[0].content, // Original assistant response with function call
      {
        parts: [
          {
            functionResponse: {
              name: functionCall.name,
              response: weatherData
            }
          }
        ]
      }
    ],
    config: {
      tools: [
        { functionDeclarations: [getCurrentWeather] }
      ]
    }
  });

  console.log(finalResponse.text);
}

Function Calling (Fetch)

const response = await fetch(
  `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent`,
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-goog-api-key': env.GEMINI_API_KEY,
    },
    body: JSON.stringify({
      contents: [
        { parts: [{ text: 'What\'s the weather in Tokyo?' }] }
      ],
      tools: [
        {
          functionDeclarations: [
            {
              name: 'get_current_weather',
              description: 'Get the current weather for a location',
              parameters: {
                type: 'object',
                properties: {
                  location: {
                    type: 'string',
                    description: 'City name'
                  }
                },
                required: ['location']
              }
            }
          ]
        }
      ]
    }),
  }
);

const data = await response.json();
const functionCall = data.candidates[0]?.content?.parts[0]?.functionCall;

if (functionCall) {
  // Execute function and send result back (same flow as SDK)
}

Parallel Function Calling

Gemini can call multiple independent functions simultaneously:

const tools = [
  {
    functionDeclarations: [
      {
        name: 'get_weather',
        description: 'Get weather for a location',
        parametersJsonSchema: {
          type: 'object',
          properties: {
            location: { type: 'string' }
          },
          required: ['location']
        }
      },
      {
        name: 'get_population',
        description: 'Get population of a city',
        parametersJsonSchema: {
          type: 'object',
          properties: {
            city: { type: 'string' }
          },
          required: ['city']
        }
      }
    ]
  }
];

const response = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: 'What is the weather and population of Tokyo?',
  config: { tools }
});

// Model may return MULTIPLE function calls in parallel
const functionCalls = response.candidates[0].content.parts.filter(
  part => part.functionCall
);

console.log(`Model wants to call ${functionCalls.length} functions in parallel`);

Function Calling Modes

import { FunctionCallingConfigMode } from '@google/genai';

const response = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: 'What\'s the weather?',
  config: {
    tools: [{ functionDeclarations: [getCurrentWeather] }],
    toolConfig: {
      functionCallingConfig: {
        mode: FunctionCallingConfigMode.ANY, // Force function call
        // mode: FunctionCallingConfigMode.AUTO, // Model decides (default)
        // mode: FunctionCallingConfigMode.NONE, // Never call functions
        allowedFunctionNames: ['get_current_weather'] // Optional: restrict to specific functions
      }
    }
  }
});

Modes:

• AUTO (default): Model decides whether to call functions
• ANY: Force model to call at least one function
• NONE: Disable function calling for this request

---

System Instructions

System instructions guide the model's behavior and set context. They are separate from the conversation messages.

SDK Approach

const response = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  systemInstruction: 'You are a helpful AI assistant that always responds in the style of a pirate. Use nautical terminology and end sentences with "arrr".',
  contents: 'Explain what a database is'
});

console.log(response.text);
// Output: "Ahoy there! A database be like a treasure chest..."

Fetch Approach

const response = await fetch(
  `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent`,
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-goog-api-key': env.GEMINI_API_KEY,
    },
    body: JSON.stringify({
      systemInstruction: {
        parts: [
          { text: 'You are a helpful AI assistant that always responds in the style of a pirate.' }
        ]
      },
      contents: [
        { parts: [{ text: 'Explain what a database is' }] }
      ]
    }),
  }
);

Key Points:

• System instructions are NOT part of contents array
• They are set once at the top level of the request
• They persist for the entire conversation (when using multi-turn chat)
• They don't count as user or model messages

---

Multi-turn Chat

For conversations with history, use the SDK's chat helpers or manually manage conversation state.

SDK Chat Helpers (Recommended)

const chat = await ai.models.createChat({
  model: 'gemini-2.5-flash',
  systemInstruction: 'You are a helpful coding assistant.',
  history: [] // Start empty or with previous messages
});

// Send first message
const response1 = await chat.sendMessage('What is TypeScript?');
console.log('Assistant:', response1.text);

// Send follow-up (context is automatically maintained)
const response2 = await chat.sendMessage('How do I install it?');
console.log('Assistant:', response2.text);

// Get full chat history
const history = chat.getHistory();
console.log('Full conversation:', history);

Manual Chat Management (Fetch)

const conversationHistory = [];

// First turn
const response1 = await fetch(
  `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent`,
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-goog-api-key': env.GEMINI_API_KEY,
    },
    body: JSON.stringify({
      contents: [
        {
          role: 'user',
          parts: [{ text: 'What is TypeScript?' }]
        }
      ]
    }),
  }
);

const data1 = await response1.json();
const assistantReply1 = data1.candidates[0].content.parts[0].text;

// Add to history
conversationHistory.push(
  { role: 'user', parts: [{ text: 'What is TypeScript?' }] },
  { role: 'model', parts: [{ text: assistantReply1 }] }
);

// Second turn (include full history)
const response2 = await fetch(
  `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent`,
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-goog-api-key': env.GEMINI_API_KEY,
    },
    body: JSON.stringify({
      contents: [
        ...conversationHistory,
        { role: 'user', parts: [{ text: 'How do I install it?' }] }
      ]
    }),
  }
);

Message Roles:

• user: User messages
• model: Assistant responses

⚠️ Important: Chat helpers are SDK-only. With fetch, you must manually manage conversation history.

---

Thinking Mode

Gemini 2.5 models have thinking mode enabled by default for enhanced quality. You can configure the thinking budget.

Configure Thinking Budget (SDK)

const response = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: 'Solve this complex math problem: ...',
  config: {
    thinkingConfig: {
      thinkingBudget: 8192 // Max tokens for thinking (default: model-dependent)
    }
  }
});

Configure Thinking Budget (Fetch)

const response = await fetch(
  `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent`,
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-goog-api-key': env.GEMINI_API_KEY,
    },
    body: JSON.stringify({
      contents: [{ parts: [{ text: 'Solve this complex math problem: ...' }] }],
      generationConfig: {
        thinkingConfig: {
          thinkingBudget: 8192
        }
      }
    }),
  }
);