google-surf-mcp

English | Korean

Web searches, papers, and GitHub repositories are stored as PKM, ontology, and lineage. The view above is generated with project_memory(action="export", export_format="html", export_view="graph", all_projects=true).

"Turn Google Search, Papers, and Codebases into an Automatic Local Knowledge Graph and lineage for AI Agents with Zero API Key, Zero External Server."

Google Surf stores search and extraction results in a project-scoped local knowledge graph.

As you search, papers, code, web sources, session intent, plans, experiments, and decisions accumulate in a personal PKM. New research searches stored knowledge and fresh web results together, reducing repeated work while continuing to discover new information.

Projects remain isolated by default. Only verifiable links such as matching DOIs, repository URLs, or explicit aliases are added, so knowledge from one project can be reused in another without merging the original records.

Retrieval runs exact search, BM25, vector search, code and graph search, and live web independently, then combines them with RRF and one shared reranker.

Live web + Papers + Codebases + Project memory
                        ↓
Exact + BM25 + Vector + Code graph + Graph PPR
                        ↓
              RRF + Shared reranker
                        ↓
        Results with evidence and provenance

Seven tools are available by default: search / search_parallel / extract / scholar_search / project_memory_search / project_memory / health.

Research mode and automatic capture are enabled by default. Set SURF_RESEARCH=false to use search and extraction without opening the database or graph sidecar; the project memory tools are not registered in that mode.

With research mode disabled, live search and search_parallel results still use a lightweight in-memory reranker. It fuses the provider order with query BM25 rank through RRF without loading the vector model or opening local storage.

Browser search needs no API key. SearchApi can be configured as an optional primary provider or fallback.

Core features

• Web, paper, and codebase search: Google web search is the default, while Scholar is used for paper-specific metadata. SearchApi can act as an optional primary provider or fallback.
• Web and academic document extraction: HTML and PDF extraction returns available titles, authors, DOIs, publication metadata, and body text. search and search_parallel can include abstracts or full bodies.
• Automatic project memory: Search results, extracted bodies, and code repositories are stored in the current project. Unread results remain metadata; extracted content becomes active RAG evidence.
• Structured codebase search: Tree-sitter links files, symbols, imports, and calls from local projects and relevant GitHub repositories. Exact, BM25, vector, and graph search retrieve the code.
• Graph hybrid retrieval: Fresh web results, papers, stored content, codebases, and the project graph are searched independently. Exact, BM25, vector, and PPR candidates are combined through RRF and one shared reranker.
• Ontology and data lineage: Web sources, papers, code, plans, experiments, and decisions become typed entities and relations. Evidence paths run from source through documents, chunks, symbols, evidence, and assertions.
• Cross-project knowledge reuse: Projects remain isolated but selected projects can be searched together. Only verifiable links connect matching entities.
• Durable research history: Session intent, plans, experiments, failures, and decisions supplied by the MCP host are stored as revisions and linked to their supporting evidence.
• Local graph analysis and export: PageRank, PPR, connected components, and Louvain communities run without a separate Neo4j server. Results export to HTML, Graphviz, D3 JSON, and Neo4j import formats.

Search

• API-key-free system Chrome search
• Dedicated logged-out profile that never reads or copies the user's Chrome profile
• Multi-strategy SERP parsing with geometric verification
• Sponsored block and knowledge panel removal
• CAPTCHA detection and environment-specific recovery
• Parser self-healing and context fallback

Numbers

Search speed

	result
search	4.0-5.1s/query
scholar_search	3.8-5.6s/query

Live web search reranker check

32 held-out queries	nDCG@5	MRR	Precision@5
Provider order	0.8949	0.8203	0.6375
BM25 + RRF	0.8971	0.8203	0.6500

• web search responses with BM25+RRF reduce large retrieval payloads by up to 75%. Measured across three uncached queries per provider on a workstation with a 1Gb/s connection. Network and Google response time vary.

Tech Stack

• Runtime: Node.js, TypeScript, Model Context Protocol SDK
• Web search: System Chrome + CDP, Playwright compatibility fallback, SearchApi fallback
• Web extraction: Mozilla Readability, Turndown
• PDF extraction: LiteParse/PDFium, optional OCR, pdf-lib metadata parsing
• Code collection: Local project roots and gated GitHub sparse download
• Code parsing: Tree-sitter for files, symbols, imports, and call relations
• Code search: Exact lookup, BM25, Multilingual E5 vector search, and graph PPR
• Local database: Embedded SurrealDB on RocksDB
• Hybrid retrieval: Live web, papers, project memory, and codebase results combined through RRF
• Ranking: Reciprocal Rank Fusion and shared vector reranking
• Graph analysis: Graphology, PageRank, PPR, connected components, and Louvain communities
• Knowledge model: Versioned ontology, data lineage, cross-project schema and entity linking
• Recovery: CAPTCHA recovery, Playwright pool fallback, and deterministic parser self-healing

Install

Requires Node 20.18.1+. Browser mode also requires Google Chrome or Chromium.

npx google-surf-mcp   # actual MCP - register in client config

First tool call auto-bootstraps the warm profile (you may see Chrome open briefly).

Or local clone:

git clone https://github.com/HarimxChoi/google-surf-mcp
cd google-surf-mcp
npm install

If auto-bootstrap fails (rare), run it manually:

npm run bootstrap

Override paths if needed:

CHROME_PATH=/path/to/chrome SURF_TZ=America/New_York npm run bootstrap

Optional Codex output protection

Google Surf can install an opt-in Codex hook that ranks oversized Bash output before it reaches the model. The host still executes the command; the hook only applies stateless source-order, exact, and BM25 ranking with RRF. It opens neither the Google Surf database nor a browser and does not store command output.

npx -y google-surf-mcp@latest hooks install --host codex

Restart Codex, then open /hooks to review and trust the definitions. Shell output is reranked from 1,500 characters, with a default 1,500-character display budget that expands up to 3,000 for distinct query-matching evidence records or blocks. An explicit override may exceed 3,000; there is no cumulative per-turn output limit. Two identical or near-duplicate searches and explicit foreground polling loops are blocked. Existing write_stdin polls for an already-running unified command remain controlled by the Codex runtime, not this hook.

JSON/JSONL summaries use compact records with input-relative JSON pointers, ancestor identity/condition references and original result values. Markdown table rows retain headers and adjacent context, including source lines. Selected records and their required context must fit together; the renderer reports omissions instead of slicing records. Plain logs retain block reranking. Selection is deterministic and lexical, without a model, database or extra subprocess. Context extraction uses structural/field-name heuristics, not semantic understanding; unselected fields remain in the original input. These excerpts do not replace experiment logs or SSOT history. Input-relative pointers are not persistent retrieval handles. Existing installed hook bundles require an update to use changed code.

npx -y google-surf-mcp@latest hooks status --host codex
npx -y google-surf-mcp@latest hooks update --host codex
npx -y google-surf-mcp@latest hooks uninstall --host codex

Use with Claude Code

Paste this into your ~/.claude.json:

{
  "mcpServers": {
    "google-surf": {
      "command": "npx",
      "args": ["-y", "google-surf-mcp"]
    }
  }
}

Restart Claude Code. All seven tools, including project_memory_search and project_memory, are available by default.

For other MCP clients, use the same JSON shape in their config file.

Search providers

Browser search remains the default. SearchApi can be selected as the primary provider or used only when browser search fails.

value	behavior
browser	Default. Uses system Chrome with a dedicated logged-out profile, keeps native search windows hidden, and does not require SEARCH_API. Multiple MCP sessions share one local browser broker.
searchapi	Uses SearchApi as the primary provider and does not initialize Chrome for that tool.
fallback	Tries the current browser tier once, then uses SearchApi on browser errors, CAPTCHA/rate limits, profile failure, or parser degradation. It does not wait for human CAPTCHA recovery. Successful and normal empty browser responses are not repeated.

SURF_SEARCH_PROVIDER controls search and search_parallel. SURF_SCHOLAR_PROVIDER controls scholar_search. SearchApi modes require your own SearchApi account, key, and available credits.

SURF_BROWSER_ENGINE=auto selects native Chrome on a local desktop and the Playwright compatibility path in cloud or remote-debug mode. Native mode uses a normal hidden Chrome window, not headless Chrome. Set native or playwright to pin the engine.

{
  "mcpServers": {
    "google-surf": {
      "command": "npx",
      "args": ["-y", "google-surf-mcp"],
      "env": {
        "SEARCH_API": "your-searchapi-key",
        "SURF_SEARCH_PROVIDER": "fallback",
        "SURF_SCHOLAR_PROVIDER": "searchapi"
      }
    }
  }
}

Local clone variant:

{
  "mcpServers": {
    "google-surf": {
      "command": "node",
      "args": ["/abs/path/to/google-surf-mcp/build/index.js"]
    }
  }
}

Tools

• search(query, limit?, extract_mode?, extract_limit?, response_content?, max_chars?) - primary single-query tool for live discovery and reading. When new sources must be found and read, set extract_mode in this call instead of downloading PDFs, cloning repositories, or calling extract separately. Use extract only when the exact public URL is already known and no discovery is needed. With project_id, stored project knowledge is fused with live results, but the call never becomes local-only. limit is 1-20. Extraction defaults to none; extract_limit is 1-10 with default 5. response_content defaults to summary to bound one-call output.
• scholar_search(query, limit?) - Google Scholar metadata search, max 10 papers. Supports browser, SearchApi primary, and fallback modes.
• search_parallel(queries[], limit?, extract_mode?, extract_limit?, response_content?, max_chars?) - primary multi-query tool for broad live discovery and reading through a continuous four-tab queue. Set extract_mode in the same call when public web pages, PDFs, papers, or GitHub repositories must be read. Use local PDF tools only for local files or visual layout work, and clone repositories only for editing, building, testing, or full Git history. limit is 1-20 per query. The call-wide extract_limit defaults to 12 and allows up to 20 for abstract; full defaults to and allows 10. response_content defaults to summary to bound one-call output.
• Integrated search extraction reports requested, applied, skipped, truncated, and total_chars. remaining_urls can be passed to extract without repeating the search.
• extract(url, max_chars?, mode?, response_content?) - secondary extraction tool for an exact public URL when no new discovery is required. If sources still need to be found, use search or search_parallel with extract_mode instead.
  • mode="full" (default): reads up to 1000000 characters for research capture. Research mode stores deterministic 4000-character chunks; response_content="full" returns up to 50000 characters and summary returns a 1500-character evidence excerpt.
  • mode="abstract": ~1500-char survey (PDF page 1 or HTML meta description). Document metadata is included and stored with the survey when research mode is enabled.
  • mode="metadata": metadata without body text. Returns available title, authors, publication, dates, DOI, description, keywords, canonical URL, and PDF properties including page count.
  • GitHub repository URLs read the README in metadata mode. Abstract and full use the same download gate and differ only in indexed source depth.
  • Response: content fields plus available document metadata. Failures return { error }, never throw.
• project_memory_search(query, query_variants?, project_id?, include_project_ids?, all_projects?, limit?, request_id?, response_deadline_ms?) - searches stored local knowledge only. Up to 19 optional variants run inside one broker request with batched query embeddings, RRF fusion, evidence-seeded graph expansion, and one final rerank against query. The response contains only bounded query-focused summaries from the final ranking; stored bodies stay in the database. A caller-supplied request_id makes a long query observable and cooperatively cancellable through project_memory. If a deadline arrives after retrieval has produced candidates, the completed lanes return as an explicitly partial result instead of being discarded. It never opens a browser or calls Google or SearchApi.
• project_memory(action, ...) - manages durable project knowledge when SURF_RESEARCH=true.
  • action="search": compatibility alias for project_memory_search.
  • action="project_update": updates an existing project name, purpose, constraints, or protected parents as a revision-checked profile revision.
  • action="context": separates the durable project purpose, confirmed session intent, provisional recent query, current plan, scientific experiment state, and observed process liveness.
  • action="get": returns exact typed or legacy records through bounded UTF-8 byte spans. Set body_bytes=0 for metadata only and use next_body_offset for lossless paging.
  • action="show": always returns a bounded summary with counts and active record IDs. detail_level="full" is accepted for compatibility but never dumps every record body. Use project_memory_search for relevant bodies or target_id for one assertion or entity.
  • action="record": stores the submitted body and returns only its ID, revision, and status. Receipts distinguish the original application from an idempotent replay.
  • action="query_status" / action="query_cancel": inspects or cooperatively stops a local retrieval identified before execution by request_id.
  • action="export": writes a standalone interactive HTML explorer, Graphviz DOT, D3 node-link JSON, or a Neo4j import bundle under <research-root>/exports.
• health() - server status, including the local research runtime.

Need	Tool
Search only previously stored research and project memory	project_memory_search
Find new information on the web	search
Compare new web results with stored project knowledge	search with project_id
Run several new web queries	search_parallel

Replayable research collection

google-surf-collect runs a versioned JSON specification through one persistent MCP session. A specification can mix live search jobs with local-only project_memory_search jobs. Live jobs can extract bodies in the same call, while local jobs reuse indexed project knowledge without opening Google.

npx google-surf-collect examples/research-collection.example.json

From a source checkout:

npm run build
npm run research:collect -- examples/research-collection.example.json

A project workflow can also record durable sessions and plans, rebuild approved code roots, search the resulting local knowledge, and export its graph:

npm run research:collect -- examples/project-memory-workflow.example.json

project_memory collection jobs allow record, rebuild, and export. Destructive forget operations are not accepted by the collection schema. Project-level project_id is inherited by every job unless an all-project export is requested.

The output is append-only JSONL. Its manifest records the normalized specification hash, package version, Git commit, Node runtime, platform, project setup, and server health. Every search, record, rebuild, and export result records the stable job id, exact tool arguments, attempt, timestamps, elapsed time, response, and error state. Successful jobs are skipped on resume; failed jobs are retried. A changed specification requires a new output file. Set project_name with project_id when the runner should create a missing project; existing projects are reused.

Set retrieval_mode to live when prior project RAG state must not affect live result ranking. Results are still captured under project_id. Use hybrid when the collection intentionally ranks new web evidence together with stored project knowledge. API keys and environment variable values are never written to the collection log.

This makes the collection procedure and returned snapshot replayable and auditable. Live web results can still change with time, locale, network route, and upstream ranking.

Graph hybrid RAG with ontology and lineage

flowchart TB
    subgraph SOURCES["1. Search and research"]
        direction LR
        LIVE["Live web<br/>Google browser • SearchApi fallback"]
        PAPER["Pages and papers<br/>extract • Scholar metadata"]
        PROJECT_INPUT["Code and project records<br/>local roots • GitHub • host-provided session/plan"]
    end

    INGEST["2. Deterministic capture<br/>normalize • deduplicate • content hash<br/>repository source gate • Tree-sitter"]

    subgraph KNOWLEDGE_BASE["3. SurrealDB knowledge base"]
        direction LR
        CONTENT["Content and code index<br/>exact • BM25 • HNSW vector<br/>documents • chunks • symbols"]
        PROV["Data lineage and provenance<br/>source → evidence → assertion<br/>valid time • recorded time • correction"]
        ONTOLOGY["Versioned ontology<br/>core/project term revisions<br/>entity type • relation • alias • merge/split"]
        MEMORY["Project memory<br/>session intent • plan revisions<br/>experiments • decisions"]
    end

    subgraph INTELLIGENCE["4. Graph intelligence"]
        direction LR
        SCHEMA["Cross-project schema linking<br/>type and relation alignment<br/>stable identifier → identity bridge"]
        SIDECAR["Typed graph sidecar<br/>PageRank • Louvain • query-time PPR"]
    end

    FUSION["5. Hybrid retrieval<br/>live • exact • BM25 • vector • graph<br/>deterministic RRF • shared reranker • fresh-web floor"]
    RESULTS["Results + provenance<br/>short storage receipt"]

    LIVE --> INGEST
    PAPER --> INGEST
    PROJECT_INPUT --> INGEST
    INGEST --> CONTENT
    INGEST --> PROV
    INGEST --> ONTOLOGY
    INGEST --> MEMORY
    ONTOLOGY --> SCHEMA
    CONTENT --> SIDECAR
    PROV --> SIDECAR
    MEMORY --> SIDECAR
    SCHEMA --> SIDECAR
    LIVE --> FUSION
    CONTENT --> FUSION
    SIDECAR --> FUSION
    FUSION --> RESULTS
    RESULTS -. "automatic search/extract capture" .-> INGEST

    classDef inputStyle fill:#eef6ff,stroke:#2563eb,color:#172554
    classDef processStyle fill:#fff7ed,stroke:#ea580c,color:#431407
    classDef storageStyle fill:#ecfdf5,stroke:#059669,color:#052e16
    classDef intelligenceStyle fill:#f5f3ff,stroke:#7c3aed,color:#2e1065
    classDef outputStyle fill:#f8fafc,stroke:#475569,color:#0f172a
    class LIVE,PAPER,PROJECT_INPUT inputStyle
    class INGEST processStyle
    class CONTENT,PROV,ONTOLOGY,MEMORY storageStyle
    class SCHEMA,SIDECAR intelligenceStyle
    class FUSION,RESULTS outputStyle

One local knowledge base

One SurrealDB instance is the authority for three distinct layers:

• Catalog: stable identities, revisions, project membership, ontology, provenance, and temporal links.
• Payloads: exact bodies, source snapshots, manifests, and artifact references. Identical content is stored once and can be fetched by ID and byte span.
• Derived views: exact, BM25, vector, code, and graph indexes plus the compact current-project view. These are versioned by source hash and can be rebuilt.

A single local research broker owns the embedded RocksDB connection. Multiple MCP sessions connect through authenticated local IPC, run bounded reads concurrently, and serialize writes without opening another database process. Original evidence and historical occurrences remain addressable even when bodies, embeddings, or graph structures are deduplicated.

Durable records and incremental indexing

project_memory writes project profiles, plans, experiments, attempts, measurements, artifacts, documents, decisions, and sessions as typed records. A stable asset ID receives revisions; an uncertain retry with the same idempotency key returns the committed revision with idempotent_replay=true and applied_this_request=false. body_path streams bodies up to 256 MiB, artifact manifests are paginated instead of sliced, and get/get_batch return exact typed or legacy records through bounded body spans. Compact write receipts report accepted bytes, references, revision, and the exact readback handle without echoing the submitted body.