Similarity Search Patterns

SkillSearch

A skill that guides an AI agent through implementing similarity search with vector databases. It covers distance metrics such as cosine and Euclidean, index types including flat, HNSW, and IVF+PQ, and the speed and accuracy tradeoffs between them. Use it when building semantic search, nearest neighbor queries, or retrieval performance tuning.

Then ask your AI: use the Similarity Search Patterns skill

Details

Instructions available. Your AI can read the instructions. Execution depends on the setup they require.

Have a vector database and a set of embeddings available to search over.

What your AI can do with it

  • Choose a distance metric: cosine, Euclidean, dot product, or Manhattan
  • Compare flat, HNSW, and IVF+PQ index types and their tradeoffs
  • Tune parameters such as ef_search and nprobe for recall and speed
  • Combine keyword and semantic search in a hybrid setup
  • Monitor recall and latency to measure search quality

Getting started

  1. Have a vector database and a set of embeddings available to search over.
  2. Add the skill to your agent so it can read the patterns and reference templates.
  3. Tell the agent what you are building, such as semantic search or a recommendation engine.
  4. Have the agent pick a distance metric and index type for your data size and recall needs.
  5. Measure recall and latency before tuning parameters or scaling the index.

What this skill tells your AI

The instructions your AI receives, as published by wshobson/agents in plugins/llm-application-dev/skills/similarity-search-patterns/SKILL.md and read by Ahel’s review.

Patterns for implementing efficient similarity search in production systems.

When to Use This Skill

  • Building semantic search systems
  • Implementing RAG retrieval
  • Creating recommendation engines
  • Optimizing search latency
  • Scaling to millions of vectors
  • Combining semantic and keyword search

Core Concepts

1. Distance Metrics

| Metric | Formula | Best For | | ------------------ | ------------------ | --------------------- | --- | -------------- | | Cosine | 1 - (A·B)/(‖A‖‖B‖) | Normalized embeddings | | Euclidean (L2) | √Σ(a-b)² | Raw embeddings | | Dot Product | A·B | Magnitude matters | | Manhattan (L1) | Σ | a-b | | Sparse vectors |

2. Index Types

┌─────────────────────────────────────────────────┐
│                 Index Types                      │
├─────────────┬───────────────┬───────────────────┤
│    Flat     │     HNSW      │    IVF+PQ         │
│ (Exact)     │ (Graph-based) │ (Quantized)       │
├─────────────┼───────────────┼───────────────────┤
│ O(n) search │ O(log n)      │ O(√n)             │
│ 100% recall │ ~95-99%       │ ~90-95%           │
│ Small data  │ Medium-Large  │ Very Large        │
└─────────────┴───────────────┴───────────────────┘

Templates and detailed worked examples

Full template library and detailed worked examples live in references/details.md. Read that file when you need the concrete templates.

Best Practices

Do's

  • Use appropriate index - HNSW for most cases
  • Tune parameters - ef_search, nprobe for recall/speed
  • Implement hybrid search - Combine with keyword search
  • Monitor recall - Measure search quality
  • Pre-filter when possible - Reduce search space

Don'ts

  • Don't skip evaluation - Measure before optimizing
  • Don't over-index - Start with flat, scale up
  • Don't ignore latency - P99 matters for UX
  • Don't forget costs - Vector storage adds up

Signals

GitHub stars
40k
Forks
4k
Last commit
Oct 2026

Questions

What distance metrics does it cover?
Cosine, Euclidean (L2), dot product, and Manhattan (L1). Cosine suits normalized embeddings, Euclidean suits raw embeddings, dot product is for when magnitude matters, and Manhattan is for sparse vectors.
Which index types does it explain?
Flat (exact, O(n) search, 100% recall, small data), HNSW (graph-based, O(log n), roughly 95 to 99% recall, medium to large data), and IVF+PQ (quantized, O(sqrt n), roughly 90 to 95% recall, very large data).
How do I tune for better recall?
Adjust parameters such as ef_search and nprobe, and measure recall and latency before and after each change. The skill advises evaluating first rather than optimizing blindly.
Advanced
Item type
skill
Key
similarity-search-patterns-wshobson
Source
github.com/wshobson/agents