paper-search-pro

Multi-source literature search with adjustable depth. Four tiers, five data sources orchestrated by you (the main agent). Python helpers handle deterministic work; LLM classification is delegated to parallel Inline SubAgents — no external API key required.

When to use this skill

• User wants to find academic papers / 找文献 / 论文搜索
• User is preparing a literature review, systematic review (SR), scoping review, or meta-analysis
• User wants to scope research on a topic for a thesis / proposal / coursework / news story
• User asks "what research exists on X" / "find me papers about Y"
• User uploads a query that suggests literature gathering (PICO, SPIDER, MeSH, RCT, etc.)

When NOT to use

• User wants to read a specific paper (use PDF reader / download tool)
• User wants to summarize a single known paper (use a summarizer)
• User wants to download PDFs given DOIs (use paper-downloader-portable)
• User already has a literature set and wants to write a review (use literature-set-review / factor-outcome-review)
• User wants concept explanation, not papers ("what is prospect theory" → just answer)

---

🤖 Called by another agent / headless mode

If you are an agent driving this Skill for your own reasoning (not for a human who wants an HTML report), do NOT hand-run the 14-STEP recipe below. There is a single structured-data channel built for you:

PYTHONPATH=$PSP_HOME python3 -m scripts.agent_search "<query>" > result.json

One command runs the whole deterministic core — multi-strategy retrieve → dedup → heuristic relevance score (computed for every paper) → saturation signal → quota snapshot → per-paper journal metric — and prints one JSON envelope (no HTML, no PRISMA, no LLM classification SubAgent). The human path below is unaffected. That command gives you a deterministic floor, not the finished job — agent mode is not meant to stop at the machine output; references/agent_mode.md is where you layer your own semantic judgement on top to reach human-recipe quality (the command guarantees the floor; you supply the quality).

📖 Read references/agent_mode.md for the full envelope schema, every flag (--verify, --min-relevance, --quartile, --min-impact, …), the relevance formula, error codes / exit codes, and source selection. This is the SSOT for agent callers — everything else in this section is just the pointer to it.

Everything from here down is the human-facing 14-STEP recipe (HTML report + exports). Use it when the consumer is a person.

---

🔥 Execution discipline (read before running anything)

Four invariants govern every step — ignoring them is the dominant failure mode in real sessions:

• A — NEVER cd into the Skill directory. cd $PSP_HOME rebinds ./ to the Skill asset folder, so ./paper-search-results/... lands inside the Skill instead of the user's workspace (and a re-install wipes it). Run every helper from the user's PWD: PYTHONPATH=$PSP_HOME python3 -m scripts.<name> … > "$SEARCH_DIR/...". $PSP_HOME (STEP 0) is the install dir; $SEARCH_DIR (STEP 0) is an absolute path under the user's PWD.
• B — Dispatch classifier SubAgents in parallel. STEP 6 puts up to 5 Task blocks in one assistant message; serial dispatch inflates Standard tier from ~10 to ~17 min. The worked example lives in STEP 6 — it is not repeated elsewhere.
• C — Announce every skip. If you skip a STEP (budget / empty data / user choice), say what you skipped, why, what's lost, and how to recover (e.g. "re-run at --tier deep"). Skipping is fine; surprising the user is not.
• D — Read a step's cited reference when that step is non-trivial for this case. Each STEP names a references/<file>.md carrying edge cases not duplicated here. You won't read all of them every run, nor should you — but skipping the reference for a step you are actually about to run is where boundary knowledge (dict-vs-list shapes, enrich-not-search, DOI casing) gets lost. Read the one in front of you.

---

Architecture at a glance

You (main agent) drive the workflow per this SKILL.md.
Python helpers do deterministic work — NO LLM inside, NO external API key.

  L1 OpenAlex (primary)  → deep top-100 multi-strategy
  L2 PubMed (medical)    → MeSH enricher (mostly; Audit-tier can search independently)
  L2 arXiv (CS/preprint) → T-0~T-4 freshness sentinel
  L3 Semantic Scholar    → influentialCitationCount + abstract fallback
  L3 CrossRef            → funder / license / clinical-trial-number

  Classification         → Inline SubAgents (parallel, file-IPC, 5 per message)
  Output                 → HTML (Shadcn) + MD + BibTeX/RIS/CSV + PRISMA-S log

---

The 4 tiers — pick first

Tier	Wall-clock	Papers	When to pick
Quick	~5-8 min	20-60	"查一下" / "几篇" / "before tomorrow" / fast scope
Standard (default)	~10-17 min	60-180	Scope a topic / write background / general lit search
Deep	~30-45 min	180-400	"thorough" / writing a review article / 综述写作
Audit	~2-3 hr	400-1000+	"systematic review" / "PRISMA" / "Cochrane" / "meta-analysis"

📖 BEFORE picking, read references/tier_decision.md. Tell the user your choice and why. For Audit, show limitations warning + get explicit confirmation before starting.

---

The recipe

For every literature search, follow these steps in order. Each step references a references/ file for details. Skip files only when the step is obviously trivial for the case at hand — and announce the skip per Rule C.

STEP 0 — Setup ($PSP_HOME + working directory)

📖 BEFORE THIS STEP, read: references/setup.md.

Resolve the Skill install path into $PSP_HOME (every later step uses PYTHONPATH=$PSP_HOME). Prefer explicit injection / agent env var; otherwise scan the known cross-agent install locations. If your harness already exposes this SKILL.md's absolute path, just export PSP_HOME="<that dir>" and skip the scan. 📖 Full rationale, why this can't be a script, and the complete path list: references/runtime_bootstrap.md.

PSP_HOME="${PSP_HOME:-${CLAUDE_SKILL_DIR:-${CODEBUDDY_SKILL_DIR:-}}}"   # explicit / agent-injected
if [ -z "$PSP_HOME" ]; then                                            # else scan known installs
  for base in "$HOME/.claude" "$HOME/.codex" "$HOME/.agents" "$HOME/.config/opencode" \
              "$HOME/.codeium/windsurf" "$HOME/.config/goose" "$HOME/.cline" "$HOME/.roo" \
              "$HOME/.copilot" ./.claude ./.codex ./.agents ./.cursor ./.opencode ./.windsurf; do
    [ -f "$base/skills/paper-search-pro/SKILL.md" ] && PSP_HOME="$base/skills/paper-search-pro" && break
  done
fi
[ -z "$PSP_HOME" ] && { echo "ERROR: paper-search-pro install not found. Set PSP_HOME to the dir containing SKILL.md."; exit 1; }
export PSP_HOME; echo "Using Skill install: $PSP_HOME"

Verify config keys (executed from any cwd, never cd into the Skill dir):

PYTHONPATH=$PSP_HOME python3 -c \
  "from scripts.config import load_config; c = load_config(); print('OK' if c.openalex_api_key and c.ncbi_email else 'MISSING — see references/setup.md')"

If "MISSING", point the user to references/setup.md (5 keys, all free, ~15 min total) and halt.

Set up the working directory variable — every subsequent step uses $SEARCH_DIR:

SEARCH_ID="<topic_slug>_<tier>_$(date +%Y%m%d_%H%M%S)"   # e.g. clt_education_quick_20260522_103045
SEARCH_DIR="$(pwd)/paper-search-results/$SEARCH_ID"
mkdir -p "$SEARCH_DIR/raw" "$SEARCH_DIR/batches" "$SEARCH_DIR/classifications"
echo "Outputs will land in: $SEARCH_DIR"

$SEARCH_DIR is now an absolute path under the user's PWD. Use "$SEARCH_DIR/..." (quoted, with the variable) in every helper command below — not ./paper-search-results/....

STEP 1 — Plan the query (MANDATORY for all tiers)

📖 BEFORE THIS STEP, read: references/query_planner.md.

Detect the report UI language — UI_LANG (zh for Chinese queries, en for everything else) selects which UI language the final HTML report renders in. Paper titles / abstracts / authors / venues are NEVER translated — only the report's UI chrome. Pass --language $UI_LANG to STEP 12b.

UI_LANG=$(PYTHONPATH=$PSP_HOME python3 -m scripts.detect_language "$USER_QUERY")

The detector routes Japanese / Korean / European queries to English (the bundle ships only EN + ZH dictionaries; English is the international academic default). 📖 The exact Unicode rule and why kana is checked before Han live in references/runtime_bootstrap.md.

Determine the search language space (search_language, axis 2 — which literature ocean, distinct from UI_LANG above which is only report chrome). 📖 The parsing SSOT is references/source_routing.md §"Language scope"; resolve the space here, before phrasing the query, because it changes how STEP 3's query is built. This is additive and opt-in — a pure English query resolves to the en space with zero new prompts or behavior (R-19); everything below fires only for Chinese queries or explicit signals.

• Read the persisted default config.search_language (auto | en | zh | both) and apply the priority ladder flags > in-query markers > config > auto. CJK presence is the mechanical fact from detect_language above; markers (CSSCI, 中文文献, SSCI, 知网, …) and non-signals (中科院一区, topic-about-China) are your semantic judgment per the §"Language scope" tables.

• auto + a Chinese (CJK) query + no language marker + no persisted value → ask ONE question before retrieving (this is the human path's job; the CLI/agent path passes through instead). Two sentences, offer to persist, and don't re-ask later this session:

  你用中文提问——文献要英文、中文,还是都要?顺便可以说"以后都这样",我就记成默认、下次不再问。

  • "中文" / "都要" → enter that space (STEP 2 discipline routing takes over; report one line there).
  • "英文" → v2.2 behavior (Chinese topic planned as an English query), report one line.
  • "无所谓 / 都行" → this run uses both (Recall > Precision), not persisted; if the same user answers "无所谓" a second time, add one light offer to set both as default, then never ask again.
  • Only an explicit "以后都…" persists to config.search_language (single answers never auto-persist).

• If a rank ambiguity (bare "Q1") also fired this run, merge both questions into ONE message — ask language + platform together, never in two rounds (over-asking is a red line).

• Once the space is known, phrase the query per references/query_planner.md §"Cross-language query handling": zh keeps Chinese search terms (no translation), en uses the English terms (v2.2 behavior), both builds two sets.

Apply PICO / SPIDER / PEO depending on domain:

• Medical/clinical → PICO (Population/Intervention/Comparator/Outcome)
• Qualitative → SPIDER
• Scoping → PEO (Population/Exposure/Outcome)
• Open-ended → just extract 2-4 concept blocks + 2-5 synonyms each

Journal-rank intent recognition (additive — only acts when the query mentions a partition). Before you extract concept blocks, check whether the user's query carries a journal-rank/partition phrase — "中科院一区", "Q1", "JCR Q1", "SJR Q2", "顶刊 / top journal". If so, that phrase is a filter condition, not a search term, and it MUST be stripped from the topic before retrieval. This roots out the failure that motivated the whole feature: "中科院一区 情绪调节" used to send "中科院一区" to the search engine as a topic word, so it searched for papers about 中科院一区 instead of papers on 情绪调节 filtered to CAS tier 1. The deterministic parser does both jobs (extract + strip) for you:

PYTHONPATH=$PSP_HOME python3 -c "
from scripts.rank_intent import parse_rank_intent
i = parse_rank_intent('''<original user query>''')
import json; print(json.dumps({
  'platform': i.platform, 'tiers': i.tiers, 'quartiles': i.quartiles,
  'top': i.top, 'ambiguous': i.ambiguous, 'cleaned_query': i.cleaned_query,
  'stripped': i.matched}, ensure_ascii=False))
"

Then act on the parse:

• cleaned_query is the real topic — use it (NOT the raw query) for STEP 3 retrieval and the query plan. When the query had no rank phrasing, cleaned_query == query and nothing changes (R-19 default path is untouched).
• platform + tiers/quartiles/top are the filter you will apply in STEP 10/11 — remember them; do not filter here.
• ambiguous == True (a bare "Q1"/"Q2" with no platform word — the recogniser never guesses a platform): ask the user one short question inline before going further — "按 JCR 还是 SJR 的 Q1 筛?顺便要不要设为以后的默认?" The CLI/headless path cannot ask, so this inline question is specifically the human path's job.
• If the query mentions no partition at all, skip this entirely — STEP 1 proceeds exactly as before.

Even Quick tier needs a lightweight version of this step — never skip silently. Output: 1-3 search strategies (concept blocks + year range + work type filter). Write to "$SEARCH_DIR/query_plan.json" so PRISMA-S logger can pick it up later (STEP 13).

STEP 2 — Route supplemental sources within the STEP-1 language space

📖 BEFORE THIS STEP, read: references/source_routing.md.

You make these routing calls by judging the query's domain — the reference's keyword tables are calibration examples, not a match list (mechanical facts — CJK detection, explicit --flags — stay deterministic). Within the language space fixed in STEP 1, route the per-discipline boosters:

• English space (en, or the English half of both) — unchanged from v2.2:
  • Medical signals (RCT, PRISMA, MeSH, clinical, disease names) → enable PubMed
  • CS/preprint signals (preprint, arXiv, NeurIPS, transformer, "最新", 2024+) → enable arXiv
  • Cross-domain (e.g. "AI in radiology") → enable both
  • Pure social science / humanities → OpenAlex only
  • (Judgment call: a core AI/CS query may also raise the primary engine to Semantic Scholar — see source_routing.md §"AI / CS queries → consider Semantic Scholar as primary".)
• Chinese space (zh, or the Chinese half of both) — route the Chinese boosters the same way, by discipline:
  • Social-science / humanities signal → add NSSD (国家哲社文献中心; carries the CSSCI 收录标识 OpenAlex has ≈0 coverage of)
  • Medical signal → add yiigle (中华医学期刊全文数据库); PubMed still covers MEDLINE-indexed 中华 journals, so the two are complementary
  • Pure sci-tech with neither → Chinese side runs on OpenAlex only (sci-tech Chinese core journals mostly register DOIs, so OpenAlex covers them well)

Report one line (axis-3 style — a statement, not a question; 22 §6.3). For an English-only run this is the existing PubMed/arXiv notice, unchanged ("I detected medical + CS signals — also searching PubMed and arXiv. Override with --no-pubmed."). For a Chinese space, e.g.:

本次按「中英都要」检索;中文侧检测到社科主题,已加 NSSD(国家哲社文献中心)。想去掉说 --no-nssd,只查一边说"只要英文/中文"。

Coverage honesty rides with the notice: if the zh space has a social-science topic but the user declined NSSD, add that OpenAlex hits ≈0 on CSSCI flagship journals (经济研究 / 管理世界 …), so that layer is missing. User can override any of this with an explicit instruction (a per-query override wins over everything).

On the --flag shorthands above (--no-nssd, --no-pubmed, --source …): on this human path they are natural-language override notation — a compact way to write what the user can say ("去掉 NSSD" / "只查 OpenAlex"), which you (the LLM) interpret. They are not executable CLI flags — no script parses them here. The only real, script-parsed flags live on the agent/headless path (agent_search), and there the Chinese-source control is opt-in: --with-nssd / --with-yiigle (there is no --no-nssd / --source there). See references/agent_mode.md.

STEP 3 — Retrieve from OpenAlex (deep)

📖 BEFORE THIS STEP, read: references/openalex_helper_cheatsheet.md.

Always run OpenAlex first. (OpenAlex is the default primary source; only if primary_source is set in config.yaml or the OpenAlex quota is exhausted, see Primary source selection & quota fallback in references/source_routing.md for the additive SS-fallback flow — default behavior is unchanged.) For Standard+ tiers, use multi-strategy deep crawl:

PYTHONPATH=$PSP_HOME \
  python3 -m scripts.openalex_helper double-sort "<query>" \
    --n 50 --year-min 2018 \
    > "$SEARCH_DIR/raw/openalex.json"

For Quick tier, single-strategy is fine:

PYTHONPATH=$PSP_HOME \
  python3 -m scripts.openalex_helper search "<query>" \
    --limit 30 --year-min 2018 \
    > "$SEARCH_DIR/raw/openalex.json"

The full subcommand + flag reference (search / double-sort / seminal / reviews / journal-list / citation-network, all verified against argparse) is in references/openalex_helper_cheatsheet.md — read it before reaching for anything beyond the two commands above. For Deep+Audit, also call topic-specific subcommands (e.g. seminal, reviews, journal-list), append outputs to $SEARCH_DIR/raw/openalex_*.json, and federate them all together in STEP 5.

STEP 4 — Run L2 boosters (if enabled by STEP 2)

📖 BEFORE THIS STEP, read: references/pubmed_helper_cheatsheet.md and references/arxiv_helper_cheatsheet.md.

PubMed — default mode is enrich, NOT search:

• Standard / Deep tier: enrich OA-found papers with MeSH terms (mutates the openalex.json file in place):

  PYTHONPATH=$PSP_HOME \
    python3 -m scripts.pubmed_helper enrich \
      --input-file "$SEARCH_DIR/raw/openalex.json" \
      --output-file "$SEARCH_DIR/raw/openalex.json"
  

• Audit tier with explicit MeSH query: independent MeSH search (produces a new file to federate later):

  PYTHONPATH=$PSP_HOME \
    python3 -m scripts.pubmed_helper search-mesh "Diabetes Mellitus, Type 2" \
      --year-min 2020 --limit 30 --pub-type "Randomized Controlled Trial" \
      > "$SEARCH_DIR/raw/pubmed.json"
  

• Generic pubmed_helper search is a fallback when no MeSH term is known — prefer enrich or search-mesh whenever possible.

arXiv — only if query contains freshness signals (preprint, 最新, 2024+):

PYTHONPATH=$PSP_HOME \
  python3 -m scripts.arxiv_helper freshness "<query>" \
    --days 4 --limit 30 \
    > "$SEARCH_DIR/raw/arxiv.json"

Subcommand reference:

• arxiv_helper freshness <query> --days N --limit M [--all-cats]
• arxiv_helper search <query> --limit M --sort submitted|relevance|lastUpdated [--all-cats]
• arxiv_helper get <arxiv_id>

NSSD / yiigle — Chinese boosters (ONLY when STEP 1-2 put this run in the zh space, and only the one(s) the discipline routing selected):

Each is an independent primary source for the Chinese space (same role as ss_helper --search) and emits the same UnifiedPaperEntity shape as openalex.json, so STEP 5 federates them identically. Keep the query in Chinese (do NOT translate — query_planner §Cross-language) and write to raw/nssd.json / raw/yiigle.json:

# NSSD — Chinese social-sciences & humanities (adds the CSSCI-tier layer OpenAlex lacks)
PYTHONPATH=$PSP_HOME \
  python3 -m scripts.nssd_helper --search "<中文检索式>" \
    --n 50 --year-min 2018 \
    > "$SEARCH_DIR/raw/nssd.json"

# yiigle — Chinese medical (中华医学期刊全文数据库; native-Chinese titles + abstracts)
PYTHONPATH=$PSP_HOME \
  python3 -m scripts.yiigle_helper --search "<中文检索式>" \
    --n 50 --year-min 2018 \
    > "$SEARCH_DIR/raw/yiigle.json"

Both degrade gracefully to [] on any network / HTTP failure (they never raise) — an empty file just federates to nothing. Both take only 题录 + 摘要 (compliance: no full-text download, no caching) and print their source attribution to stderr. --year-min filters client-side; drop it to keep all years.

config search_language: en but a Chinese query arrived (hard boundary 2): do NOT enable Chinese boosters, but say one line — never a silent translation (22 §6.4):

按你的默认(只查英文),我把中文主题规划成英文检索式了。想要中文文献这次说一声即可,想改默认说"以后…"。

User names 知网 / CNKI / 万方 / 维普 (marker hit + compliance): these are closed subscription databases PSP does not scrape. Say one line, offer the substitute, don't re-argue (22 §6.5):

PSP 不接知网/万方(合规原因,不做封闭库抓取)。中文侧用 OpenAlex 中文底座 + NSSD(社科,含 CSSCI 标识)/yiigle(医学)覆盖;如需知网全文,结果里的题录可去知网人工检索。继续吗?

STEP 5 — Federate (dedup + merge)

📖 BEFORE THIS STEP, read: references/source_routing.md §"Field priority table".

Combine all retrieval results into a single deduped KG. Default output is a dict keyed by canonical_key — that's what rcs_parser expects later, so do NOT pass --as-list:

PYTHONPATH=$PSP_HOME \
  python3 -m scripts.federated_kg_resolver \
    --input-files "$SEARCH_DIR/raw/openalex.json" \
                  "$SEARCH_DIR/raw/pubmed.json" \
                  "$SEARCH_DIR/raw/arxiv.json" \
    --output "$SEARCH_DIR/kg.json"

Pass only the input files you actually produced — skip ones that were not enabled by STEP 2. If STEP 4 ran the Chinese boosters, add "$SEARCH_DIR/raw/nssd.json" / "$SEARCH_DIR/raw/yiigle.json" to the same --input-files list — they carry the identical entity shape and federate exactly like the others (CJK-safe dedup is handled by the Phase 0 canonical-key fix, so distinct Chinese titles don't collapse). For an English-only run those files don't exist, so the call is byte-identical to v2.2 (R-19). This handles DOI normalization (arXiv X→x case), version stripping, E5b guard (same title+year but different DOIs are kept separate), and field-priority merge.

--as-list exists but is only for consumers that want a sorted list (by citation_count); do not use it in this pipeline.

STEP 6 — Classify in parallel batches (LLM happens here — main agent + SubAgents)

📖 BEFORE THIS STEP, read: references/classifier_subagent_prompt.md and references/rcs_rubric.md.

Split the KG into batches of 10 papers each. Write to "$SEARCH_DIR/batches/batch_NNN.jsonl".

Before dispatch, expand $PSP_HOME/references/rcs_rubric.md into the actual absolute path (e.g. /Users/alice/.claude/skills/paper-search-pro/references/rcs_rubric.md) and substitute it for {rubric_path} in the classifier prompt template. Each SubAgent runs in its own shell where $PSP_HOME is not exported — passing the literal $PSP_HOME token would leave the SubAgent unable to find the rubric, which silently degrades scoring quality. See references/classifier_subagent_prompt.md for the full placeholder table.

🔥 PARALLELISM IS MANDATORY (Rule B):

You MUST dispatch up to 5 classifier SubAgents in a single assistant message using multiple Task tool_use blocks. Serial dispatch (one Task per message, waiting for each result) is the single biggest performance failure observed — it inflates Standard tier from ~10 min to ~17 min.

✅ CORRECT — in ONE assistant message:

Task tool_use #1  → subagent_type="general-purpose", prompt="<classifier prompt for batch_001.jsonl>"
Task tool_use #2  → subagent_type="general-purpose", prompt="<classifier prompt for batch_002.jsonl>"
Task tool_use #3  → subagent_type="general-purpose", prompt="<classifier prompt for batch_003.jsonl>"
Task tool_use #4  → subagent_type="general-purpose", prompt="<classifier prompt for batch_004.jsonl>"
Task tool_use #5  → subagent_type="general-purpose", prompt="<classifier prompt for batch_005.jsonl>"

All five tool_use blocks live in the same <assistant> message. The harness fires them in parallel; you receive five tool_result blocks back together.

❌ WRONG — five separate messages (this is what serial dispatch looks like):

Message N:    Task tool_use #1 ─→ wait for result
Message N+1:  Task tool_use #2 ─→ wait for result   ← SERIAL, makes Standard run 70% slower
Message N+2:  Task tool_use #3 ─→ wait for result
...