Self-Review Skill

You are helping a medical researcher check their own manuscript before journal submission. The goal is to anticipate reviewer comments by applying the same critical lens used in peer review across medical journals.

This is NOT about writing a review. It's about producing an actionable list of anticipated reviewer comments with specific fix suggestions, so the manuscript can be strengthened before reviewers ever see it.

Optional Flags

• --fix: After generating the review report, automatically apply fixes for all issues where fixable_by_ai is true. Edits the manuscript in place, then reports a diff summary. Does NOT fix issues marked fixable_by_ai: false (e.g., missing data, design flaws). Maximum 2 fix-and-re-review iterations.
• --json: Output the structured JSON block (see Phase 3c below) in addition to the markdown report. Default when called from /write-paper Phase 7.
• --panel: Run the multi-agent panel review (Phase 2.6) — several domain-expert reviewers in parallel plus an editor synthesis — instead of the single-pass review. Opt-in and off by default (a panel spawns N reviewer agents + 1 editor, so it costs several times more tokens). Reserve it for a high-stakes pre-submission final pass on a top-tier target. Do not combine with --fix: a panel diagnoses and prioritizes; run --fix as a separate follow-up pass once the author has triaged the panel's findings.

Severity Framing

When flagging issues, classify severity:

• Fatal: Fundamental design flaw that cannot be fixed with existing data (e.g., data leakage that invalidates all results, absence of any reference standard, label-feature circularity). The manuscript likely needs redesign. Submission would likely result in Reject.
• Fixable: Significant but addressable with existing data (e.g., missing calibration analysis, unclear exclusion criteria, absent CIs, incomplete reporting). These are the most actionable findings.

Most issues are Fixable. Reserve Fatal for true design-level problems.

Two Objectives: the Floor and the Ceiling

A submission-ready manuscript optimizes two things at once, and most of this skill (and the gate stack behind it) only optimizes the first:

• Floor — minimize rejection-for-cause. Fabricated citations, numbers that do not reconcile, overclaims, missing checklist items, leakage. Categories A–K and the deterministic gates (Phases 2.5–2.5f) do this, and they are right to. Many of them raise the floor by adding material: a hedge, a caveat, a disclosure, an audit trail, a checklist row.
• Ceiling — maximize editorial-championing. Will a handling editor read a confident narrative (problem → design → result → meaning) and want to send it out, or a defensive audit and bounce it? Nothing in the floor stack pushes here, and several floor gates push the other way. Iterated, a manuscript over-hardens: every individual gate finding is correct, yet the accumulated product reads as a rebuttal letter — over-hedged, audit-trail-heavy, Abstract buried under caveats, the strongest sensitivity result hidden in Limitations, too long.

These objectives can conflict, so the order matters: the floor gates run first and secure accuracy; then the ceiling pass (category L / Phase 2.5g) reads the accurate manuscript as a whole and recommends SUBTRACTION — REMOVE, MOVE, or TIGHTEN — so the same content is read confidently. The ceiling pass is advisory and never blocks; it cannot relax a floor gate. Without it, repeated self-review monotonically over-defends. Surface the ceiling findings as their own first-class output (Phase 3), not folded silently into the "add this" comments. Phase 2.5i (the loop controller) then reads the floor + ceiling state to declare when the loop is done — including a zero-edit PASS — so an accurate draft is not over-hardened by a pass it does not need.

Workflow

Phase 1: Intake

1. Get the manuscript -- PDF, Word doc, or pasted text.

2. Ask the user:

   • Target journal? (affects reporting standards and scope expectations)
   • Manuscript type? (original research / review / perspective / technical note / letter / meta-analysis / case report)
   • Anything they're already worried about?
   • Review depth? The default is a single-pass review. For a high-stakes pre-submission final pass, a multi-agent panel (--panel, Phase 2.6) is available — several domain-expert reviewers run independently, then an editor consolidates them (more thorough, but it spawns several agents so it costs several times more tokens). On an interactive run, surface this option once in one line and offer it; then proceed with the single-pass review unless the user opts in. Do not surface or auto-apply the panel when invoked with --json or from /write-paper — those stay single-pass.

3. Read the full manuscript.

4. SSOT gate — confirm there is one manuscript, not several. Self-review reads a single input file, so a divergence between a legacy working copy and the live submission copy is structurally invisible to it. Before a --panel run (or any pre-submission pass), check for multiple copies and reconcile first:

   find . \( -path '*manuscript*' -o -path '*main_document*' \) -name '*.md' | grep -v node_modules
   

   If more than one manuscript-like file exists, confirm which is the SSOT and run /sync-submission's divergence gate before reviewing — a STALE_COPY (an SSOT numeric claim or heading that did not propagate to the other copy) is a P0 that must clear first:

   python3 "${MEDSCI_SKILLS_ROOT:-$HOME/workspace/medsci-skills}/skills/sync-submission/scripts/detect_copy_divergence.py" \
     --ssot <ssot>.md --copy <other-copy>.md
   

   Review the SSOT copy; do not review a stale copy and pass it.

   In --panel mode this is a blocking precondition, not advice. A panel spawns N reviewer agents + an editor, so reviewing a stale copy wastes the whole pass (a prior panel's top finding was literally "you reviewed the wrong file"). If the find above returns more than one manuscript-like .md and the SSOT is not pinned — no SSOT.yaml with truth.manuscript_md and no explicit --ssot <path> argument — STOP before spawning any reviewer and have the user name the SSOT (and clear any STALE_COPY). Do not auto-pick the longest/newest file. The single-pass review may proceed on the one file it was given, but the panel must not.

Phase 2: Systematic Check

Run the manuscript through each applicable category below. For each item, assess whether a reviewer would raise it as a Major or Minor comment. Use the Research-Type Adaptation table (below) to determine which categories apply fully, partially, or not at all.

The categories (A–L). The per-item check tables — what to look for under each — live in references/phases/phase2_systematic_check.md; read it once you have the manuscript and know its type, and work the categories the adaptation table marks as applicable.

	Category	What it asks
A	Study Design & Data Integrity	patient-level splits, leakage, input-text contamination, analysis unit
B	Reference Standard & Ground Truth	definition specificity, timing, annotator independence
C	Validation & Statistical Reporting	CIs, calibration, comparator, effect size, power-aware nulls, equivalence margins, interaction anchoring
D	Clinical Framing & Importance	intended use, overclaiming, novelty, endpoint↔conclusion scope
E	Reproducibility	preprocessing, model detail, hardware/software, data & code availability
F	Reporting Completeness	abstract↔body consistency, flow diagram, ethics, missing data, word cap
G	Reporting Guideline Compliance	match the type to its checklist; /check-reporting does the item-level audit
H	Circularity	label–feature overlap, tautological prediction, circular validation
I	Protocol Heterogeneity	multi-site acquisition, harmonization, temporal protocol drift
J	Method Transparency	model provenance, fine-tuning, classical-style body conventions
K	Reviewer-team consistency	SR/MA only — dual-vs-single conjunction, LLM-as-reviewer (both fabrication-grade)
L	Editorial impression & defensiveness	advisory, never blocking — the ceiling category: REMOVE / MOVE / TIGHTEN

Run the deterministic gates. These are greps and counts, so they belong in a gate rather than in eyeballing. Run them at Phase 2 entry, on every path:

# D. endpoint↔conclusion scope
python3 "${CLAUDE_SKILL_DIR}/scripts/check_scope_coherence.py" \
  --manuscript manuscript.md --out qc/scope_coherence.json --strict

# J. classical-style body conventions
python3 "${CLAUDE_SKILL_DIR}/scripts/check_classical_style.py" \
  --manuscript manuscript.md --out qc/classical_style.json --strict

# K. reviewer-team consistency (SR/MA only; pass the extraction JSON file or directory)
python "${CLAUDE_SKILL_DIR}/scripts/check_reviewer_team_consistency.py" \
    --manuscript manuscript.md --prospero prospero/record.md \
    --extraction-json extraction/ --out _audit_self/reviewer_team_consistency.md

# L. editorial impression (advisory; exits 0 even under --strict)
python3 "${CLAUDE_SKILL_DIR}/scripts/check_editorial_impression.py" \
  --manuscript manuscript.md --out qc/editorial_impression.json

# J/D. Perspective structure (genre-gated: silent unless article_type is a Perspective).
# Pass the known type via --type; it also self-detects from the front-matter article_type.
python3 "${CLAUDE_SKILL_DIR}/scripts/check_perspective_structure.py" \
  --manuscript manuscript.md --type "${TYPE:-}" --out qc/perspective_structure.json

Verdict mapping: CROSS_SECTIONAL_PROGNOSTIC, SURROGATE_CARE_DIRECTIVE, SECTION_SYMBOL, INBODY_AI_DISCLOSURE, and any reviewer-team hit (exit 1) are Anticipated Major Comments. CROSS_SECTIONAL_YIELD_LANGUAGE, ELIGIBILITY_PROSE, DECIMAL_INCONSISTENCY, EM_DASH_OVERUSE, PERSPECTIVE_HEADING_NOT_ASSERTION, PERSPECTIVE_ABSTRACT_NO_AUTHORIAL_MOVE, and every check_editorial_impression verdict are Minor. The per-verdict rationale and the resolution paths are in the reference file.

Read on demand:

Then check that every analysis you report was ever defined. The detectors in this skill ask whether a number is correct. None asks whether the analysis that produced it was defined — and that is the gap a reviewer walks straight into:

"The outcome (dependent variable) for the multivariable Cox model is not specified." … "The ground truth (reference standard) against which discrimination and calibration were assessed is not defined." … "This section is largely incomprehensible in its current form."

python3 "${CLAUDE_SKILL_DIR}/scripts/check_analysis_definitions.py" \
  --manuscript manuscript.md --out qc/analysis_definitions.json --strict

MODEL_OUTCOME_UNDEFINED (a Cox / Fine–Gray / logistic model with no outcome named), MODEL_NOT_IN_METHODS, and REFERENCE_STANDARD_UNDEFINED (discrimination or calibration with nothing to score against) are Anticipated Major Comments. TIER_LABEL_UNDEFINED is Minor.

ANALYSIS_LOAD is informational and never a verdict. The reviewer who wrote "too many analyses have been performed and reported" also named the mechanism — "this appears to have contributed to omissions of critical information in the Materials and Methods section" — while a second reviewer of the same manuscript listed its sensitivity analyses as a strength. Load is the cause, not the crime. Do not cut analyses to satisfy this gate; restore the definitions the analyses crowded out. If load is genuinely high, move the defensive analyses to the supplement — same defence, far less reader burden and far less attack surface.

File	Read it when	Cost if read blindly
references/phases/phase2_systematic_check.md	you are working the A–L manual pass and know the manuscript type	~5,600 tokens — and a run that halts at Phase 1, or a panel-mode review, never reaches it

Research-Type Adaptation

Not all categories apply equally to every study type. Use this routing table:

Category	AI/ML	Observational	Educational	Meta-Analysis	Case Report	Surgical
A. Study Design	Full	Full	Partial	N/A	N/A	Full
B. Reference Standard	Full	Full	N/A	Per-study	Partial	Full
C. Validation & Stats	Full	Full	Full	Special*	Partial	Full
D. Clinical Framing	Full	Full	Full	Full	Full	Full
E. Reproducibility	Full	Partial	Partial	Partial	N/A	Full
F. Reporting	Full	Full	Full	Full	Full	Full
G. Guideline Compliance	Full	Full	Full	Full	Full	Full
H. Circularity	Full	Partial	N/A	N/A	N/A	Partial
I. Protocol Heterogeneity	Full	Full	N/A	Per-study	N/A	Full
J. Method Transparency	Full	Partial	Partial	N/A	N/A	Partial
K. Reviewer-team consistency	N/A	N/A	N/A	Full	N/A	N/A
L. Editorial impression	Full	Full	Full	Full	Full	Full

*Meta-analysis: Replace C with heterogeneity assessment (I-squared, prediction intervals), publication bias (funnel plot, Egger), and sensitivity/subgroup analyses.

Type-Specific Additional Checks:

• Observational studies: Confounding assessment (DAG or adjustment strategy), selection bias, exposure measurement validity. Run Phase 2.5e (Confounding Completeness), then apply the O-probes in references/domain-probes/observational_confounding.md — the two deterministic ones are O1 (a covariate imbalanced by exposure in Table 1 yet absent from the adjustment set) and O8 (records > subjects with the analysis unit undisclosed; check_cohort_arithmetic.py --id-col), and O7 is their opposite-direction twin (adjusting for a consequence/mediator of the outcome). If the manuscript develops or compares a clinical prediction model (TRIPOD / TRIPOD+AI, nested predictor-set comparison), also apply the CP-probes in references/domain-probes/clinical_prediction_model.md. The module is the single source for the probe list and its numbering; do not re-enumerate it here.
• Educational studies: Learning outcome measurement validity, Kirkpatrick level, control group adequacy, curriculum fidelity
• Meta-analyses: Search comprehensiveness (2+ databases), screening reproducibility (2 reviewers), RoB assessment per study, GRADE certainty
• Case reports: Diagnostic reasoning transparency, timeline completeness, informed consent, generalizability disclaimer
• Surgical studies: Learning curve consideration, surgeon volume/experience, complication grading (Clavien-Dindo), operative detail completeness

Domain probe modules (load when the manuscript type matches):

These modules carry the same domain-specific critique probes used by /peer-review, vendored here so self-review reaches the same depth (in particular, survival/time-to-event manuscripts now get a dedicated probe set that the routing table above does not otherwise cover).

Manuscript type / signal	Probe module
Systematic Review / Meta-Analysis	references/domain-probes/sr_ma.md (P0–P19)
Time-to-event / survival / prognostic model (Cox, Fine-Gray, DeepSurv, nomogram, risk-stratification cutoff)	references/domain-probes/survival_prognostic.md (S1–S9)
Radiomic feature reproducibility / acquisition-parameter sweep / reliability-based feature filtering	references/domain-probes/radiomics.md (R1–R4)
Cross-modality image synthesis (MRI→PET / MRI→CT / non-contrast→contrast / low-dose→full-dose) claiming functional/molecular information or target-modality substitution	references/domain-probes/image_synthesis.md (IS1–IS4)
Narrative / review article / primer / state-of-the-art	references/domain-probes/narrative_review.md (RV1–RV9)
Perspective / opinion / viewpoint (argumentative essay — npj DM long-essay, Lancet Comment, NEJM AI / RYAI short-structured)	references/domain-probes/narrative_review.md (RV1–RV9) + the deterministic check_perspective_structure.py gate above (IMRAD-heading + abstract authorial-move tells)
AI/ML primary study with a clinical claim (generalizable / outperforms clinicians / deployment-ready / can replace a reader)	references/domain-probes/ai_overclaiming.md (AO0–AO7)
Engineer-built medical-imaging model (segmentation / classification / detection; CNN / U-Net / nnU-Net / transformer) being validated — partition/leakage, seed & run variance, metric selection, reproducibility, reference-standard quality; plus saliency-map faithfulness, uncertainty/OOD/abstention, and deployment feasibility when a clinical-use claim is made	references/domain-probes/model_development.md (MD0–MD11)
LLM / MLLM evaluated on a clinical task (radiology report generation, visual question answering, clinical text extraction/classification; closed API or open weights)	references/domain-probes/mllm_evaluation.md (ME0–ME8)
Randomised controlled trial (parallel / crossover / cluster / stepped-wedge)	references/domain-probes/rct_trial.md (RC0–RC7)
Diagnostic test accuracy (DTA) primary study / multi-reader multi-case (MRMC) reader study (index test vs reference standard, AI-vs-reader, AI-assisted reading, modality comparison)	references/domain-probes/diagnostic_accuracy.md (D1–D12)
Case report / case series / single-patient clinical narrative (incl. adverse-event/pharmacovigilance and imaging-led radiology/nuclear-medicine/IR reports)	references/domain-probes/case_report.md (CR1–CR9)
AI/ML, prediction, or diagnostic study claiming cross-population performance (generalizable / deployment-ready / "works for patients"), or presenting subgroup analyses as a fairness/equity argument	references/domain-probes/equity_fairness.md (EQ0–EQ6)
Mendelian randomization (genetic variants as instrumental variables: two-sample summary-data, one-sample, multivariable MR, drug-target / cis-MR, non-linear MR)	references/domain-probes/mendelian_randomization.md (MR1–MR8)
Polygenic risk score / polygenic score (PRS / PGS) developed, validated, or applied as a predictor or risk-stratifier	references/domain-probes/polygenic_risk_score.md (PG1–PG8)
Network meta-analysis (≥3 interventions via direct + indirect evidence, treatment ranking, incl. component NMA)	references/domain-probes/network_meta_analysis.md (NM1–NM8)
Health economic evaluation (cost-effectiveness / cost-utility / cost-benefit / budget-impact; trial-based or decision-model-based — decision tree, Markov, DES)	references/domain-probes/health_economic_evaluation.md (HE1–HE8)
Observational study using routinely-collected health data (administrative claims / EHR / disease or population registry / health-checkup DB, linked or not)	references/domain-probes/record_routinely_collected_data.md (RD1–RD8)
Self-report survey / questionnaire study (KAP, physician/patient survey, cross-sectional questionnaire, web/e-survey)	references/domain-probes/survey_research.md (SV1–SV8)
Scoping review (maps the breadth/nature of evidence, clarifies concepts, identifies gaps; PCC framing, charting, optional appraisal — not a focused effectiveness/accuracy question)	references/domain-probes/scoping_review.md (SC1–SC8)
Qualitative study (interviews, focus groups, ethnography, grounded theory, phenomenology, document analysis; reflexivity, trustworthiness, thematic analysis — not quantitative validity)	references/domain-probes/qualitative_research.md (QL1–QL8)
Self-improving / self-evaluating system (an agent that critiques and rewrites its own output; training on model-generated data; an LLM used as the judge that scores the training signal; "self-evolving" clinical agents)	references/domain-probes/self_improving_system.md (SI1–SI7) + skills/peer-review/scripts/check_self_improvement_claims.py

For a classifier / NLP / tabular ML manuscript, also run the deterministic feature-selection-leakage gate — a data-driven selection (feature selection, log-odds / univariate filtering, vocabulary construction, a threshold) fit on the FULL dataset before cross-validation inflates the CV metric:

python3 "${CLAUDE_SKILL_DIR}/scripts/check_cv_leakage.py" \
  --manuscript manuscript.md --out qc/cv_leakage.json

CV_SELECTION_LEAKAGE (Major) fires when a selection token co-occurs with cross-validation and no fold-nesting is disclosed ("within each fold" / "nested CV" suppresses it). This is distinct from patient-vs-image split leakage (model-validation/check_split_leakage.py).

When the manuscript matches a row, read ${CLAUDE_SKILL_DIR}/references/domain-probes/<module>.md and apply each probe as an additional source of Anticipated Major / Minor Comments. The module severity words (MAJOR / MINOR) map to this skill's framing as follows: a conclusion-threatening or design-level finding becomes a Fatal Anticipated Major Comment, a reporting-level finding becomes a Fixable Anticipated Minor Comment, and each is tagged with the closest category letter (A–K). These probes complement categories A–K above; they do not replace them. (The modules are vendored byte-identical from /peer-review; do not edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync.)

Phase 2.5: Numerical Cross-Verification (Internal)

Before generating the report, verify internal consistency:

1. Abstract vs Body: Do all numbers in the Abstract match the Results section and Tables?
2. Table vs Text: Cross-check key metrics (sample sizes, primary outcomes, p-values) between tables and narrative text.
3. Figure vs Text: Do figure legends match the data described in Results?
4. Percentage arithmetic: Verify that n/N percentages are calculated correctly (e.g., 23/150 = 15.3%, not 15.0%).
5. CI plausibility: Do confidence intervals seem reasonable given sample sizes?
6. Rate back-calculation: every reported rate must invert to its own numerator/denominator — an incidence rate ≈ events / person-years × scale (±rounding). A rate that does not recompute from the stated events and person-time (or that implies more events than the cohort can supply) is a Major, not a Minor.
7. Exclusion-cascade and complete-case arithmetic (cohort/observational): the STROBE flow must balance — start N − Σ(exclusions) == final analytic N — and any complete-case statement must balance — total − missing == complete. A footnote N that does not equal the subtraction is a Major.

For cohort/observational manuscripts, run the deterministic gate instead of eyeballing it (it parses prose equations + GFM tables, and recomputes from a committed CSV when given one):

python3 "${CLAUDE_SKILL_DIR}/scripts/check_cohort_arithmetic.py" \
  --manuscript manuscript.md --data analysis/cohort.csv --id-col mockid \
  --out qc/cohort_arithmetic.json --strict

RATE_BACKCALC / CASCADE_SUM / PARTITION_OVERLAP rows are Anticipated Major Comments (category: A. Study Design & Data Integrity); the partition check is the Phase 2.5b cohort branch below. Pass --id-col (or let it auto-detect a subject-ID column) on health-screening / EMR / registry data so the gate also runs the analysis-unit check: when records > unique subjects and the manuscript states neither the analysis unit nor a one-record-per-subject sensitivity, it emits ANALYSIS_UNIT_UNDISCLOSED (Major — non-independent observations give anti-conservative CIs; probe O8). Flag any remaining internal-consistency discrepancies as Anticipated Minor Comments (category: F. Reporting Completeness).