Testing Traps

Config-discovery isolation

ryl's project-config discovery climbs from each input through its ancestors up to HOME, so a test whose inputs live under the system temp dir can walk into that shared dir and discover a stray ryl.toml/.ryl.toml/pyproject.toml/.yamllint* left by another test, a concurrent process, or a manual smoke run — silently overriding the test's setup (and a TOML candidate outranks a tempdir's .yamllint, so an adjacent YAML config does not shield it).

Any test that exercises discovery (does not pass -c/-d, and has no adjacent TOML config in its input's directory) must build its command via common::cli::ryl(<its tempdir>), which sets HOME to bound the walk at the tempdir. Tests that pass -c/-d bypass discovery and need no isolation; tests that write an adjacent .ryl.toml are already shielded. Correspondingly, manual/agent smoke runs must keep scratch configs in a dedicated subdirectory and never drop a config-candidate-named file at the temp root.

Assert format-agnostically

CLI/system tests that drive env!("CARGO_BIN_EXE_ryl") run under CI's environment, where GITHUB_ACTIONS makes ryl auto-select the GitHub output format (::error file=…,line=L,col=C::L:C [rule] message) rather than the standard format ( L:C level message (rule)).

Match the bare line:col (present verbatim in both formats) and the bare rule id (colons, never (colons) — the GitHub format renders it [colons]). Do not force --format to dodge this and do not assert a specific format's (rule) parens or ANSI. The cli_*_rule tests follow this; only tests that exercise formatting itself (cli_format_options, yamllint_compat_*) pin or scrub the format via --format/env_remove.

Unicode fixtures

For Unicode-heavy fixtures, assert with multibyte characters (e.g. "café"/"å") and reuse crate::rules::span_utils rather than reinventing byte/char conversions, to cover character-vs-byte offset logic.

Committed JSON config schemas

ryl.toml.schema.json / ryl.yaml.schema.json are generated by ryl --print-toml-config-schema / --print-yaml-config-schema. Always run prek after regenerating: --print emits schemars insertion order, but the pretty-format-json hook rewrites JSON with recursively sorted keys, so the committed form is sorted (committing raw --print output causes the recurring "prek reordered the schema" churn).

Because the canonical form is sorted, regeneration only changes lines when schema content changes — a structural change that doesn't alter content yields a zero diff after sorting, so don't commit a reordered file (leave it at HEAD). tests/config_schema.rs compares order-insensitively, but the files must still be committed sorted to keep prek idempotent; when an options type is renamed, update the RuleEntryFor…/RuleOptionsFor… names asserted there too.

SchemaStore

The vendored SchemaStore yamllint snapshot lives at tests/fixtures/schemastore-yamllint.json; refresh it with uv run scripts/update_yamllint_schemastore_snapshot.py instead of fetching from the network in normal tests.

The SchemaStore TOML projection comes from uv run scripts/print_ryl_schemastore_schema.py; it targets only ryl.toml / .ryl.toml because SchemaStore cannot attach directly to [tool.ryl] inside pyproject.toml.

Docs config examples

Config examples in the docs/ Markdown sources are validated through ryl's finalized config path (the discover_config -c path the CLI uses) by tests/docs_config_examples.rs, so misspelled rule names and options are caught, not just bad values (docs/llms*.txt are generated from those sources and drift-guarded, so they are covered transitively).

A block is recognised as ryl config structurally from its content (a toml block with a [tool.ryl] table or a table named in the TOML config schema; a yaml block with a top-level key in the YAML config schema), so other tools' TOML and rule-input YAML are skipped, and a malformed config example is caught. Put <!-- ryl-config-check: skip --> on the line before a fence to exempt an intentional counter-example (e.g. the YAML-1.1 config in yaml-version.md shown as failing).