fleet-audit — Audit Findings to a Ledger Issue

Every autonomous audit watchdog ends the same way: findings must reach a human somewhere durable, reviewable, and de-duplicated. This skill is that ending, in two tiers:

• Tier 1 — the ledger. Each audit stream owns exactly one open GitHub issue, rewritten in full on every run and closed as completed when the fleet comes back clean. An operator watches one issue per stream instead of drowning in chat logs.
• Tier 2 — the fixes. When a finding's remediation is a file in this repository, it travels separately as a narrow pull request carrying only that fix, linked back to the ledger.

The split is the point. A report is not a change, so a report is not a pull request — and a fix is not a report, so it carries a real diff a reviewer can read in one screen.

./skills/fleet-audit/scripts/audit_report.py owns every deterministic operation: credential minting, label creation, issue creation and rewriting, branch handling, staging, committing, pushing, pull-request creation, closing, the run-over-run delta, and every timestamp. Your job is to inspect the fleet read-only and emit a findings.json. You never hand-write an issue body or a PR body, never invent a timestamp, and never call gh issue create or gh pr create yourself — that is precisely why every ledger looks the same and why the delta between runs is computable.

Audit streams

Only these seven audit ids may own a ledger. Any other id is rejected before a single git or gh command runs. The issue title is [audit] <human name> — <n> findings (<c> critical) (singular 1 finding when there is exactly one), where the human name is the one cron/jobs.json gives that watchdog — not a prettified form of the audit id:

Audit id	Rendered ledger title
compliance-audit	[audit] Security & RBAC Posture Audit — 7 findings (2 critical)
security-patch-orchestrator	[audit] Upgrade & Patch Readiness Audit — 7 findings (2 critical)
obtainability-audit	[audit] Workload Reliability Audit — 7 findings (2 critical)
fleet-wide-cost-analysis	[audit] Fleet Waste Audit — 7 findings (2 critical)
fleet-consistency-drift	[audit] Fleet Consistency Drift Audit — 7 findings (2 critical)
ai-security-audit	[audit] AI Workload Security Audit — 7 findings (2 critical)
stockout-prevention	[audit] Fleet Stockout Prevention & Capacity Audit — 7 findings (2 critical)

The mapping lives in AUDITS at the top of audit_report.py and mirrors cron/jobs.json; a test fails if the two drift apart. Do not restate a title anywhere else.

Running a stream on demand

Each stream's cron job id is its audit id, so an operator asking for a run off-schedule is asking for one command per stream:

HERMES_HOME=/opt/data/profiles/platform /opt/hermes/.venv/bin/hermes cron run compliance-audit

Every stream's cron job lives in this profile's own roster, ticked once a minute by the Chat Agent's profile-cron-tick. hermes cron run marks the job due rather than running it here; the next tick picks it up within a minute and runs it through the identical path the 06:20 tick uses, with the stream's prompt verbatim, its skills preloaded, and this profile's max_turns.

cronjob(action='run') is not the route. Where the session cannot take a detached result — a one-shot hermes -z, a stateless HTTP turn, a Kanban worker, a nested cron run — or where the dispatch pool is full, it executes the job synchronously inside the session that calls it, which is the re-enactment the next paragraph exists to prevent. Elsewhere it hands the run to the background delegation executor and returns a handle; that is closer to what you want, but hermes cron run is the one route that behaves identically on every runtime and always runs in a fresh process.

Your shell cannot reach that command, and there is no substitute yet. It runs on the gateway pod, where hermes and /opt/data/profiles are; your shell runs in the sandbox pod, which has neither, so command not found there is the split working as designed rather than a broken install. When you hit it, say the on-demand trigger is unavailable and that the stream will run on its 06:20 schedule. That does not license either fallback: not cronjob(action='run'), and not running the audit yourself — see the next paragraph. The gap is a deliberate deferral of the shell-sandbox design, not an oversight.

Do not run the audit yourself in the session that received the request. A triggered run gets its own process and its own turn budget. A session that improvises the audit instead has neither — and when the request is "run them all", it has one turn budget for work the schedule spreads across every stream and two days. That is not a hypothetical failure mode: on 2026-08-03 a single worker asked to run all five streams that existed then issued zero kubectl commands, hand-typed five empty findings documents, and published a fleet-wide all-clear.

The scheduler holds a per-job lock for the length of a run, so a stream already in flight is not started a second time and cannot write its ledger issue twice. cronjob(action='runs') shows what is running and what each attempt did.

Each run reports on itself. Your own answer is a roll-up, not a copy. Answer with one line per stream — the stream, and that it is queued for the next tick. The reports arrive through each run's own deliver setting; repeating them here sends the same content twice.

The two-command lifecycle

Run both commands from your normal working directory — the profile directory, where ./skills/... resolves. You are not in a git checkout, and you do not need to be. The audit crons start in the profile directory; the harness establishes its own workspace at /opt/data/gitops/<audit-id>/<owner>__<name> and resolves every remediation.path against it. The workspace is keyed by audit id because the audit streams share the volume with each other and with every kanban worker: each one gets a tree nobody else writes in, so a colliding schedule can no longer reset another stream's working copy out from under it. The repository comes from the $GITOPS_STATE_CONFIGMAP ConfigMap, which the operator manages and which is readable before any workspace exists.

What that workspace contains depends on the install, and start tells you which one you have as its mode field:

• content — the workspace is an empty directory. The repository lives in the credential broker, which owns the only checkout; you write manifests into the directory and the harness hands the bytes over. Nothing on your side is a git repository, so there is nothing here to read the repository out of: use list and fetch (below) for that.
• directory — the workspace is a clone of the GitOps repository on the shared volume, and the harness runs checkout, add, commit and push inside it.

Everything else is identical, including where you write manifests and what finish publishes. Where the two differ, this file says which mode it is talking about.

Step 1 — start

Before inspecting anything, claim the workspace:

./skills/fleet-audit/scripts/audit_report.py start \
  --audit <audit-id> \
  [--repo "<owner>/<repo>"]

This resolves the target repository (using --repo if specified, falling back to the single configured repo in $GITOPS_STATE_CONFIGMAP, or failing if ambiguous across multiple repos), mints a repo-scoped GitHub token, establishes a clean workspace, ensures the audit's labels exist, locates the stream's open ledger issue, and clears any findings document a crashed run left behind. If the user asked for a specific repository that is not yet registered, instruct the user or cluster administrator to add it to $GITOPS_STATE_CONFIGMAP. It creates no branch — there is no report branch. It prints exactly one JSON line:

{
  "issue": 128,
  "repo": "acme/fleet",
  "mode": "content",
  "workspace": "/opt/data/gitops/compliance-audit/acme__fleet",
  "findings_path": "/opt/data/scratch/findings_compliance-audit.json",
  "pending_remediation_requests": ["netpol-missing-payments"],
  "sop": "governance/compliance_audit_sop.md",
  "checks": ["privileged-container", "host-namespace", "…"],
  "checks_contract": "Run every check above against every cluster you can read. …"
}

Write your findings to the findings_path it gives you. Do not pick your own path.

checks is your stream's full roster, handed over so coverage never depends on how far into the SOP you read. It is the work list, not a substitute for the SOP — the slug says which check, the SOP says what the check is and what counts as a violation, so read the whole file before you start. sop names it.

workspace is where your manifests go. Every remediation.path is resolved against it, so a manifest written anywhere else is a file the harness will never find — the finding degrades to a manual one and no pull request opens. start scrubs that directory before handing it to you; finish does not, which is what lets the files you write in between survive.

mode is content or directory, and it changes one thing you can see: in content mode the workspace is empty rather than a checkout. Read it rather than guessing from what is on disk.

pending_remediation_requests lists the findings a repository writer has already asked to be fixed, parsed from the ledger's comments. Write those manifests during inspection — if the finding is still reproducing at finish, its pull request opens immediately instead of a week later.

Step 2 — Inspect the fleet (reasoning phase)

Enumerate the clusters in scope and inspect them read-only (kubectl get/describe, gcloud ... describe/list). For every deviation you intend to report, capture the exact command you ran and the output that proves it.

Keep a per-cluster tally as you go: for each check in the roster start printed, the slug and the exact command you issued for it, appended the moment that check completes. finish requires it as checks_run and rejects a slug with no command. Reconstructing the tally afterwards from memory is how a check that never ran gets recorded as one that did — and now that each entry carries a command that gets published, reconstructing it from memory is also how you end up publishing a command you never issued.

If a remediation is a declarative file, write that file under the workspace directory start reported and name its repo-relative path in the finding. The harness puts it on a branch of its own.

Directory mode only: do not leave unrelated uncommitted work in that tree during an audit. Opening a remediation pull request there requires switching branches, and the harness forces the switch. It snapshots and restores every path you declared, and returns you to the branch you started on — but a file it was never told about is not covered by that guarantee. Content mode switches no branch and writes nothing back into the workspace, so this does not apply.

Reading the repository in content mode

The clone is gone, so three commands stand in for it. grep searches inside the files, list names them, and fetch copies the ones you name into the workspace:

./skills/fleet-audit/scripts/audit_report.py grep  --audit <audit-id> --pattern 'namespace: payments'
./skills/fleet-audit/scripts/audit_report.py list  --audit <audit-id> --prefix clusters/prod-us-east
./skills/fleet-audit/scripts/audit_report.py fetch --audit <audit-id> --path clusters/prod-us-east/payments-netpol.yaml

grep runs broker-side and answers with matching lines; it is a fixed string unless --regex, and --prefix narrows it. Use it when what you know is what a file says. list answers with paths and sizes, never content — use it when what you know is where the file lives. The broker caps what each returns, so read truncated on both and pass --prefix on a large repository. fetch writes each file into the workspace at its repo-relative path, which is exactly where a remediation editing that file has to end up; fetch it, edit it in place, and name the same path in the finding.

All three take --branch, and a second round needs it. Without it they answer from the base, so a file the remediation branch has already changed — by an earlier run or by a reviewer — comes back as the base has it, and committing the edit onto that branch reverts the change. The revert fast-forwards, so nothing objects. Pass the remediation branch whenever the remote already has one; a branch it does not have falls back to the base, which is what a first round wants anyway.

All three exit 2 in directory mode, where the clone already holds the file.

Step 3 — finish

./skills/fleet-audit/scripts/audit_report.py finish \
  --audit <audit-id> \
  --findings-file <findings_path> \
  [--repo "<owner>/<repo>"]

The script validates the document, reconciles every finding against the pull requests already open for this stream, rewrites (or opens) the ledger issue, comments the delta, opens pull requests for the fixes that qualify, and closes the ones whose findings have stopped reproducing. It prints one JSON line with nine fields — status, issue_url, new, resolved, prs_opened, prs_closed, partial, coverage_gaps, and silent_ok:

• {"status":"OPENED","issue_url":"…","new":7,"resolved":0,"prs_opened":["…"],"prs_closed":[],"partial":false,"coverage_gaps":[],"silent_ok":false} — the stream had no open ledger.
• {"status":"UPDATED","issue_url":"…","new":2,"resolved":3,"prs_opened":[],"prs_closed":["…"],"partial":false,"coverage_gaps":[],"silent_ok":false} — the existing ledger was rewritten.
• {"status":"CLEAN","issue_url":"…","new":0,"resolved":5,"prs_opened":[],"prs_closed":["…"],"partial":false,"coverage_gaps":[],"silent_ok":false} — zero findings; the ledger closed as completed and its open fixes closed with it.

Add --dry-run to validate and print the rendered ledger body — and every PR body it would open — to stdout with zero git or gh side effects. It applies the same grouping and the same degradation as the real run, so the branch names it names are the branch names it would create. It resolves every remediation.path against the same workspace directory the real run uses, not against the directory you happen to be standing in, so "the manifest is missing" is a finding of the dry run and not a surprise at publish time. Use it whenever you are unsure your document is well formed.

Exit 0 means published. Exit 2 means the run was rejected before publishing anything — fix what the message names and re-run; never delete the finding that tripped it. Three things reach exit 2: the document failed a field rule, the file named by --findings-file is missing or is not valid JSON, or --audit is not one of the registered ids above. Exit 1 is fatal and means something else broke.

Partial coverage

partial is true exactly when the run could not speak for the whole fleet: any entry in scope.skipped, any cluster carrying a limitations note, or any cluster whose checks_run is short of the checks that apply to it. coverage_gaps says which, and why — so partial is true if and only if coverage_gaps is non-empty, and you can report from either.

A check the cluster's shape rules out is not a gap. Declaring it in that cluster's checks_not_applicable (below) takes it out of the denominator, so a cluster that ran everything that can apply to it is a fully covered cluster. Without that, a fleet of Autopilot clusters is permanently partial: the ledger never closes, resolved is pinned at 0, and no stale remediation pull request is ever cleaned up.

It does not mean "the description was truncated." A ledger too long for GitHub's body limit says so in its own body and still carries true totals in its title; the audit saw everything, so nothing about what the run may conclude changes. Coverage is the only thing partial tracks.

A gap changes what the run is allowed to conclude, because a finding's absence from an unread cluster is not evidence that it was fixed. Over a partial run the harness:

• reports resolved: 0 and posts no "resolved" delta, rather than announcing fixes it cannot see;
• closes no remediation pull request as stale, so a fix survives to the next complete run;
• does not close the ledger, even with zero findings — status is still CLEAN, but the issue stays open and gains a comment naming the gaps. The stream self-heals the day the fleet is fully readable again.

A partial run is never [SILENT] — finish returns silent_ok: false for it. Report the issue URL and say which clusters were not covered. See The clean run for the full rule.

The findings document

{
  "audit": "compliance-audit",
  "scope": {
    "clusters": [
      {
        "name": "prod-us-east",
        "location": "us-east1",
        "project": "acme-prod",
        "checks_run": [
          {
            "check": "privileged-container",
            "command": "kubectl --context prod-us-east get pods -A -o jsonpath='{range .items[*]}{.metadata.namespace}{\"/\"}{.metadata.name}{\"\\t\"}{.spec.containers[*].securityContext.privileged}{\"\\n\"}{end}'"
          },
          {
            "check": "netpol-missing",
            "command": "kubectl --context prod-us-east get networkpolicy -A -o custom-columns=NS:.metadata.namespace --no-headers"
          },
          {
            "check": "workload-identity-off",
            "command": "gcloud container clusters describe prod-us-east --location us-east1 --project acme-prod --format='value(workloadIdentityConfig.workloadPool)'"
          }
        ]
      },
      {
        "name": "prod-autopilot",
        "location": "us-central1",
        "project": "acme-prod",
        "checks_run": [
          {
            "check": "netpol-missing",
            "command": "kubectl --context prod-autopilot get networkpolicy -A -o custom-columns=NS:.metadata.namespace --no-headers"
          },
          {
            "check": "workload-identity-off",
            "command": "gcloud container clusters describe prod-autopilot --location us-central1 --project acme-prod --format='value(workloadIdentityConfig.workloadPool)'"
          }
        ],
        "checks_not_applicable": [
          {
            "check": "legacy-metadata",
            "reason": "GKE Autopilot: no user-managed node pools to carry a metadata setting."
          },
          {
            "check": "hostpath-mount",
            "reason": "GKE Autopilot: hostPath volumes are rejected by the admission webhook."
          }
        ],
        "limitations": "RBAC denied `list clusterrolebindings`; check 2.4 did not run."
      }
    ],
    "skipped": [{ "cluster": "dr-west", "reason": "control plane unreachable" }]
  },
  "findings": [
    {
      "id": "netpol-missing-payments",
      "severity": "critical",
      "title": "payments namespace has no NetworkPolicy",
      "cluster": "prod-us-east",
      "namespace": "payments",
      "object": "Namespace/payments",
      "evidence": {
        "command": "kubectl --context prod-us-east get networkpolicy -n payments",
        "excerpt": "No resources found in payments namespace."
      },
      "impact": "All east-west traffic into the PCI namespace is unrestricted.",
      "recommendation": {
        "action": "Apply a namespace default-deny NetworkPolicy, then allow the two known callers.",
        "rationale": "Default-deny at the namespace is the smallest change that closes the exposure. A mesh AuthorizationPolicy would only cover injected pods, and payments runs two that are not.",
        "risk": "Unlabelled cross-namespace traffic breaks on apply. Run `kubectl -n payments get pods --show-labels` first to confirm the callers."
      },
      "remediation": {
        "kind": "manifest",
        "path": "clusters/prod-us-east/payments-netpol.yaml",
        "note": "Apply a default-deny NetworkPolicy."
      }
    }
  ]
}

Field rules the validator enforces — a violation exits 2 naming the offending finding index and field, and publishes nothing:

• audit must equal the --audit argument. An audit may only write to its own ledger.

• scope.clusters must be non-empty. An audit that enumerated nothing is a failure, not a clean run — if you could not list the fleet, say so loudly instead of reporting zero findings.

• checks_run is required on every cluster (the example above shows three entries per cluster for brevity; a real run carries one per check it ran). Each entry is an object with two required fields:

  • check — the backticked slug from the SOP heading that defines it (netpol-missing, not "2.6" and not prose). An unknown slug or a duplicate is rejected.
  • command — the literal invocation you issued on that cluster for that check, with its --context/--project and the namespace or resource it targeted. It must name one of kubectl, gcloud, gsutil, bq, helm, or curl; echo, cat, python3 -c, a call back into audit_report.py, and anything under eight characters are all rejected. One command per entry — the one that produced the evidence, not a summary of your approach.

  An empty list is rejected too, unless that cluster's limitations says why nothing ran. Enumerating a cluster and checking nothing on it is not a clean cluster — it is an audit that did not happen, and without this field the harness cannot tell the two apart. See Scope, skipped, and limitations.

• checks_not_applicable is optional, and says which checks the cluster's shape rules out. Each entry is an object with two required fields:

  • check — the same slugs checks_run uses. An unknown slug, a duplicate, or a slug that also appears in this cluster's checks_run is rejected: a check either ran or could not.
  • reason — why the check cannot apply here, naming the property of the cluster that rules it out ("GKE Autopilot: no user-managed node pools to carry a metadata setting"). Anything under sixteen characters is rejected, which is enough to stop "N/A" and "n/a — autopilot".

  These checks leave the coverage denominator instead of counting as missing, so a cluster that ran everything that can apply to it is fully covered. That is the difference between a fleet whose ledger can close and one that is permanently partial. Use it only for a check the cluster's shape forbids — a check you could have run and did not is a limitations note and a real gap. Every entry is published in the ledger under Not applicable, with its reason, where a reviewer who knows the cluster can call an excuse for what it is.

• check is required, and is the backticked slug in the heading of the SOP check that produced the finding. Anything outside that SOP's roster is rejected.

• Do not write an id. The harness derives it as <check>.<cluster>.<namespace>.<object> — one grammar for all audit streams — lowercasing each part, replacing every run of non-alphanumerics with -, and substituting _ for an absent namespace. Any id in the document is discarded.