A Documentation Site That Checks Itself Against the Live Cluster

Purpose

This is a follow-up to the first drift detector post, which compared image tags in posts with what was running. In October 2026 a freshly published migration post said the cluster’s local-zfs volumes were “1 GiB to 20 GiB each”. A 256Mi volume existed. The detector had no check for storage, so nothing caught it.

The fix has three layers. None of them can block a publish. The pipeline itself is described in the docs pipeline post.

Deployment

The root cause was one rule in the doc-writer skill. It said the brief’s facts were already verified and should not be re-checked. That rule quietly overrode another one, “live tool beats brief”. Only facts with a hard verify rule, versions and counts, were ever checked against the cluster.

The three layers:

1. The writer re-derives what its read-only tools can reach, and tags the rest.
2. The weekly detector gained two structured checks.
3. A draft-time “drift shadow” puts an advisory table in the pull request.

flowchart LR
    A["Draft states a fact"]
    B["Writer re-derives from kubectl<br/>or tags brief-only"]
    C["Publish wrapper<br/>(read-only kubectl)<br/>snapshots nodes and PVs"]
    D["Gate (no kubectl)<br/>renders drift shadow table"]
    E["PR body advisory"]
    F["Brad merges"]
    G["Weekly sweep:<br/>k8s_version and pv_size_range"]
    H["Drift report"]
    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    G --> H

Configuration

Layer 1 is doc-writer skill 1.9.2, deployed 2026-10-26. It re-derives versions, counts and storage with kubectl get pv and kubectl get sc. Anything its profile cannot reach is tagged brief-only: host placement, schedules, hardware specs and one-off measured numbers. The writer’s sandbox cannot reach pct, the Proxmox backup job config or the backup manager, so those facts stay cited from the brief rather than gaining new privileges.

Layer 2 adds two checks to the weekly detector:

• k8s_version: a post names a vX.Y.Z+k3sN that no node runs. Lines with history words such as “upgraded from” are skipped, so upgrade stories are not flagged.
• pv_size_range: a post states a PV size range whose floor is above the live minimum or whose ceiling is below the live maximum. The comparison is scoped by storage class.

Each check stays inert unless its snapshot is passed, so CI without a cluster behaves as before:

kubectl get nodes -o json > nodes.json
kubectl get pv -o json > pv.json
docs-drift.py ... --nodes nodes.json --pv pv.json

An unscoped first pass flagged the “20 GiB ceiling” against two 1 TiB NFS volumes that the post had described separately. Scoping by storage class removed that false positive. Over all 61 published posts the checks found one true positive, the 1 GiB floor against the 256Mi volume, with no false positives and no k8s_version findings. A re-run over all 62 posts with live snapshots on 2026-10-30 found nothing for either check.

Layer 3 is the drift shadow. The publish wrapper runs as the agent user, which has read-only kubectl, and drops the two snapshots into the clone. The privileged gate has no kubectl. It only imports the clone’s own detector module to render the advisory. The advisory is a ## Drift shadow (advisory, non-blocking) heading with a Post / Documented / Live table, appended to the PR body or to the review-fix comment.

Storage

The live state the checks compare against, read on 2026-10-30:

• All 4 nodes report kubelet v1.30.2+k3s2.
• One storage class, local-zfs: default, WaitForFirstConsumer, reclaim Retain, no expansion.
• Its volumes: 12 x 1Gi, 9 x 2Gi, 4 x 5Gi, 4 x 10Gi, 1 x 20Gi and 1 x 256Mi, so the range is 256Mi to 20Gi.
• Two 1Ti volumes with no storage class, the NFS volumes, are the largest of any class.

Access

Brad sees the result in the PR he reviews. A draft that states a fact that no longer matches live state gets a row in the advisory table, next to the live value. On 2026-10-26 a throwaway draft deliberately stated v1.29.4+k3s1. The PR body showed the advisory row beside the live v1.30.2+k3s2, and the publish still succeeded.

Notes

• Nothing here blocks a publish. Infra facts are timing-sensitive and the human merge is already the gate. Blocking is kept for deterministic checks: sanitize, gitleaks and lint.
• Counts of pods, nodes and apps are deliberately not checked. They change too often and would give false positives.
• One contradictory rule in a prompt beat the rest of it: “don’t re-check the brief” won over “live tool beats brief”.
• The gate importing a module from the clone left a __pycache__ that the agent user could not delete. Setting sys.dont_write_bytecode fixed it.
• Tests: 10 new unit tests, and the drift tool’s test file now has 31. The publish pipeline harness passes 50 checks, 3 of them for the shadow.