My Test Profile

I keep a voice profile so my writing stays mine no matter who drafts it. This is the same idea for testing: the operational summary of every check, lint, gate and proof layer I've had to build, kept current so the next project starts with them instead of re-earning them one incident at a time. Everything below exists because something real broke, and every mechanism links to its implementation — in recon-gen or claude-plan-bridge — so you can see exactly how it's done, not just that it should be.

This is a LIVING page — I update it in place as projects teach me new checks, version footer at the bottom. It's organized by how far a check travels: the overarching ones first, then what's specific to a language, then what's specific to a problem domain.

The principles

Every check below is an instance of one of these:

One source of truth, verified against the EMITTED artifact. Duplicates drift. When a law or schema must exist twice (the meaning and the SQL that hunts for it), machine-check the bridge — and check the real emitted output, never a re-read of the source that emitted it.
Fail loud, never silent. A teardown failure fails the build. A skipped test is gated. An unannotated detector is a hard error, not a missing row nobody counts.
Lock the meaning, not the bytes. Byte-snapshots cry wolf on every harmless refactor; a snapshot of what the system FOUND fires exactly when behavior changed.
Every claim carries a quantifier. "Proven" with no domain attached is a doc bug. The vocabulary I use: proven for all integers, proven on an enumerated domain, tested on the states exercised or trusted with a named mitigation.
A check that can't fail is broken. Every custom lint ships a planted bad example proving it fires; the test domain itself is scored by a mutation gate. Adequacy is measured, not assumed.
Suppressions carry a WHY. An ignore comment without a reason is a decision with the rationale thrown away — and a lint enforces that rule on the other lints' ignores.
Bit-identity is designed, not hoped. When output must reproduce byte-for-byte — run-to-run, across platforms, fast path == slow path — pin the nondeterminism at the seam: one parallelism door, a reducer whose non-associative form WON'T COMPILE, total-order comparators, output canonicalized so provenance never leaks into the bytes. Then back the construction with a run-to-run byte check — by-construction is only as strong as the proof no later edit opened a second door.

Overarching — any language, any domain

The checks that travel regardless of stack or problem. Ordered cheapest-first, author-time before the harness.

Type discipline — the cheapest layer

Runs at author time, and in a typed-enough language most of it is free (the Rust section below has the receipts). The concept is universal; what it COSTS in Python vs Rust is in the language sections.

NewType every identifier string (ids.py). SheetId, VisualId, ParameterName — function params, dict keys and dataclass fields take the wrapper, never bare str. Zero runtime cost; a swapped argument becomes a type error instead of a zero-row query.
Money is an integer type end-to-end (money.py). Native cents, no float anywhere on the money path.
Validate at construction, not in a test (controls.py is a dense example). __post_init__ raises on a blank description, an out-of-range value list, a control bound to the wrong parameter shape — the invariant fails at the wiring site with a stack trace, not three layers later in generated output.
Equality is a decision. A bare @dataclass silently gives structural equality on mutable state; tree nodes must declare frozen=True or eq=False. Closed enums over status strings, exhaustive handling over stringly-typed branches.
Strict type checking scoped by an include list that only grows (pyproject.toml, [tool.pyright]). New files enter strict scope; nothing leaves. Explicit Any needs a WHY comment.

Harness integrity — the tests need testing

This layer watches the harness, the fixtures and the CI config.

Session-start gates (conftest.py). The type checker runs as a subprocess before any test; a type error kills the session with the same verdict CI would give. Generated assets (the built CSS) are rebuilt to temp and byte-diffed against the committed copy (build_app2_css.py) — with an honest platform limit: minifier output isn't byte-identical across OSes, so that one gate is local-only.
Fail-loud plumbing (tests/conftest.py). Fixture teardown failures collect per-worker and force a failing exit at session end — a silent teardown failure doesn't hurt this run, it poisons the NEXT one. Tests without a tier mark error at collection. A skip or xfail touching the canonical example fixture fails the build unless its exemption reason starts with an approved prefix.
Completeness gates. Drive the runner's real per-layer selectors through pytest --collect-only and assert every e2e file is claimed by at least one layer (test_layer_coverage_reconciliation.py) — a test that runs nowhere is green forever. Every importorskip must name a dependency actually declared in pyproject (test_importorskip_declared.py), or the module skips everywhere silently.
CI config is code, so lint it. A unit test parses the release workflow and asserts every job runs on hosted runners (test_release_yml_secret_isolation.py) — publish secrets never touch self-hosted; exceptions live in an allowlist requiring a one-line WHY. faulthandler_timeout stays on so a hang dumps every thread's stack instead of eating the job's time limit.
Git hooks for the friction loops (.githooks/). Pre-commit rebuilds and re-stages generated assets when their sources are staged; pre-push runs the cheapest full tier (~3-4 min, no containers) before a CI cycle gets burned. What pre-push mirrors matters: it must run the CI's LINT gate, not just a compile — a lint-only failure passes a bare compile and reddens CI anyway, and the fix is cheap (fab-scad's pre-push runs clippy -D warnings, not cargo check; the Rust section has the receipt).
A fuzzer that doesn't build is a green checkmark testing nothing (fab-scad's fuzz workflow). A "fuzz-from-first-commit" campaign was dead at BUILD for a stretch and nobody noticed: cargo-fuzz 0.13 defaults its sanitizer target to x86_64-linux-MUSL (static libc → ASan can't instrument, musl std absent) so every run died at build; an edition bump left one target's set_var uncompilable; and a whole set of targets ran in NO workflow. Three ways to HAVE fuzz targets and do zero fuzzing. Gate on the campaign EXECUTING — exec/s, coverage climbing (seeding the differential target from real corpus code took it 1k→13k on the first live run) — never on the target file merely existing.

Building the tests themselves

Three portable habits from claude-plan-bridge (a Rust round-trip CLI) — not about types or proofs, just about building the tests so they can't quietly lie to you.

Dogfood a REAL artifact, not just hand-built minimal cases (the dogfood test). The unit tests feed the parser tidy two-line plans; this one feeds it a genuine mid-pivot PLAN.md from a different live repo and asserts it round-trips clean. A format that survives your own fixtures but mangles a real document has proven nothing. Honest caveat baked in: the sibling repo is opportunistic, so the test SKIPS when it isn't checked out — the one place I let a skip stay silent, because a cross-repo fixture can't be a hard build dependency.
Build cross-platform fixtures THROUGH the real serializer, never by hand (the stale-cwd e2e). This one cost a passed-locally, failed-only-on-Windows CI cycle. A test that hand-rolls a JSON settings file with format! and a Windows path (C:\proj\…) emits unescaped backslashes — \U, \A, \L are invalid JSON escapes that blow up ONLY on the Windows runner, and the fallout is silent (the file parses as nothing, every downstream assertion quietly reads the wrong state). Build the fixture through the real serializer (serde_json) so escaping is the code's job, branch the absolute paths on cfg!(windows), and let a tri-OS matrix (fail-fast: false) catch the class.
Scratch dirs that can't collide (test_utils.rs). The recipe is process-id + wall-clock nanos + an atomic counter, so a filesystem test can't collide under cargo test's 16-thread parallelism OR a cargo-watch loop firing before the OS reclaims the previous run's temp dir. Consolidated from 13 copies into one pub/#[doc(hidden)] helper — DRY reaches the scaffolding too. And the collision axis isn't only threads: nextest runs each #[test] in its OWN process, all sharing ONE CARGO_TARGET_TMPDIR — so a OnceLock-guarded fixture materializer (which serializes only WITHIN a process) lets N processes truncate-then-write the same files, and a reader catching a mid-write empty file flakes ~25% under load, invisible solo (fab-scad's loader corpus — pid-scope the fixture path; a process guard can't reach across processes). The fix earned its "fixed" from a CONTROLLED before/after under artificial CPU saturation — 12 yes hogs, 40 runs each, 10/40 flaked pre-fix, 0/40 post — because a load-dependent flake needs a load harness and a RATE, not a "seems better."

Language-specific

Same concerns, different cost. Python pays for them in custom tooling; Rust mostly gets them from the compiler.

Python — custom lints stand in for the compiler

One pytest module — test_typing_smells.py, the single most reusable file in this whole profile — holds an AST-lint framework: a Check base class, ast.NodeVisitor subclasses, a Smell record and a registry. No plugin packaging, runs anywhere pytest runs, reviewers read plain Python. The rules are project-specific by nature — a generic linter was never going to ship them. Two meta-rules keep the framework honest: suppressions require a 3+ word reason, and every check has a planted smoke test (an AST lint WILL silently break when the code idiom shifts under it; the plant is how you find out). Every check in the table below lives in that one file.

The catalog, grouped by the bug family it kills:

Family	Checks
Determinism	no unseeded `random.*` in generators; no `datetime.now()`/`date.today()` outside a tiny allowlist; no module-top-level randomness in test files (pytest-xdist workers collect DIFFERENT test sets — brutal to diagnose); `json.dumps` needs explicit `indent=` or `separators=`
Type discipline	ID-named params must use the NewType; no bare `Any`; env vars only through the typed registry (env_keys.py — raw `os.environ["MY_*"]` literals banned); no `float` in money modules
Boundaries	tests can't import the browser library outside the driver layer (an ~18-verb protocol — base.py — is the only surface); deleted dependencies stay deleted (a ban lint lands the same day the dep is removed); private registries unreachable around their public helpers
Drift	no ≥100-char string in `tests/` that also appears in `src/` (a copied SQL snippet keeps passing against yesterday's query — import the constant); no test assert against a literal matching a `src/` constant; no raw-string equality with enum values; prose lints ban retired-tech mentions and "in progress" narration in docstrings

Rust — the compiler is the first test

recon-gen is Python, and the honest question this catalog forces: which checks are genuinely mine, and which are custom tooling standing in for work a compiler does elsewhere? Roughly half of the Python catalog above melts away in Rust — which is the receipts behind my usual claim that strong typing is testing-before-the-test.

Python custom check	Rust equivalent	Cost there
NewType adoption lint	newtype structs — misuse is a COMPILE error	free
explicit-any lint	no gradual-typing escape hatch to police	free
why-comment on suppressions	`#[expect(lint, reason = "…")]` + deny `clippy::allow_attributes_without_reason` and `clippy::allow_attributes`	one config line
typed env registry bypass	`disallowed-methods = ["std::env::var"]` in clippy.toml, allowed inside the registry	config
determinism bans (`random`, `now()`)	`disallowed-methods` on `rand::random`, `SystemTime::now`, `Utc::now`	config
no-float-money	integer-cents newtype + deny `clippy::float_arithmetic` in money modules	config
deleted deps stay deleted	drop it from Cargo.toml and the import stops compiling	free
layer-boundary import lint	crate boundaries — the test crate doesn't DEPEND on the browser crate	free
private-registry reach-around	`pub(crate)` and private fields	free
raw-enum-equality in tests	closed enums + exhaustive `match`	free
dataclass eq/frozen footgun	equality is opt-in via `derive(PartialEq)`	free
type-checker session gate	compilation IS the gate	free
xdist collection nondeterminism	tests enumerated at compile time	doesn't exist
test-runs-nowhere reconciliation	`cargo test` compiles the tree (watch `cfg(feature)`-gated tests — the residual hole)	mostly free

Three of these rows have a sharp edge current clippy (1.96) will cut you on — each verified with a planted bad example, the profile's own standard turned on itself:

#[expect], not #[allow] — the intuitive recipe doesn't compile. Deny allow_attributes too (it's the dead-suppression detector), and #[allow(clippy::X, reason = "…")] still trips it WITH the reason attached. The only suppression that satisfies both lints is #[expect(clippy::X, reason = "…")] — which is strictly better anyway: it self-destructs via unfulfilled_lint_expectations the day it stops suppressing anything, so a suppression can't outlive the problem it was hiding.
disallowed-methods silently drops a path it can't resolve — a dep not yet in the graph, OR a typo — no error, exit 0, even under -D warnings. The upside: pre-banning a method from a dependency you haven't added yet is safe. The trap: a mistyped ban is green FOREVER, which is "a check that can't fail is broken" wearing a config hat. Every ban wants a planted call site that MUST fire. Same silent failure for placement — a clippy.toml not sitting next to Cargo.toml loads nothing, no warning.
clippy::float_arithmetic is auto-suppressed inside #[test] fns and #[cfg(test)] mods. A control lint fires there, so clippy IS running — the arithmetic family just gets a built-in carve-out. So the no-float-money lint has a hole exactly where a test helper does the money math, and that's the receipt behind leading the row with the integer newtype: a COMPILE error has no test carve-out. The lint is backup; the type is the guard.

The Rust CI shape that carries the rest — cargo clippy --all-targets -- -D warnings, fmt --check, a tri-OS test matrix — is live in claude-plan-bridge's ci.yml. One receipt on making that gate leave the machine: a clean COMPILE is not a clean push. Lint-ONLY clippy failures — doc_lazy_continuation, unnecessary_sort_by — pass cargo check and pass a normal test run, then redden the clippy job; on fab-scad they did it twice, once with a release tag one commit away. So the pre-push hook runs cargo clippy --workspace --all-targets -- -D warnings, not cargo check — clippy compiles too, so it's one pass for both the compile and the lint gate (the Overarching git-hooks bullet has the general form).

Two Rust-native test idioms show up in the checks elsewhere rather than here, filed by where they bite: cfg!(windows) fixture branches built through serde_json (overarching — "build fixtures through the serializer") and #[serde(default)] for forward-compatible persisted state (the durable-state domain below).

What stays custom in ANY language: the SQL-duplication lint, semantic locks, asserting against the emitted artifact, the mutation gate, generated-asset drift, CI-config linting, teardown-fail-loud — and the whole laws-and-proofs stack below. Those are domain checks; no compiler knows your DDL went stale.

Domain-specific

Checks you only build if your problem has the shape. Five shapes so far.

A rule that must exist as meaning AND query — generated SQL

This is the deep end, built for a domain where the same business rule must exist twice: what "drift" MEANS (a sentence) and the SQL that hunts for it across millions of rows, once per database engine. The two drift apart the moment you stop watching. The full story is in the formal-methods post; the design decision that shaped everything — exhaustive enumeration on the real engine, with the solver demoted to supporting jobs — is worked through in the tie spike doc. Here's the reusable method.

The claim vocabulary comes first (the signed spec). Four levels, strongest to weakest — and every verification claim names its level:

Claim	Quantifier	Checked object
Proven for all integers	every integer value, rows bounded	the production law function, symbolically executed
Proven on a domain	every DB state in a finite enumerated domain	the REAL emitted SQL on a real engine
Tested	the states actually exercised	fixtures, metamorphic transforms, property samples
Trusted with mitigation	none	a named dependency + its named mitigation

The layers, in the order you'd rebuild them:

One law function per invariant, authored from the WRITTEN rule — never transliterated from the SQL (residuals.py). An SQL-derived spec would verify the SQL against itself. The law ("residual") computes the rule's value over a small frozen-dataclass state model; the SQL detector stays the detector, the law is what it's checked against. Nothing else restates the rule.
Known-answer vectors, derived BY HAND (the vectors, the runner, the arithmetic shown). Each law gets JSON vectors whose expected values were worked out on paper from the prose rule. Independence is the point: a law bug and a test bug can't cancel unless the same mistake was made twice by different routes. Some vectors are born-divergent on purpose — they encode the DECIDED rule, disagree with today's SQL and become the red-first proof the fix worked.
The enumeration gate — the workhorse (harness.py, the gate). Emit the real schema, load an exhaustive set of boundary-derived states into an actual database, run the real detectors and assert the flagged rows are EXACTLY the rows the law calls violations — keys and values, every state, no sampling inside the quantifier and no model of the SQL anywhere. Only execution puts the engine's own behavior (NULL logic, refresh ordering, UNIQUE-index contracts) inside the verified object; the decisive early bug was caught not by any comparator but by a unique index blowing up at refresh time — a detection channel no hand-built model can contain.
Boundary values derived from RESOLVED config, not SQL text (domains/_base.py). Four detectors' thresholds never appear in the emitted SQL at all — they're config data joined at query time, so a text-based coverage check passes vacuously on exactly the detectors that matter. Derive every comparison constant from the loaded config and assert its ±1 neighborhood is in the test domain.
A mutation gate that scores the domain, not the code (mutations.py, the gate). A battery of named semantic corruptions (operator flips, day-boundary off-by-ones, status-filter narrowing AND widening, dropped supersession filters, config-resolution corruption) is applied to the emitted SQL; every mutant must be visibly caught. A survivor is a measured hole — you widen the domain, you never delete the mutant. Several domain axes exist only because a mutant forced them.
Metamorphic laws for what enumeration can't see (metamorphic.py, the driver). Enumeration checks states one at a time; some properties are relations BETWEEN states — insert-order permutation changes nothing, a failed leg is inert, superseding with identical content is a no-op, unrelated accounts can't interfere. Prove them against a base state where every detector already alarms, so "inert" is proven against live sets rather than empty ones.
Property-based sampling for the escape class (test_ds37_hypothesis_tail.py, topology fuzzer in fuzz.py). The grid is exhaustive but bounded; Hypothesis samples exactly what's outside it — int64-scale magnitudes, cardinalities above the enumerated counts and randomly-generated valid topologies. Derandomized, so a failure reproduces bit-identically.
The solver lane, scoped honestly (tests/prover/). The SAME law functions run through z3 unchanged — a whitelist lint (test_ds1_residual_lint.py) keeps them branch-free (all selection through one when(cond, a, b) combinator, no division or modulo since Python and SMT disagree on negative operands) and the symbolic adapter (symbolic.py) swaps exactly that combinator for z3.If. A dual-run canary re-runs every hand-derived vector both concrete and symbolic and asserts agreement, so the term the solver quantifies over provably computes the law. Then the theorems: inertness, idempotence, non-interference and scaling laws proven for ALL integers by handing z3 the negation and pinning unsat. Every obligation pins its expected verdict (solver.py) — a counterexample can never be xfail-ed, and solver timeouts raise their own exception type so a triage-approved xfail structurally cannot mask a real counterexample.
A structured waiver for the one non-integer detector (anomaly_contract.py). The z-score detector's law runs in exact rationals (variance as fractions, thresholds compared squared so no square root exists) with exactly two tolerance constants for the irreducibly-float edge, ordered so any row the engine could misround is classified band-edge by the law BEFORE the engine gets a vote. Bringing it into the framework caught a real cross-dialect bug on arrival: one engine's bare NUMERIC silently quantized the whole pipeline to 3 decimals — same SQL text, divergent semantics, invisible to every prior test.
Completeness by annotation + cross-check, not a registry (math_invariant.py, the gate). Registries fail silently. Each detector class carries an annotation naming its law, vectors and matview; one gate direction walks the AST (every detector class is annotated), the other walks the emitted schema (every emitted matview is annotated or on an explicit plumbing list with a reason). The default for anything unlisted is a build failure — that's how a detector that had shipped its whole life with zero coverage got found.
Semantic locks as the regression floor (the check, the locks). The committed snapshot is the violation SET each detector finds at a pinned anchor date — the absolute record of what the system finds at a known-good point, so a law shift can't land without a deliberate re-lock.

What none of this proves

Stated plainly because the limits are part of the method:

No production database engine is formally verified — the engines are trusted with a named mitigation (multi-renderer agreement + the enumeration gate running THROUGH them).
You cannot have unbounded values, unbounded rows and the real engine simultaneously. Unbounded-row SQL equivalence is undecidable; the for-all-integers claims bound row count, the on-domain claims bound the domain.
The solver lane covers the money family only — counting, thresholding and graph-walk laws get near-vacuous solver theorems, so they lean on enumeration instead. Scaling laws are proven at concrete scalars (a symbolic scalar is nonlinear, outside the decidable fragment).
Mutation coverage is an argument, not a theorem — the battery is as good as the corruptions you thought to name.

Round-trip formats + durable state

claude-plan-bridge's shape: a document that must survive parse → edit → serialize → re-parse without drift, plus a state file that outlives its own schema. Two checks that only exist if your problem looks like that.

Round-trip stability, asserted on the AST not the bytes (the harness). The property that matters is parse → serialize → parse landing the SAME tree, NOT the same string. roundtrip_stable compares the re-parsed AST (plan1 == plan2), so the serializer stays free to normalize (collapse a bare space to the canonical - separator, coalesce blank runs) without the test crying wolf — it fires ONLY when a round-trip actually drops or changes meaning. Byte-exact assertions still exist, but only for the canonical-output tests where the exact string IS the contract. Same "lock the meaning, not the bytes" as the semantic locks above, one layer shallower.
A persisted struct evolves by serde-default, and you TEST the old shape loading (state.rs). Every field on the state struct is #[serde(default, skip_serializing_if = …)]: an old file missing today's field loads clean (forward-compat), a fresh file writes no dead keys. The proof is a test that loads a hand-written OLD-schema file and asserts the absent field defaults sanely (legacy_state_file_without_active_phase_field_loads_clean) — though a negative control run while adopting this elsewhere caught the test passing for the wrong reason: it guards an Option field, and serde defaults a missing Option to None on its own, so deleting the #[serde(default)] it appears to protect leaves it GREEN. The attribute earns its keep on the NON-Option fields; a sharper fixture omits one of THOSE. (A check that passes for the wrong reason is exactly what the principles up top name — and this one was caught on my own code, which is the point of running the control.) What I deliberately do NOT reach for is deny_unknown_fields — it turns every future field into a hard break for an older reader. And the escape hatch when a state file DOES corrupt: it's bridge-owned and rebuildable from PLAN.md (baseline), so a bad one is never fatal.

Bit-identical output — determinism as a designed property

When the same input must produce the same BYTES — run-to-run, machine-to-machine, and fast-path == slow-path if you have execution tiers — determinism stops being something you hope emerges and becomes something you engineer at a seam. fab-scad's geometry kernel MUST render byte-identical (a differential harness and a content-addressed cache both assume it); the C++ predecessor did not — one model rendered three different STL hashes across three runs at identical volume and genus (a non-total-order comparator plus atomic-slot races in the parallel assembly, unreachable from outside the library). The pure-Rust reimplementation designed the class out, then proved it two independent ways.

One parallelism door, gated by construction (par.rs). rayon is lint-banned everywhere else, and the one door is TYPED for determinism: a reducer must carry a CommutativeAssociative marker to run in parallel, so a non-associative float sum WON'T COMPILE — it routes to a fixed-order Kahan serial fold instead. Sorts take total-order comparators (morton.then(original_index), sort.rs); the output pass canonicalizes vertices on POSITION so provenance ids never leak into the emitted bytes. Nondeterminism has nowhere to enter.
A run-to-run byte guard, kept (determinism_render.rs). By-construction is only as strong as the proof nobody opened a SECOND door, so a standing test renders the same op-tree N times under full parallelism and asserts one hash — on the actual nasty models (non-convex, cavity-riddled), never toy cubes.
An adversarial audit for the load-bearing claim. Because "deterministic" underwrites the whole differential story, the byte check was BACKED by a fan-out of finders each hunting one nondeterminism class (unordered iteration, un-gated parallel reduce, non-total-order sort, a global atomic leaking into output, float), each candidate then run past an independent skeptic prompted to REFUTE it. Zero survived — and the audit caught the one comparator the byte check hadn't stressed (a piece-ordering sort that held only because its input pass was serial), which was hardened to a self-contained total order. (Whether "an important claim gets a second adversarial pass" is a profile PRACTICE or a mechanism is an open question in the sidecar.)
Tier bit-identity is the same claim one level down. A JIT that must match its interpreter is a determinism obligation ACROSS tiers; a differential fuzzer (jit_diff) asserts the two produce bitwise-equal output — with a NaN-class carve-out, because any NaN ≡ any NaN for an info-carrying value, so the compare goes through a tier_eq helper, never raw to_bits (a Cranelift constant-fold (-s)*(-s)→s*s surfaced exactly there). This is also the ONLY tool that runs the JIT's native code — see the reimplementation shape next for why the MIR tools can't.

Reimplementing an existing system — the original is your oracle

If you're rebuilding something that already exists (a language, a format, a protocol), the reference implementation is a free, comprehensive oracle — point everything you have at DISAGREEING with it. fab-scad reimplements OpenSCAD, and the whole correctness story is differential.

Sweep the reference's OWN test suite through your reimplementation; a divergence is the WIN, not the failure. Run the upstream corpus (here, a large third-party OpenSCAD library's .scadtest files) through your engine, triage every mismatch into a NAMED bucket, and land each fix as a regression test in your own semantics/ tree. You're mining the reference for the behaviors you got wrong — success is finding one.
Know which verifier can REACH your native code. A reimplementation with a JIT or FFI carries unsafe the type-level and MIR-level tools can't touch: miri and Kani both operate over MIR, so NEITHER can execute a Cranelift-emitted function or a C++ FFI call — the actual native seam. Only fuzzing runs it — the JIT-vs-interpreter differential (jit_diff) under ASan is the one tool that puts the emitted machine code on the stand. Pick the verifier by the surface it reaches, and name the surfaces each one CAN'T: miri for Rust UB it can see, Kani for bounded proofs over MIR, fuzz+ASan for the native seam.

A GUI — the compile gate can't see a frame

A suite that compiles and asserts on state stays green while the UI renders garbage or panics on a frame it never drew. A GUI needs a verifier that actually RENDERS.

Screenshot-verify UI changes, and know which capture path tells the truth (screenshot.rs). A one-resource change (a new Bevy resource for a console panel) panicked the panel system on first frame — Bevy validates a system's parameters at RUNTIME, and the resource was left un-initialized in one of three app builders — yet the 866-test gate passed CLEAN, because the headless harness tests don't run that system. The offscreen --screenshot caught it. The capture path is load-bearing too: a windowed screenshot from a CLI-spawned window came back BLACK; the offscreen render is the one that draws the real UI. GUI changes get a rendered-frame check, not just a green compile.

New Project Starter

In order of payoff:

Strict type checking (or -D warnings) from the first commit, wired to run before any test does.
The AST-lint test module with the suppressions-need-a-WHY rule — grow it one check at a time, each with its planted smoke test.
NewType the ID strings, put env vars behind a typed registry, make money an integer.
The determinism family the moment anything is snapshot-diffed.
Teardown-fail-loud, then the completeness gates once the harness has layers.
If a rule must exist as both a meaning and a query: one law function from the prose, hand-derived vectors, then the enumeration gate on the real engine. The solver comes LAST, scoped to what it can actually decide.
If you parse and re-emit a format, or persist state across versions: round-trip on the AST not the bytes, and give every persisted struct a forward-compatible default plus a test that loads the old shape.
If output must reproduce bit-for-bit: make the nondeterminism a compile-gate (one parallelism door, a marker trait that refuses non-associative reduces, total-order comparators), not a hope — then keep a run-to-run byte check on your real inputs.
If you reimplement an existing tool: its own test suite is your oracle — sweep it through, success is a divergence. And if you emit native code (a JIT, FFI), remember miri/Kani stop at MIR; only fuzz+ASan runs it.
If it's a GUI: screenshot-verify. A compile-and-state gate stays green through a mis-rendered or panicking frame.

Cover Image from lecroitg, CC0, via Wikimedia Commons

v0.4 — 2026-07-19. First source with a JIT/FFI native surface, a reference-implementation oracle, a GUI, and a determinism doctrine — fab-scad / scad-rs (a pure-Rust OpenSCAD-over-Manifold reimplementation, links pinned to web-v0.19.0). Added a principle (bit-identity is DESIGNED, not hoped) and three Domain shapes — deterministic output (a compile-gated parallelism seam + a run-to-run byte guard + an adversarial audit), reimplementing an existing system (the original as oracle; miri/Kani stop at MIR, only fuzz+ASan reaches a JIT/FFI seam), and a GUI (screenshot-verify — a runtime resource-validation panic passed 866 tests, the offscreen shot caught it). Sharpened three existing rows: harness-integrity (a fuzz campaign dead at BUILD is testing nothing), scratch-dir isolation (nextest's process-per-test shares one CARGOTTARGET—PDIR — proven 10/40→0/40 under load), and the Rust CI shape (pre-push runs clippy -D warnings, not bare check; lint-only failures reddened CI twice). Receipts in the sidecar. v0.3 — 2026-07-10. First source that VERIFIED the profile instead of feeding it — job-board-mcp adopted the Rust mapping and ran a planted bad example per row on clippy 1.96.1. Corrected the suppression recipe (#[expect], not #[allow] — the intuitive form doesn't compile), added two Rust caveats (disallowed-methods silent-drops unresolved/typo'd paths; float_arithmetic auto-off inside #[test]), and flagged that the plan-bridge legacy-load test passes for the wrong reason (guards an Option, which serde defaults on its own). Receipts in the sidecar. v0.2 — 2026-07-05. Reorganized into overarching / language-specific / domain-specific (was Layers 0–3) so language and domain guidance can fan out cleanly. Folded in the claude-plan-bridge lessons — round-trip AST-stability, forward-compatible persisted state, cross-platform fixtures, real-artifact dogfood, scratch-dir isolation — links pinned to v1.2.0. v0.1 — 2026-07-04. Sources: recon-gen (Python, the bulk of it), claude-plan-bridge and FeOphant (the Rust side). Updated in place; changes logged here.