I keep a voice profile so my writing stays mine no matter who drafts it. This is the same idea for testing: the operational summary of every check, lint, gate and proof layer I've had to build, kept current so the next project starts with them instead of re-earning them one incident at a time. Everything below exists because something real broke, and every mechanism links to its implementation — in recon-gen or claude-plan-bridge — so you can see exactly how it's done, not just that it should be.

This is a LIVING page — I update it in place as projects teach me new checks, version footer at the bottom. It's organized by how far a check travels: the overarching ones first, then what's specific to a language, then what's specific to a problem domain.

The principles

Every check below is an instance of one of these:

• One source of truth, verified against the EMITTED artifact. Duplicates drift. When a law or schema must exist twice (the meaning and the SQL that hunts for it), machine-check the bridge — and check the real emitted output, never a re-read of the source that emitted it.
• Fail loud, never silent. A teardown failure fails the build. A skipped test is gated. An unannotated detector is a hard error, not a missing row nobody counts.
• Lock the meaning, not the bytes. Byte-snapshots cry wolf on every harmless refactor; a snapshot of what the system FOUND fires exactly when behavior changed.
• Every claim carries a quantifier. "Proven" with no domain attached is a doc bug. The vocabulary I use: proven for all integers, proven on an enumerated domain, tested on the states exercised or trusted with a named mitigation.
• A check that can't fail is broken. Every custom lint ships a planted bad example proving it fires; the test domain itself is scored by a mutation gate. Adequacy is measured, not assumed.
• Suppressions carry a WHY. An ignore comment without a reason is a decision with the rationale thrown away — and a lint enforces that rule on the other lints' ignores.

Overarching — any language, any domain

The checks that travel regardless of stack or problem. Ordered cheapest-first, author-time before the harness.

Type discipline — the cheapest layer

Runs at author time, and in a typed-enough language most of it is free (the Rust section below has the receipts). The concept is universal; what it COSTS in Python vs Rust is in the language sections.

• NewType every identifier string (ids.py). SheetId, VisualId, ParameterName — function params, dict keys and dataclass fields take the wrapper, never bare str. Zero runtime cost; a swapped argument becomes a type error instead of a zero-row query.
• Money is an integer type end-to-end (money.py). Native cents, no float anywhere on the money path.
• Validate at construction, not in a test (controls.py is a dense example). __post_init__ raises on a blank description, an out-of-range value list, a control bound to the wrong parameter shape — the invariant fails at the wiring site with a stack trace, not three layers later in generated output.
• Equality is a decision. A bare @dataclass silently gives structural equality on mutable state; tree nodes must declare frozen=True or eq=False. Closed enums over status strings, exhaustive handling over stringly-typed branches.
• Strict type checking scoped by an include list that only grows (pyproject.toml, [tool.pyright]). New files enter strict scope; nothing leaves. Explicit Any needs a WHY comment.

Harness integrity — the tests need testing

This layer watches the harness, the fixtures and the CI config.

• Session-start gates (conftest.py). The type checker runs as a subprocess before any test; a type error kills the session with the same verdict CI would give. Generated assets (the built CSS) are rebuilt to temp and byte-diffed against the committed copy (build_app2_css.py) — with an honest platform limit: minifier output isn't byte-identical across OSes, so that one gate is local-only.
• Fail-loud plumbing (tests/conftest.py). Fixture teardown failures collect per-worker and force a failing exit at session end — a silent teardown failure doesn't hurt this run, it poisons the NEXT one. Tests without a tier mark error at collection. A skip or xfail touching the canonical example fixture fails the build unless its exemption reason starts with an approved prefix.
• Completeness gates. Drive the runner's real per-layer selectors through pytest --collect-only and assert every e2e file is claimed by at least one layer (test_layer_coverage_reconciliation.py) — a test that runs nowhere is green forever. Every importorskip must name a dependency actually declared in pyproject (test_importorskip_declared.py), or the module skips everywhere silently.
• CI config is code, so lint it. A unit test parses the release workflow and asserts every job runs on hosted runners (test_release_yml_secret_isolation.py) — publish secrets never touch self-hosted; exceptions live in an allowlist requiring a one-line WHY. faulthandler_timeout stays on so a hang dumps every thread's stack instead of eating the job's time limit.
• Git hooks for the friction loops (.githooks/). Pre-commit rebuilds and re-stages generated assets when their sources are staged; pre-push runs the cheapest full tier (~3-4 min, no containers) before a CI cycle gets burned.

Building the tests themselves

Three portable habits from claude-plan-bridge (a Rust round-trip CLI) — not about types or proofs, just about building the tests so they can't quietly lie to you.

• Dogfood a REAL artifact, not just hand-built minimal cases (the dogfood test). The unit tests feed the parser tidy two-line plans; this one feeds it a genuine mid-pivot PLAN.md from a different live repo and asserts it round-trips clean. A format that survives your own fixtures but mangles a real document has proven nothing. Honest caveat baked in: the sibling repo is opportunistic, so the test SKIPS when it isn't checked out — the one place I let a skip stay silent, because a cross-repo fixture can't be a hard build dependency.
• Build cross-platform fixtures THROUGH the real serializer, never by hand (the stale-cwd e2e). This one cost a passed-locally, failed-only-on-Windows CI cycle. A test that hand-rolls a JSON settings file with format! and a Windows path (C:\proj\…) emits unescaped backslashes — \U, \A, \L are invalid JSON escapes that blow up ONLY on the Windows runner, and the fallout is silent (the file parses as nothing, every downstream assertion quietly reads the wrong state). Build the fixture through the real serializer (serde_json) so escaping is the code's job, branch the absolute paths on cfg!(windows), and let a tri-OS matrix (fail-fast: false) catch the class.
• Scratch dirs that can't collide (test_utils.rs). The recipe is process-id + wall-clock nanos + an atomic counter, so a filesystem test can't collide under cargo test's 16-thread parallelism OR a cargo-watch loop firing before the OS reclaims the previous run's temp dir. Consolidated from 13 copies into one pub/#[doc(hidden)] helper — DRY reaches the scaffolding too.

Language-specific

Same concerns, different cost. Python pays for them in custom tooling; Rust mostly gets them from the compiler.

Python — custom lints stand in for the compiler

One pytest module — test_typing_smells.py, the single most reusable file in this whole profile — holds an AST-lint framework: a Check base class, ast.NodeVisitor subclasses, a Smell record and a registry. No plugin packaging, runs anywhere pytest runs, reviewers read plain Python. The rules are project-specific by nature — a generic linter was never going to ship them. Two meta-rules keep the framework honest: suppressions require a 3+ word reason, and every check has a planted smoke test (an AST lint WILL silently break when the code idiom shifts under it; the plant is how you find out). Every check in the table below lives in that one file.

The catalog, grouped by the bug family it kills:

Family	Checks
Determinism	no unseeded random.* in generators; no datetime.now()/date.today() outside a tiny allowlist; no module-top-level randomness in test files (pytest-xdist workers collect DIFFERENT test sets — brutal to diagnose); json.dumps needs explicit indent= or separators=
Type discipline	ID-named params must use the NewType; no bare Any; env vars only through the typed registry (env_keys.py — raw os.environ["MY_*"] literals banned); no float in money modules
Boundaries	tests can't import the browser library outside the driver layer (an ~18-verb protocol — base.py — is the only surface); deleted dependencies stay deleted (a ban lint lands the same day the dep is removed); private registries unreachable around their public helpers
Drift	no ≥100-char string in tests/ that also appears in src/ (a copied SQL snippet keeps passing against yesterday's query — import the constant); no test assert against a literal matching a src/ constant; no raw-string equality with enum values; prose lints ban retired-tech mentions and "in progress" narration in docstrings

Rust — the compiler is the first test

recon-gen is Python, and the honest question this catalog forces: which checks are genuinely mine, and which are custom tooling standing in for work a compiler does elsewhere? Roughly half of the Python catalog above melts away in Rust — which is the receipts behind my usual claim that strong typing is testing-before-the-test.

Python custom check	Rust equivalent	Cost there
NewType adoption lint	newtype structs — misuse is a COMPILE error	free
explicit-any lint	no gradual-typing escape hatch to police	free
why-comment on suppressions	#[expect(lint, reason = "…")] + deny clippy::allow_attributes_without_reason and clippy::allow_attributes	one config line
typed env registry bypass	disallowed-methods = ["std::env::var"] in clippy.toml, allowed inside the registry	config
determinism bans (random, now())	disallowed-methods on rand::random, SystemTime::now, Utc::now	config
no-float-money	integer-cents newtype + deny clippy::float_arithmetic in money modules	config
deleted deps stay deleted	drop it from Cargo.toml and the import stops compiling	free
layer-boundary import lint	crate boundaries — the test crate doesn't DEPEND on the browser crate	free
private-registry reach-around	pub(crate) and private fields	free
raw-enum-equality in tests	closed enums + exhaustive match	free
dataclass eq/frozen footgun	equality is opt-in via derive(PartialEq)	free
type-checker session gate	compilation IS the gate	free
xdist collection nondeterminism	tests enumerated at compile time	doesn't exist
test-runs-nowhere reconciliation	cargo test compiles the tree (watch cfg(feature)-gated tests — the residual hole)	mostly free

Three of these rows have a sharp edge current clippy (1.96) will cut you on — each verified with a planted bad example, the profile's own standard turned on itself:

• #[expect], not #[allow] — the intuitive recipe doesn't compile. Deny allow_attributes too (it's the dead-suppression detector), and #[allow(clippy::X, reason = "…")] still trips it WITH the reason attached. The only suppression that satisfies both lints is #[expect(clippy::X, reason = "…")] — which is strictly better anyway: it self-destructs via unfulfilled_lint_expectations the day it stops suppressing anything, so a suppression can't outlive the problem it was hiding.
• disallowed-methods silently drops a path it can't resolve — a dep not yet in the graph, OR a typo — no error, exit 0, even under -D warnings. The upside: pre-banning a method from a dependency you haven't added yet is safe. The trap: a mistyped ban is green FOREVER, which is "a check that can't fail is broken" wearing a config hat. Every ban wants a planted call site that MUST fire. Same silent failure for placement — a clippy.toml not sitting next to Cargo.toml loads nothing, no warning.
• clippy::float_arithmetic is auto-suppressed inside #[test] fns and #[cfg(test)] mods. A control lint fires there, so clippy IS running — the arithmetic family just gets a built-in carve-out. So the no-float-money lint has a hole exactly where a test helper does the money math, and that's the receipt behind leading the row with the integer newtype: a COMPILE error has no test carve-out. The lint is backup; the type is the guard.

The Rust CI shape that carries the rest — cargo clippy --all-targets -- -D warnings, fmt --check, a tri-OS test matrix — is live in claude-plan-bridge's ci.yml.

Two Rust-native test idioms show up in the checks elsewhere rather than here, filed by where they bite: cfg!(windows) fixture branches built through serde_json (overarching — "build fixtures through the serializer") and #[serde(default)] for forward-compatible persisted state (the durable-state domain below).

What stays custom in ANY language: the SQL-duplication lint, semantic locks, asserting against the emitted artifact, the mutation gate, generated-asset drift, CI-config linting, teardown-fail-loud — and the whole laws-and-proofs stack below. Those are domain checks; no compiler knows your DDL went stale.

Domain-specific

Checks you only build if your problem has the shape. Two shapes so far.

A rule that must exist as meaning AND query — generated SQL

This is the deep end, built for a domain where the same business rule must exist twice: what "drift" MEANS (a sentence) and the SQL that hunts for it across millions of rows, once per database engine. The two drift apart the moment you stop watching. The full story is in the formal-methods post; the design decision that shaped everything — exhaustive enumeration on the real engine, with the solver demoted to supporting jobs — is worked through in the tie spike doc. Here's the reusable method.

The claim vocabulary comes first (the signed spec). Four levels, strongest to weakest — and every verification claim names its level:

Claim	Quantifier	Checked object
Proven for all integers	every integer value, rows bounded	the production law function, symbolically executed
Proven on a domain	every DB state in a finite enumerated domain	the REAL emitted SQL on a real engine
Tested	the states actually exercised	fixtures, metamorphic transforms, property samples
Trusted with mitigation	none	a named dependency + its named mitigation

The layers, in the order you'd rebuild them:

1. One law function per invariant, authored from the WRITTEN rule — never transliterated from the SQL (residuals.py). An SQL-derived spec would verify the SQL against itself. The law ("residual") computes the rule's value over a small frozen-dataclass state model; the SQL detector stays the detector, the law is what it's checked against. Nothing else restates the rule.
2. Known-answer vectors, derived BY HAND (the vectors, the runner, the arithmetic shown). Each law gets JSON vectors whose expected values were worked out on paper from the prose rule. Independence is the point: a law bug and a test bug can't cancel unless the same mistake was made twice by different routes. Some vectors are born-divergent on purpose — they encode the DECIDED rule, disagree with today's SQL and become the red-first proof the fix worked.
3. The enumeration gate — the workhorse (harness.py, the gate). Emit the real schema, load an exhaustive set of boundary-derived states into an actual database, run the real detectors and assert the flagged rows are EXACTLY the rows the law calls violations — keys and values, every state, no sampling inside the quantifier and no model of the SQL anywhere. Only execution puts the engine's own behavior (NULL logic, refresh ordering, UNIQUE-index contracts) inside the verified object; the decisive early bug was caught not by any comparator but by a unique index blowing up at refresh time — a detection channel no hand-built model can contain.
4. Boundary values derived from RESOLVED config, not SQL text (domains/_base.py). Four detectors' thresholds never appear in the emitted SQL at all — they're config data joined at query time, so a text-based coverage check passes vacuously on exactly the detectors that matter. Derive every comparison constant from the loaded config and assert its ±1 neighborhood is in the test domain.
5. A mutation gate that scores the domain, not the code (mutations.py, the gate). A battery of named semantic corruptions (operator flips, day-boundary off-by-ones, status-filter narrowing AND widening, dropped supersession filters, config-resolution corruption) is applied to the emitted SQL; every mutant must be visibly caught. A survivor is a measured hole — you widen the domain, you never delete the mutant. Several domain axes exist only because a mutant forced them.
6. Metamorphic laws for what enumeration can't see (metamorphic.py, the driver). Enumeration checks states one at a time; some properties are relations BETWEEN states — insert-order permutation changes nothing, a failed leg is inert, superseding with identical content is a no-op, unrelated accounts can't interfere. Prove them against a base state where every detector already alarms, so "inert" is proven against live sets rather than empty ones.
7. Property-based sampling for the escape class (test_ds37_hypothesis_tail.py, topology fuzzer in fuzz.py). The grid is exhaustive but bounded; Hypothesis samples exactly what's outside it — int64-scale magnitudes, cardinalities above the enumerated counts and randomly-generated valid topologies. Derandomized, so a failure reproduces bit-identically.
8. The solver lane, scoped honestly (tests/prover/). The SAME law functions run through z3 unchanged — a whitelist lint (test_ds1_residual_lint.py) keeps them branch-free (all selection through one when(cond, a, b) combinator, no division or modulo since Python and SMT disagree on negative operands) and the symbolic adapter (symbolic.py) swaps exactly that combinator for z3.If. A dual-run canary re-runs every hand-derived vector both concrete and symbolic and asserts agreement, so the term the solver quantifies over provably computes the law. Then the theorems: inertness, idempotence, non-interference and scaling laws proven for ALL integers by handing z3 the negation and pinning unsat. Every obligation pins its expected verdict (solver.py) — a counterexample can never be xfail-ed, and solver timeouts raise their own exception type so a triage-approved xfail structurally cannot mask a real counterexample.
9. A structured waiver for the one non-integer detector (anomaly_contract.py). The z-score detector's law runs in exact rationals (variance as fractions, thresholds compared squared so no square root exists) with exactly two tolerance constants for the irreducibly-float edge, ordered so any row the engine could misround is classified band-edge by the law BEFORE the engine gets a vote. Bringing it into the framework caught a real cross-dialect bug on arrival: one engine's bare NUMERIC silently quantized the whole pipeline to 3 decimals — same SQL text, divergent semantics, invisible to every prior test.
10. Completeness by annotation + cross-check, not a registry (math_invariant.py, the gate). Registries fail silently. Each detector class carries an annotation naming its law, vectors and matview; one gate direction walks the AST (every detector class is annotated), the other walks the emitted schema (every emitted matview is annotated or on an explicit plumbing list with a reason). The default for anything unlisted is a build failure — that's how a detector that had shipped its whole life with zero coverage got found.
11. Semantic locks as the regression floor (the check, the locks). The committed snapshot is the violation SET each detector finds at a pinned anchor date — the absolute record of what the system finds at a known-good point, so a law shift can't land without a deliberate re-lock.

What none of this proves

Stated plainly because the limits are part of the method:

• No production database engine is formally verified — the engines are trusted with a named mitigation (multi-renderer agreement + the enumeration gate running THROUGH them).
• You cannot have unbounded values, unbounded rows and the real engine simultaneously. Unbounded-row SQL equivalence is undecidable; the for-all-integers claims bound row count, the on-domain claims bound the domain.
• The solver lane covers the money family only — counting, thresholding and graph-walk laws get near-vacuous solver theorems, so they lean on enumeration instead. Scaling laws are proven at concrete scalars (a symbolic scalar is nonlinear, outside the decidable fragment).
• Mutation coverage is an argument, not a theorem — the battery is as good as the corruptions you thought to name.

Round-trip formats + durable state

claude-plan-bridge's shape: a document that must survive parse → edit → serialize → re-parse without drift, plus a state file that outlives its own schema. Two checks that only exist if your problem looks like that.

• Round-trip stability, asserted on the AST not the bytes (the harness). The property that matters is parse → serialize → parse landing the SAME tree, NOT the same string. roundtrip_stable compares the re-parsed AST (plan1 == plan2), so the serializer stays free to normalize (collapse a bare space to the canonical - separator, coalesce blank runs) without the test crying wolf — it fires ONLY when a round-trip actually drops or changes meaning. Byte-exact assertions still exist, but only for the canonical-output tests where the exact string IS the contract. Same "lock the meaning, not the bytes" as the semantic locks above, one layer shallower.
• A persisted struct evolves by serde-default, and you TEST the old shape loading (state.rs). Every field on the state struct is #[serde(default, skip_serializing_if = …)]: an old file missing today's field loads clean (forward-compat), a fresh file writes no dead keys. The proof is a test that loads a hand-written OLD-schema file and asserts the absent field defaults sanely (legacy_state_file_without_active_phase_field_loads_clean) — though a negative control run while adopting this elsewhere caught the test passing for the wrong reason: it guards an Option field, and serde defaults a missing Option to None on its own, so deleting the #[serde(default)] it appears to protect leaves it GREEN. The attribute earns its keep on the NON-Option fields; a sharper fixture omits one of THOSE. (A check that passes for the wrong reason is exactly what the principles up top name — and this one was caught on my own code, which is the point of running the control.) What I deliberately do NOT reach for is deny_unknown_fields — it turns every future field into a hard break for an older reader. And the escape hatch when a state file DOES corrupt: it's bridge-owned and rebuildable from PLAN.md (baseline), so a bad one is never fatal.

New Project Starter

In order of payoff:

1. Strict type checking (or -D warnings) from the first commit, wired to run before any test does.
2. The AST-lint test module with the suppressions-need-a-WHY rule — grow it one check at a time, each with its planted smoke test.
3. NewType the ID strings, put env vars behind a typed registry, make money an integer.
4. The determinism family the moment anything is snapshot-diffed.
5. Teardown-fail-loud, then the completeness gates once the harness has layers.
6. If a rule must exist as both a meaning and a query: one law function from the prose, hand-derived vectors, then the enumeration gate on the real engine. The solver comes LAST, scoped to what it can actually decide.
7. If you parse and re-emit a format, or persist state across versions: round-trip on the AST not the bytes, and give every persisted struct a forward-compatible default plus a test that loads the old shape.

Cover Image from lecroitg, CC0, via Wikimedia Commons

---

v0.3 — 2026-07-10. First source that VERIFIED the profile instead of feeding it — job-board-mcp adopted the Rust mapping and ran a planted bad example per row on clippy 1.96.1. Corrected the suppression recipe (#[expect], not #[allow] — the intuitive form doesn't compile), added two Rust caveats (disallowed-methods silent-drops unresolved/typo'd paths; float_arithmetic auto-off inside #[test]), and flagged that the plan-bridge legacy-load test passes for the wrong reason (guards an Option, which serde defaults on its own). Receipts in the sidecar. v0.2 — 2026-07-05. Reorganized into overarching / language-specific / domain-specific (was Layers 0–3) so language and domain guidance can fan out cleanly. Folded in the claude-plan-bridge lessons — round-trip AST-stability, forward-compatible persisted state, cross-platform fixtures, real-artifact dogfood, scratch-dir isolation — links pinned to v1.2.0. v0.1 — 2026-07-04. Sources: recon-gen (Python, the bulk of it), claude-plan-bridge and FeOphant (the Rust side). Updated in place; changes logged here.