Hardening: Overview

The opening of Part III: given the Core Model’s quantities, how do you engineer good values of them? One objective — eliminate confident error — and six families of mechanism that achieve it, each developed on its own page. This overview states the objective, maps the families onto the two axes, gives the threat-coverage picture and the composition algebra that span families, and notes the machine affordances that let every mechanism run at superhuman intensity.

Note

Opens Part III. The six-family taxonomy is a working organizing scheme, not a settled one; families overlap and several constructions pull more than one lever. Grades follow The Core Model, plus [speculative] for constructions with no precedent.

The objective: eliminate confident error

The only epistemic state that can hurt a judge without bound is being confidently wrong. A calibrated process turns that open-ended tail risk into a bounded one the judge can price, so the safety-relevant target is tail calibration, $P(\text{false}\mid \text{asserts} \ge c)$ for $c$ near 1, rather than accuracy. A process can always hedge or abstain, so “never confidently wrong” is achievable where “accurate” often is not [heuristic]. Calibration develops this, including its limit: a deceiver that lies rarely, on the decision that matters, can stay calibrated everywhere else. Every family below is a way to make confident error rare or expensive.

Two axes, six families

A hardening mechanism moves a process on one of the two axes — it makes validated information cheaper, or it makes the output costlier to corrupt. The six families sort onto those axes, and the sort explains the Catalogue’s empty quadrant:

Family	What it does	Primarily buys
Calibration	make stated confidence match reality	cheaper trustworthy bits (the output self-rates)
Verifiability asymmetry	make checking cheaper than producing	cheaper validated bits
Independence / decorrelation	manufacture genuinely independent signals	cheaper validated bits + higher corruption cost
Invariance / low sensitivity	output insensitive to controllable inputs	higher corruption cost
Incentive-compatibility	make honesty the best strategy	higher corruption cost
Deterrence	off-equilibrium penalties	higher corruption cost at low information cost

The non-obvious structural point: most corruption-resistance is bought with verification, which costs real information-extraction work every time the process runs — so it raises both axes together and never reaches the cheap-and-incorruptible corner. Deterrence is the partial exception: bonds, clawbacks, and tamper-evidence cost little on the honest path beyond the audits that detect cheating, which makes it the most promising route into the empty quadrant, and makes identity/track-record infrastructure (which sets the maximum enforceable penalty) unusually leveraged. Each family is developed on its own page, with its mechanism, its limit, its constructions, and worked bounds.

What each family defends — and the gaps

Sorting the families by the threats they close shows both the coverage and, more usefully, where there is none. (Primary defenders listed; many threats are addressed weakly by several families.)

Threat	Primary defending families	Where coverage is thin
Sybil / identity forgery	Deterrence, Invariance	needs identity infrastructure to bind bonds
Prompt-injection / framing	Invariance, Calibration	—
Persuasion / sycophancy	Incentive-compatibility, Verifiability, Calibration	claims that never resolve
Collusion	Independence, Incentive-compatibility	shared-training correlation
Sandbagging / capability-hiding	Independence, affordances (read-state)	interpretability is unreliable
Resolution-criteria gaming	Deterrence, Incentive-compatibility	who defines the criteria
Selection / optional-stopping	Verifiability, Deterrence	—
Correlated error	Independence (weakly)	no clean AI-specific defense yet
Goodhart-after-publication	Deterrence, externally-grounded families	style-mimicry of high-tier forms

Coverage leans on the corruption-cost axis: 9 of the 15 primary marks belong to Invariance, Incentives and Deterrence, 4 to Deterrence alone. Correlated error is the one threat with no primary defender; Independence, the family meant to supply decorrelation, is named only weakly. Blank cells mean “not a primary defender”, not “no effect”: the page notes that many threats are addressed weakly by several families.

The standout is correlated error. Every aggregation, ensemble and panel defense assumes independence, and shared training pipelines erode it. Engineering has met this before, in N-version programming (Knight & Leveson 1986), and prices it with common-cause models such as the beta factor (Independence has the details). Those models measure correlation; they do not remove it. No family here removes it for AI reasoners, which makes this row a research priority.

Composition: families stack, but how?

The family pages treat constructions as atoms, but real systems are stacks. Model a hardened stack as an AND/OR attack tree (Mauw & Oostdijk 2005): an attacker needs one successful path, so the stack’s corruption cost is the cheapest path, with costs summed over AND nodes and minimized over OR nodes. A pipeline in which corrupting any one stage corrupts the output is an OR node, so without randomization it inherits the cheapest stage; checks that must all be defeated form an AND node, and their costs add. Randomized escalation with probability $p$ to an expensive stage adds roughly $p\cdot C_{\text{expensive}}$ to every path that must survive escalation; this is the inspection-game frontier made compositional, and the same mechanism the Process Catalogue bets on for the empty quadrant. Whether real stacks behave this way is unmeasured.

The cross-cutting multiplier: machine-oversight affordances

The Process Catalogue lists processes evolved for humans — who have rights, privacy, fatigue, and exist as single un-forkable instances. None bind for an AI reasoner, and the slack may be the largest untapped resource here. The affordances are not a seventh family; each lets you turn one of the six cranks harder than any human process allows:

Affordance	Turns up which family	Example construction
Reset / re-run	Independence	mind-wiped re-derivation
Fork mid-reasoning	Independence	checkpoint-fork cross-examination
Decorrelated resampling	Independence	confession-as-invariant interrogation
Sandboxed information diet	Invariance	reject conclusions resting on un-logged retrievals
Narrow, fixed action space	Invariance	lower the Lipschitz constant by removing levers
Full logging / no privacy	Deterrence	tamper-evident reasoning trace, free
Forced bets / bonds	Deterrence + Incentives	set the enforceable penalty $B$
Read internal state	(all, weakly)	activation-honesty cross-check — [speculative]

The “read their minds” affordance is the highest-leverage and the least reliable: truth appears linearly represented in activations (Marks & Tegmark 2024; Goldowsky-Dill et al. 2025), but chain-of-thought is often unfaithful (Chen et al. 2025), deception survives safety training (Hubinger et al. 2024), and training against an honesty probe can induce evasion (Cundy et al. 2025). Use it as corroboration, never as the load-bearing check.

The family pages

• Calibration — make confidence claims true; tail calibration, abstention, the Kelly filter.
• Verifiability asymmetry — check cheaper than produce; the PCP-analogue for estimates, forced precision.
• Independence & decorrelation — manufacture independent signals; commit-reveal, forking, decorrelated interrogation.
• Invariance & low sensitivity — breakdown point, Byzantine-robust aggregation, randomize-everything.
• Incentive-compatibility — scoring rules, bets, the persuasion-budget bound.
• Deterrence — the inspection-game frontier, tamper-evident logs, provenance bonds.

A capstone — one fully-worked hardened stack on a real task — is planned but not written; Chapter 18, Certification and Gyms, is where its measurements would live.

Open questions

• Are six families the right cut, or do calibration and verifiability collapse into one (both make the output self-checkable), and does independence belong under invariance?
• Which families compose multiplicatively and which merely add the weakest link?
• Is there a single inequality the whole Part is special cases of — corruption cost as a function of sensitivity, enforceable penalty, and effective independence?
• What closes the correlated-error row, the one threat with no clean defender?