Robust Reasoning Processes

Notes on measuring reasoning processes such as peer review, prediction markets, audits and LLM judges: what they deliver, what they cost, and what it costs to corrupt them. This is an early research exploration, with drafts and conjectures rather than settled results.

Start with Chapter 1, or go straight to The Process Catalogue.

The map

Each process gets two numbers: what it costs to run, per bit of validated information, and what an adversary must pay to bend its output. Plotting the catalogue’s 17 processes on those two axes leaves one corner empty: nothing yet is both cheap to run and hard to corrupt.

Shapes are the catalogue’s three families: grounded in resolution or track records, institutional judgment, AI-native. Positions are our own rough judgments from The Process Catalogue, so the empty corner is a conjecture to test, not a finding.

The arrows show the two directions in which hardening can move a cheap process. Hardening bought with verification costs work on every run, so it raises both costs and stays on the diagonal. Deterrence (bonds, clawbacks, occasional audits) raises the cost to corrupt while adding little to the cost to run, which is why it looks like the most promising route into the corner. Neither move has been measured.

Planned chapters are listed where they will go, without links.

Start here

Robust Reasoning Processes — the problem and the three core quantities
Cruxes — the central uncertainties, and what would move them

Part I — The Judge

The consumer of reasoning, who cannot check the work directly.

The Core Model — the basic quantities, graded by how far to trust them
Epistemic Impact Analysis — valuing information by how it improves beliefs
Constructing Utility Functions — weighted question portfolios
What Can You Use from an Untrustworthy Source? — trust costly signals

Interlude: The Funding Effect — a measured case of distortion

Part II — The Processes

The procedures themselves, and what their verdicts rest on.

The Process Catalogue — 17 processes rated on both axes
What Grounds an Oversight Protocol? — debate, markets and more
Consistency Evaluations — can an AI estimator be arbitraged?
What Is a Strong Reasoner? — an AI whose judgments deserve weight

Part III — Hardening

How to make a process harder to corrupt, and at what cost.

Hardening: Overview — the six families of lever, compared
Calibration — make confidence claims true
Verifiability Asymmetry — make checking cheaper than producing
Independence & Decorrelation — make an attacker beat every signal
Invariance & Low Sensitivity — ignore what an adversary controls
Incentive-Compatibility — make honest reporting the best strategy
Deterrence — penalties that cost little unless someone cheats
Certification and Gyms — planned

Part IV — The Environment

The game the processes live inside: producers, resolvers, adversaries.

Identity and Track-Record Infrastructure — making penalties stick
The Market for Truth — planned
Law, Liability, and Private Deliberation — planned
Culture, Consent, and Adoption — planned

Part V — Applications

What strong reasoners could be used for, and what has been tried.

The Reliability Ladder — five tiers of use, ordered by the trust each needs
Building Strong Reasoners — five directions with cheap first tests
Overseeing Automated Research — managing research you cannot check
LLM Epistemics in Production — QURI’s systems, failures included
Open Problems — gathered from every chapter

Terms and symbols are indexed in the Glossary & Notation.

How to read these notes

Formalisms are graded [exact], [standard shape] or [heuristic], by how far each can be trusted. See The Core Model.
Failures are published. LLM Epistemics in Production reports false positives, overconfidence and a miscounted competition alongside the wins.

Other CAIRN sites: Delegation Risk, Evaluation Engineering, CAIRN Handbook.

Maintained by the Quantified Uncertainty Research Institute. Source on GitHub.