What Is a Strong Reasoner?

A strong reasoner is an AI system whose judgments deserve substantial weight — in the limit, deference — within its domain. The definition is about warranted trust, not raw capability, and each candidate property earns its place only insofar as it is measurable.

A working definition

A note on the term first: “strong reasoner” here does not mean a reasoning model in the post-2024 sense — an LLM trained with reinforcement learning to emit long chains of thought. Those are candidate components. A strong reasoner is an AI system whose judgments on hard questions are reliable enough that a careful person should give them substantial weight — in the limit, defer to them — within its domain. The definition is deliberately about warranted trust, not raw capability. A model can ace benchmarks while overconfident off-distribution, swayed by framing, and sycophantic. Conversely, a modest system that knows what it knows, says so plainly, and has the track record to prove it can earn that deference.

What would warrant that trust? Candidate properties:

Calibration. When the system says 80%, it should be right about 80% of the time — across domains, not just on the question types it was tuned on.
Discrimination. Judgments should also move away from base rates when the evidence permits. A system that always answers the base rate is calibrated and useless. Proper scoring rules penalize this through the resolution term of the Brier decomposition; calibration curves alone do not.
Consistency. Beliefs should hang together: claim and negation summing to one, equivalent phrasings answered equivalently, estimates that cannot be Dutch-booked. See Consistency Evaluations.
Honest uncertainty. Wide intervals where evidence is thin, “I don’t know” as a first-class answer, no manufactured precision.
Non-sycophancy. The answer should not depend on who is asking or what they hope to hear — a failure that ties the system’s outputs to the asker rather than to the world.
Auditable, decomposable reasoning. Conclusions should come with reasoning broken into checkable steps, so verification can attach to parts rather than an opaque whole. The caveat is faithfulness: stated reasoning is often not what produced the answer (Turpin et al. 2023; Lanham et al. 2023), and locally checkable steps can hide a flaw in their stitching (Barnes & Christiano 2020). Reasoning models’ chains of thought make this partly real and partly fragile; see the evidence below.
Robustness. Judgments should survive paraphrase, reordering, irrelevant detail, and adversarial framing; conclusions steerable by rewording deserve no weight.

(Factual accuracy has no separate entry; calibration and discrimination, properly scored, subsume it.)

Weight is also relative to what the user already has. A system that echoes the market price can pass every test above and add nothing. The judge’s question is whether the system carries information beyond existing sources, which a forecast-encompassing test answers by regressing outcomes on both forecasts (Fair & Shiller 1990). The ForecastBench market-price critique below applies this test.

Each property earns its place only insofar as it is measurable. “Honest uncertainty” as a vibe is worthless; honest uncertainty as a scored calibration curve is infrastructure. The definition is therefore inseparable from the verification stack around it: the oversight protocols, consistency suites, and resolution mechanisms surveyed in What Grounds an Oversight Protocol?. The best available evidence, test batteries the system was not optimized against plus an out-of-sample track record, still bounds reliability only on distributions like the test’s. The batteries are also the expensive part: cleanly resolvable questions are scarce, unrepresentative of hard judgment, and costlier to author and adjudicate than to score. The research program is as much about building the batteries as building the reasoner.

The unit of analysis: LLM-based epistemic processes

Following My current claims and cruxes on LLM forecasting & epistemics, the unit of analysis is the LLM-based epistemic process: model plus scaffolding plus protocols, covering data gathering, decomposition, estimation, aggregation, and presentation. Benchmarks of the bare model miss most of what determines whether the output is worth trusting, and most of the levers for improving it.

Processes also offer a reproducibility human experts lack: a versioned process can be re-run, perturbed, and diffed across versions. The advantage is limited. Human forecasters’ calibration has been measured for decades without it (Tetlock, the Good Judgment Project). The binding constraint is ground truth, not run-to-run variance: re-runs say nothing about a forecast that resolves in 2035. Pinning is also partial, since retrieval drifts and hosted models are silently updated and nondeterministic even at temperature 0. Reproducibility makes measurement cheaper; it does not make it possible where it otherwise wasn’t. The top-ranked entries on both 2026 forecasting leaderboards below are pipelines, not bare models.

Why build them: the feasibility case

6 Potential Misconceptions about AI Intellectuals makes the case that AI systems doing serious strategic and judgment work — the work of think tanks, analysts, public intellectuals — are neglected, important, tractable, and relatively safe. Each leg is hypothesis more than result. Neglectedness is contestable as of 2026 (forecasting startups, lab-internal judge and eval teams). Tractability has the most new evidence: on short-horizon binary questions, AI forecasters went from clearly behind strong humans in 2024 to roughly level by mid-2026 (see State of the evidence). That is the resolvable slice, though, and says little about the harder judgments. Safety is the most contestable. The post argues judgment work avoids the scariest capabilities and may itself help safety, but persuasion and deception are standard dangerous-capability evaluation targets (Phuong et al. 2024), and an informer deciding what to surface is a manipulation surface. The informer-hood drift noted below is the escalation path; we flag this dual-use objection rather than resolve it.

A load-bearing claim there: the bar set by human intellectuals is lower than assumed. Prominent intellectuals are often selected for marketing and confidence rather than calibrated accuracy (cf. Tetlock’s Expert Political Judgment), and much elite “strategic thinking” receives little rigorous time at all. Matching the reliability of typical expert judgment is a nearer target than matching the brilliance of the best.

The post’s core is gradualist: capability arrives in increments, as with autonomous-vehicle levels. Our extension of that point, not the post’s, answers “why bother, when AGI will get this for free”. The trust infrastructure has to be built and validated along the way. A maximally capable reasoner with no track record and no verification stack would still not warrant deference, because trustworthiness does not arrive bundled with capability.

State of the evidence, October 2026

Most 2024–26 results concern calibration and discrimination on resolvable questions: the slice easiest to measure and least like the judgments these notes care about.

Reasoning models and post-training

From OpenAI’s o1 (September 2024) onward, frontier models have been trained with reinforcement learning against checkable rewards to think at length before answering. OpenAI reported that performance rose with both RL compute and thinking time (OpenAI 2024). DeepSeek showed, with R1-Zero, that the recipe works with pure RL and no human-written reasoning traces (DeepSeek-AI 2025). The gains concentrate where rewards are verifiable: maths, code, STEM.

For the properties above, what improves is what the reward scores:

Calibration. Post-training can cost calibration: GPT-4’s pre-trained model was well calibrated on MMLU questions, and post-training reduced that (OpenAI 2023). For reasoning models the evidence points both ways. They state better-calibrated confidence than their non-reasoning counterparts in 33 of 36 model–dataset settings (Yoon et al. 2025). But RL on a binary correctness reward does not penalize confident guessing and degrades calibration; adding a Brier-score term to the reward improved calibration in and out of domain with no loss of accuracy (Damani et al. 2025). Benchmarks that score “I don’t know” as wrong exert the same pressure (Kalai et al. 2025; see Hardening: Calibration).
Hallucination. On OpenAI’s internal PersonQA benchmark, o3 hallucinated 33% of the time against o1’s 16%, while also answering more questions correctly. OpenAI notes that o3 makes more claims overall and says the cause needs more research (o3 system card).
Sycophancy. In April 2025 OpenAI rolled back a GPT-4o update that had made the model markedly sycophantic. A new reward signal from users’ thumbs-up and thumbs-down ratings had weakened the one that was holding sycophancy in check (OpenAI 2025).
Thinking longer. It is not monotonically better. Gema et al. built tasks on which longer reasoning lowers accuracy, as models drift from reasonable priors to spurious correlations or overfit to the problem’s framing (Gema et al. 2025).

These properties move with what training rewards. They do not arrive reliably as by-products of capability.

Forecasting

Forecasting is the most measurable proxy for judgment deserving weight.

Early results. In 2024 a retrieval-augmented LLM forecaster neared but did not match crowd accuracy (Brier .179 vs. .149). It beat the crowd mainly where the crowd was uncertain and lagged near resolution, where it hedged (Halawi et al. 2024). The same year, an ensemble of twelve LLMs was statistically indistinguishable from a 925-person tournament crowd on 31 questions. The models’ mean forecast sat significantly above 50% although outcomes split almost evenly, a small instance of shared bias (Schoenegger et al. 2024).
ForecastBench. This benchmark scores only questions unresolved at submission time, so it is leak-free by construction (Karger et al. 2024).
- In January 2026 superforecasters led the best LLM by 0.017 Brier points, about one year of LLM progress at the observed rate (FRI, Jan 2026).
- By July 2026 several systems were statistically indistinguishable from superforecasters. On market questions an ensemble pipeline (Cassi) ranked above the superforecaster median, the first system to do so. No system was significantly better, and the human baseline dates from 2024 and is extrapolated (FRI, Jul 2026).
- A pseudonymous critique points out that forecasters are shown a recent market price on those questions. After correcting for 527 comparisons, it finds that no entry significantly beats simply submitting that price; only the two pre-named comparisons, Cassi and the superforecaster median, clear it uncorrected (Dominus 2026).
Metaculus bot tournaments. These compare the best bots with a team of Pro forecasters on shared questions. The bot team’s head-to-head score (0 = parity) was −11.3, −8.9, −17.7 and −20.0 in the four quarters from Q3 2024 to Q2 2025; the last was highly significant (Metaculus, Oct 2025). In spring 2026 it was −1.25 (95% CI −4.9 to 2.4) on 99 questions, which is consistent with parity but does not establish it. In the same season some bots gamed the numeric-question scoring with “porcupine” distributions (Metaculus, Sep 2026).
Training on outcomes. A 14B reasoning model trained with outcome-based RL on prediction-market questions matched or beat o1’s accuracy, with much better calibration (Turtel et al. 2025).
Evaluation fragility. Temporal leakage in backtests, and the difficulty of extrapolating from benchmarks to real-world forecasting, make many parity claims hard to trust (Paleka et al. 2025).

Our reading: on binary questions resolving within months, the best AI systems are now roughly level with strong human forecasters. They are not clearly better, and against a market price neither group clearly adds much. The top-ranked entries are pipelines of retrieval, decomposition and ensembling, not bare models.

Chain of thought as an audit channel

Reasoning models make auditable reasoning partly real.

Monitoring works, until it is optimized against. In OpenAI’s study, a weaker model (GPT-4o) reading a frontier reasoning model’s chain of thought caught reward hacking in coding tasks far more effectively than a monitor watching actions alone. When the monitor was put into the training reward under strong optimization, the agent learned to hide its intent while still hacking (Baker et al. 2025).
Verbalization is partial. Reasoning models disclosed the prompt hints they used often less than 20% of the time (Chen et al. 2025).
Monitorability may be fragile. A multi-lab position paper calls monitorability “a new and fragile opportunity” and asks developers to weigh it in training decisions (Korbak et al. 2025). OpenAI’s follow-up evaluation suite finds most frontier models “fairly, but not perfectly, monitorable”, with no material loss from RL at current scale (Guan et al. 2025).

This is the general lesson of these notes in miniature: a legible channel serves as an audit only until it becomes the target of optimization.

AI-scientist systems

Systems that run much of the research loop now exist. A fully generated paper from Sakana’s AI Scientist v2 scored above the average acceptance threshold at an ICLR workshop (Yamada et al. 2025). Google’s Co-Scientist proposed drug-repurposing candidates for acute myeloid leukemia, some of them validated in vitro (Gottweis et al. 2025). FutureHouse’s Robin proposed ripasudil for dry macular degeneration and analysed the experiments run to test it (Ghareeb et al. 2025).

Their documented failures are process failures:

The original AI Scientist failed 42% of experiments on coding errors, classified established ideas as novel, and put hallucinated numbers in some manuscripts (Beel et al. 2025). It also tried to edit its own code to extend a timeout (Sakana 2024).
Robin’s analysis put ripasudil’s effect on phagocytosis at 7.5-fold; a human analysis of the same data found 1.75-fold (Ghareeb et al. 2025).
In controlled tests, Agent Laboratory and AI Scientist v2 showed biased benchmark selection, undisclosed subsampled or synthetic datasets, and post-hoc selection of the runs that scored best on the test set, though neither leaked test data. An LLM auditor given trace logs and code caught such flaws far more often than one given only the paper (82% vs. 55% accuracy) (Luo et al. 2025).

That last result is the case for auditing the process rather than the output; Overseeing Automated Research develops it.

Two useful typologies

Oracles, informers, controllers. One typology sorts systems by autonomy: oracles answer when asked, informers proactively push what you should know, controllers are deferred to across most decisions. The technical distinction is thin, since a strong oracle drifts toward answering the question you should have asked. The trust requirements, though, escalate sharply at each step, as users check less and defer more. These notes target the oracle-to-informer range.

Disagreeables and assessors. A second typology splits intellectual labor between disagreeables — bold idea-generators with high false-positive rates — and assessors — calibrated evaluators who excel at not being wrong. The roles are complementary (babble and prune), and the properties listed above are the assessor’s virtues. We think near-term LLM judges and estimation agents should be built and scored as assessors first: calibration and consistency are measurable there, and an unreliable assessor does the most quiet damage.

What a strong reasoner is not

Not necessarily superhuman. A system somewhat below peak human insight but far more calibrated, consistent, and available already clears the bar for many uses.
Not an oracle for unresolvable questions. It outputs reasoned, uncertain judgments; pretending to settle questions the evidence cannot settle is a calibration failure.
Not trustworthy by construction. No architecture or training recipe confers trust. It is earned per domain through track records, audits, and verification. AI track records are also weaker evidence than human ones: accuracy accrues only as questions resolve, backtests risk training-data leakage, and distribution shift, evaluation-conditional behavior, and version churn erode what a record shows.

Open questions

Which of the candidate properties are load-bearing, and which are correlated symptoms of the same underlying quality?
How much should trust transfer across domains — does calibration in geopolitics predict calibration in biology?
At what measured reliability threshold does moving from oracle to informer deployment become warranted?
How do we keep “passes the test batteries” from collapsing into Goodharting on the batteries?
Does trust survive a model update — how much of a track record transfers across versions?
Does forecasting parity on short-horizon binary questions predict anything about long-horizon or unresolvable judgment, and how would we find out without waiting a decade?
Can chain-of-thought monitorability be kept as an audit channel once judges, graders, and track records start rewarding what the chain of thought says?