AI reasoning works, but chain-of-thought may not be the receipt

Quanta’s analysis frames the AI reasoning debate around a tension that has become harder to ignore. Large reasoning models can solve increasingly difficult math, logic, and puzzle tasks. But the chain-of-thought text they produce may not be a faithful account of the internal process that led to the answer.

The article walks through the whiplash of recent evidence. Research associated with Apple criticized reasoning models for collapsing under simple conditions. Work from the Santa Fe Institute showed models solving carefully designed benchmarks through surface-level shortcuts. At the same time, frontier systems have produced impressive mathematical results, including OpenAI’s reported one-shot solution to a known research problem and Google DeepMind’s work with Terence Tao on rediscovering or improving solutions across 67 math problems.

The harder question is what the intermediate text means. Melanie Mitchell’s summary is useful: reasoning models do improve performance; the chain-of-thought they generate is not necessarily faithful to what is happening inside the model; and some of that text may not be useful at all. The article cites research from Arizona State, NYU, Northeastern, and UC Berkeley suggesting that irrelevant traces, meaningless filler tokens, or removed thinking steps can sometimes leave performance largely intact.

HN commenters pushed on the same distinction. One line of discussion revived the “Clever Hans” analogy: a system can produce the right answer while relying on cues or shortcuts that do not match the human explanation. Another line argued that the debate risks becoming semantic, because the practical question is not whether the model “really reasons” in a human sense, but when its outputs can be trusted and audited.

That distinction matters for deployment. If chain-of-thought is treated as a transparent proof, users may overtrust a polished narrative. If it is treated as one model-generated artifact among others, teams can demand external checks, causal tests, and task-specific validation. The useful position is not that reasoning models are fake or solved. They are powerful systems whose explanations still need evidence of their own.