Reasoning models and when to use them: the hype is ahead of the practice

Something shifted in late 2024 when OpenAI released its o1 model and made the chain-of-thoughtchain-of-thoughtA prompting technique where a language model is guided to produce intermediate reasoning steps before giving a final answer, improving accuracy on complex tasks.View full definition → process visible to users. Watching a model reason through a problem step by step, backtrack, reconsider, and arrive at an answer felt qualitatively different from standard completions. The AI press ran with it. By 2026, "use a reasoning model for anything complex" has hardened into received wisdom across enterprise AI teams, consultant decks, and vendor webinars. The framing is intuitive and not entirely wrong. It is, however, oversimplified in ways that matter.

The consensus view

The standard position goes roughly like this: reasoning models, o3, Claude 3.7 Sonnet, Gemini 2.5 Pro with thinking enabled, DeepSeek R1, are slower and more expensive than standard models, but the tradeoff is worth it whenever accuracy matters. For anything involving multi-step logic, mathematical reasoning, code debugging, legal analysis, or scientific problem-solving, you should reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → for a reasoning model. Standard models are for drafting emails and summarising documents.

This view has genuine merit. The benchmarks support it on specific tasks. On AIME (a competitive mathematics benchmark) and on the SWE-bench software engineering evaluations, reasoning models outperform their non-reasoning counterparts by wide margins. OpenAI (a vendor, so treat their figures accordingly) has reported that o3 solves roughly 72% of SWE-bench problems compared to roughly 49% for GPT-4o. The logic of chain-of-thought is also theoretically sound: decomposing a problem into sequential steps reduces the chance that the model will pattern-match its way to a confident but wrong answer.

So the consensus is not a strawman. It reflects real performance differences on structured, verifiable tasks.

Where the consensus goes wrong

The first blind spot is task misidentification. Practitioners are applying reasoning models to problems that look complex but are not actually bottlenecked by logical depth. Summarising a 40-page contract, extracting structured data from a PDF, writing a first draft of a market analysis memo: these tasks have high stakes in the business context but do not require multi-step deductive reasoning. They require good language modelling and instruction-following. Running them through o3 instead of GPT-4o mini or Claude 3.5 Haiku burns tokenstokensA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition →, increases latency, and produces outputs that are, in practice, indistinguishable in quality. Teams doing this at scale are paying a premium of three to ten times the tokentokenA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition → cost for no measurable gain.

The second problem is that reasoning models hallucinate differently, not less. This is underappreciated. Standard models tend to hallucinate through confident confabulationconfabulationA hallucination is when an AI model generates output that is fluent and confident but factually wrong, fabricated, or unsupported by its source data.View full definition →: they fill gaps with plausible-sounding text. Reasoning models can hallucinate through faulty reasoning chains. The model produces a logical structure that looks rigorous, follows visible steps, and arrives at a wrong answer. Because the process looks careful, users are more likely to trust the output without verification. A 2025 study from researchers at MIT CSAIL found that users shown chain-of-thought outputs were significantly more likely to accept incorrect conclusions than users shown direct outputs from standard models, because the visible reasoning created an illusion of due diligence. Confidence in process is not the same as accuracy of output.

The third issue is latency and its operational consequences. A reasoning model taking 30 to 90 seconds to return a response is acceptable for an analyst running a one-off analysis. It is a serious friction point inside an agentic workflow where multiple model calls are chained together, or in any customer-facing application where response time affects satisfaction. Many teams have discovered this only after building pipelines around reasoning models, then needing to re-architect after deployment.

There is also a subtler second-order effect worth naming: over-reliance on reasoning models can reduce the quality of prompt engineeringprompt engineeringPrompt engineering is the practice of designing and refining text inputs to guide large language models toward accurate, relevant, and reliable outputs.View full definition → discipline inside an organisation. When a model visibly thinks through a problem, there is less pressure on the human to define the problem well. The model's reasoning becomes a substitute for clear thinking by the operator. Over time, teams lose the muscle of precise specification, which hurts performance on every model class, reasoning or otherwise.

What a sharp operator should actually do

The practical reframe is to treat reasoning models as specialised tools for a specific class of problem, not as a general upgrade over standard models.

The class of problem that genuinely benefits has three characteristics: the answer can be verified (math, code, formal logic), the error cost is high and asymmetric (a wrong answer in legal contract analysis or financial modelling is worse than a slow answer), and the task cannot be decomposed into simpler subtasks that a cheaper model handles well.

For everything else, the default should be a strong standard model with a well-constructed prompt. Claude 3.5 Sonnet, GPT-4o, and Gemini 1.5 Pro handle the vast majority of enterprise language tasks competently and at a fraction of the cost. The decision about which model to use should live in a routing layer, whether that is a purpose-built orchestration system or a simple classification step that categorises incoming tasks before dispatching them.

On verification: regardless of which model you use, the habit of output verification matters more than model selection. Reasoning models do not remove the need for human review on high-stakes outputs. They shift the form of potential error, they do not eliminate it. Build review steps into your workflows rather than treating model choice as a substitute for them.

Finally, run your own benchmarks on your own tasks. The published benchmarks measure performance on standardised academic problems. Your legal team's document review tasks, your finance team's variance analysisvariance analysisVariance analysis compares actual financial results against budgeted or planned figures to quantify differences and explain why they occurred.View full definition → prompts, your product team's specification writing, these have their own distributions of difficulty and their own quality criteria. Spending two days building a small evaluation set of fifty representative examples will tell you more about which model to use than any vendor comparison chart.

Reasoning models are a genuine capability advance for a defined set of problems. The mistake is treating them as a universal accuracy upgrade. Match the tool to the task, build verification into the process, and stop paying reasoning-model prices for standard-model work.