Reasoning models and when to use them: the hype is ahead of the practice

August 18, 2026 · 5 min · Free, with full transcript

Reasoning models like OpenAI's o3 and Google's Gemini 2.0 Flash Thinking have captured attention by visibly "thinking through" problems before answering. The consensus says to use them everywhere you need accuracy, but that prescription is wrong in ways that will cost you money and slow your teams down.

Listen to the podcast

4 min

Read the full transcript

Welcome to the Leaders Insights Podcast.

Today's episode, reasoning models and when to use them.

The hype is ahead of the practice.

Everyone's telling me to route every serious prompt through a reasoning model.

You're about to tell me that's dumb.

Go.

It's not dumb.

It's expensive and slow for problems that never needed it.

A reasoning model, one that generates a chain of intermediate steps before answering, like

OpenAI's O3 or Google's Gemini 2.0 Flash Thinking, bills you for all that thinking.

You're paying for tokens the user never sees.

How much are we actually talking?

Give me a number that stings.

On the reasoning tiers, you're often paying 3 to 10 times more per request than a standard

model, and latency jumps from under a second to 15 or 20.

I've seen a support triage pipeline where swapping to a reasoning model raised the monthly

bill from about $4,000 to $31,000 for classification that a smaller model nailed at 98% accuracy

anyway.

So the model was thinking hard about a problem that didn't require thought.

Exactly.

Classifying is this ticket about billing.

Is pattern matching?

There's no multi-step logic to reason through.

You're asking a mathematician to alphabetize your bookshelf.

It'll do it slowly and charge you consultant rates.

Fine.

Where does the reasoning actually earn its keep?

When the answer depends on a sequence of dependent steps where an early mistake poisons

the rest.

Multi-step math, code that has to compile and pass tests, logic puzzles, planning tasks

with constraints.

Anything where the model benefits from checking its own work mid-flight.

Give me the concrete case.

A fintech team I know does reconciliation, matching thousands of transactions across ledgers

with weird edge cases, partial refunds, currency conversions.

A standard model hit roughly 70% correct matches.

The reasoning model got to 94% because it could hold intermediate state and catch its

own contradictions.

There, paying 10 times more is trivial next to a human rechecking mismatches.

So the durable principle is, what matched the model to the shape of the problem?

Matched the compute to the depth of the reasoning required.

This isn't new.

The best operators have run this playbook for a decade with regular software.

You don't put your whole database behind the most expensive query engine.

You profile the workload, find the 5% of requests that are genuinely hard, and route

only those to the heavy machinery.

But how do I know in advance which requests are the hard 5%?

That's the uncomfortable part.

You don't guess, you measure.

Run a cheap model first and add a confidence check or a validator.

If the cheap model is sure, ship it.

If it's uncertain or the output fails a test, code that won't compile, a total that doesn't

balance, escalate that single request to the reasoning model.

It's called a cascade, and it's just triage.

A cascade being a fallback ladder.

Cheap model then medium, then the expensive one.

Right, and the economics are lovely.

If 85% of your traffic clears the cheap tier, you're paying premium prices on 15% of requests

instead of 100%.

That same reconciliation team cuts spend by roughly 60% versus routing everything to 03

and lost less than a point of accuracy.

Vendors numbers or theirs?

Theirs from their own billing dashboards, not a model provider's benchmark, which

you should always distrust because the provider profits when you use the biggest model.

Their marketing quietly assumes maximum consumption.

So the hype is basically a pricing strategy dressed as best practice?

Partly yes.

Use reasoning everywhere is convenient for the people selling reasoning tokens.

It's terrible engineering advice.

The visible thinking is genuinely impressive, and that's exactly why people overapply

it.

It looks smart, so it feels safer.

Being safe is not a budget line.

What about the argument that models get cheaper, so why bother optimizing now?

They do get cheaper, but your traffic grows faster.

Optimizing your routing is a habit, not a one-time fix.

And the teams that build the measurement muscle now will still be ahead when prices drop

because they'll know where the hard problems actually live.

Give me the one thing to do Monday morning.

Take your top three AI workflows, log what fraction of requests are genuinely multi-step,

and route only those to a reasoning model.

Everything else goes to the cheap tier with a validator.

You'll cut cost and probably improve speed the same week.

Done.

Want to know where you actually stand?

Take the AI self-assessment at MBA-training.com.

Read and go deeper