Bloomberg's bet on fine-tuning: what it teaches every enterprise about the RAG-vs-fine-tune decision

August 25, 2026 · 5 min · Free, with full transcript

Bloomberg built a domain-specific large language model from scratch rather than retrieving over generic ones, and the results clarified a decision that still confuses most enterprise AI teams. The logic behind that choice, and where it breaks down for other organizations, is more instructive than the model itself.

Listen to the podcast

4 min

Read the full transcript

Welcome to the Leaders Insights Podcast.

Today's episode, Bloomberg's bet on fine tuning, what it teaches every enterprise about

the rag versus fine tune decision.

Everyone's been screaming, just use rag for three years, and then Bloomberg goes and spends

a fortune training a model from scratch.

So who's wrong?

Nobody's wrong yet.

But a lot of people copied Bloomberg for the wrong reasons.

Let's be precise, rag means retrieval augmented generation, where you bolt a search system

onto a generic model, so it pulls in your documents at question time.

Bloomberg didn't do that.

They built Bloomberg GPT, a 50 billion parameter model, trained on roughly 700 billion tokens,

a token being a chunk of text, about three quarters of a word, and more than half of

that was their own financial data.

That sounds like a flex, not a strategy.

It was a flex that made sense in early 2023, when they announced it.

Back then, the open models were weak, and Bloomberg had something almost nobody else

has.

40 years of proprietary financial language, sitting in-house, already clean and labeled.

That's the real asset, not the compute.

Juan does.

The data mode.

But here's the uncomfortable part.

It's 2026.

GPT class models have moved twice since then.

Is Bloomberg GPT already a museum piece?

Partly, yes.

That's exactly the lesson.

The moment you fine-tune or train from scratch, you freeze a snapshot.

The base model keeps improving without you, and you're stuck maintaining your fork.

Bloomberg could afford that treadmill.

Most enterprises absolutely cannot.

So when someone in a boardroom says, we should build our own model like Bloomberg,

what do you tell them?

I ask them one question.

Does your knowledge change daily, or does your language change slowly?

If your problem is the model doesn't know today's contract terms, that's a knowledge

problem.

You want retrieval, RAG, because you can update the documents without touching the

model.

If your problem is the model literally doesn't speak our dialect, think derivatives desks,

radiology, legal citations.

That's a behavior problem, and that's where fine-tuning earns its keep.

Give me the failure case.

Where does copying Bloomberg blow up?

A mid-sized bank I talked to spent about eight months in a seven-figure budget fine-tuning

a model on their internal reports.

By the time they finished, an off-the-shelf model with a decent retrieval layer beat it.

Cheaper.

And it actually stayed current.

They'd solved a knowledge problem with a behavior tool.

Classic.

They confused the model sounds off with, the model is missing facts.

How do you even tell those two apart before you burn the eight months?

Keep test, take 50 hard questions, paste the relevant documents directly into the prompt,

just stuff the context in manually.

If the model suddenly gets them right, you have a knowledge gap, and RAG fixes that for

a fraction of the cost.

If it still fumbles even with the answer sitting right in front of it, now you've

got a genuine behavior gap, and fine-tuning is on the table.

That's suspiciously simple.

What's the catch?

The catch is people don't want the cheap answer.

Plumbing sounds like ownership, like a defensible asset.

RAG sounds like plumbing.

But plumbing is what most of them need.

And there's a hybrid nobody talks about enough.

Retrieval for the facts, a light fine-tune, only to fix tone and format.

Bloomberg's own follow-up work leaned harder on retrieval too, because even they couldn't

retrain fast enough to keep facts fresh.

So even Bloomberg quietly admitted that from scratch, BET had limits.

They didn't renounce it.

The model still does its job on financial language tasks, like sentiment, and named

entity recognition, which is pulling out companies and tickers from messy text.

But yes, for anything time-sensitive, they wrap retrieval around it.

The from scratch build was a foundation, not the whole house.

Run the paste-the-document test before you approve a single dollar of fine-tuning.

If stuffing the answer into the prompt fixes your problem, you have a retrieval problem

and anyone pitching you a custom model is either lazy or selling compute.

Fine-tune only when the model fails with the answer in front of its face.

That one experiment saves most companies a year and a couple million dollars.

Cheaper than a data science team learning it the hard way.

That's the show.

That's a wrap.

Fresh AI briefings drop daily at MBA-training.com.

Read and go deeper