# The AI Operating Model and Platform
In 2019, a large European bank counted 160 machine learning models across its business units. When a new regulation forced them to audit every model touching credit decisions, the CDO's team needed six weeks just to *locate* them. Eleven were running in production on a data scientist's personal cloud account. Three had been retired by people who had since left. Two different teams had independently built the same churn model on the same data, reaching different conclusions that two different executives were citing in the same meeting.
This is what AI sprawl looks like at scale, and it is the default outcome. Every successful POC creates a new orphan: a model with no owner, no monitoring, no retraining schedule, and no line back to a platform. The question that separates CDOs who scale AI from those who accumulate technical debt is not "how do we build more models?" It is "what operating model makes the 50th model cheaper to build and safer to run than the fifth?"
That is an organizational and architectural design problem, and it is yours to solve.
Every AI operating model sits somewhere on a spectrum defined by where talent reports and where decisions are made. The two poles both fail predictably.
Fully centralized (a Center of Excellence that owns all data scientists) delivers consistency, reusability, and governance, but it starves on business context. The central team becomes a bottleneck, ranks work by its own priorities, and ships models that the business quietly ignores because nobody in the P&L helped shape them. You get technically excellent solutions to problems no one urgently has.
Fully embedded (data scientists reporting into business units) delivers relevance and speed but reproduces the European bank's chaos. Each unit reinvents feature pipelines, picks its own tools, sets its own risk tolerance, and builds no shared assets. You get 160 models and no leverage.
The durable answer is hub-and-spoke, but the phrase is used so loosely that it hides the real design decisions. The distinction that matters is *what the hub owns versus what the spokes own*, and getting that boundary right is 80% of the job.
A useful rule: the hub owns the how; the spokes own the what.
The hub (your central AI platform organization) owns the paved road—the platform, the reusable components, the MLOpsMLOpsMachine Learning Operations: combining ML and DevOps practices to industrialise, deploy, monitor, and retrain models reliably in production.View full definition → tooling, the model governance framework, the standards for monitoring and documentation, and a small bench of specialist talent (ML engineering, responsible AI, platform SREs). The spokes (embedded squads inside business domains) own use-case selection, domain feature engineering, model development against business metrics, and the P&L accountability for outcomes.
The failure mode of most CDOs is drawing this line by *role* ("all data scientists report to me") instead of by *responsibility*. Draw it by responsibility and the reporting lines follow naturally: platform and ML engineers report to the hub; applied data scientists sit in the spokes with a dotted line to the hub for standards and career development.
Before you commit to a structure, run the federation test on your current state. Score your organization on three axes:
A retailer with concentrated demand in merchandising and supply chain should not build a sprawling federation; two embedded squads plus a lean platform team is right. A global bank with AI demand in every function and heavy regulation needs a strong hub with formal standards. Match the model to the diagnosis, not to the org-chart fashion of the moment.
CDOs say "we're building an AI platform" and mean anything from a Databricks license to a governance committee. Be precise. A production AI platform is a set of shared services that turn one-off model builds into assembly-line production. The test of a real platform is simple: the marginal cost and time to deploy model N+1 falls as N grows. If each new model costs the same as the last, you have a toolset, not a platform.
Four platform layers deliver that declining marginal cost.
1. The feature layer. The single highest-leverage reusable component is the feature storefeature storeA centralised repository managing ML features, ensuring consistency between training and serving environments.View full definition →. When your fraud team and your credit team both need "customer transaction velocity over 30 days," they should consume the same governed, monitored, versioned feature—not build it twice with subtle differences. The feature storefeature storeA centralised repository managing ML features, ensuring consistency between training and serving environments.View full definition → is where the European bank's duplicate-churn-model problem gets solved at the root: one definition, one owner, many consumers.
2. The pipeline and training layer. Standardized, templated training pipelines so a new project starts from a working scaffold, not a blank notebook. This is where you encode "the paved road": a project generated from the template automatically gets experiment tracking, lineage capture, and a model-registry entry.
3. The deployment and serving layer. Common patterns for batch and real-time serving so a data scientist does not negotiate infrastructure for every launch. Self-service deployment behind guardrails is the goal.
4. The monitoring and governance layer. Automated drift detection, performance monitoring, and the model registry that would have answered the audit question in six minutes, not six weeks.
The registry is the connective tissue. A minimal registry entry makes the operating model enforceable because *nothing reaches production without one*:
model:
name: credit_churn_v3
owner_squad: retail-lending
business_metric: 90d_retention_uplift
training_data: feature_store/customer_txn_velocity@v4
approval: risk_committee_2024_03
monitoring:
drift_check: daily
performance_review: weekly
retrain_trigger: auc < 0.72This is not bureaucracy; it is the machine-readable contract between the spoke that owns the model and the hub that owns the platform. It is also your audit trail, your kill switch, and your reuse catalog in one artifact.
You will not build all four layers yourself, and you should not. Apply a sharp filter: buy the plumbing, build the paved road, partner on the frontier.
Buy commodity infrastructure—serving, orchestration, experiment tracking—where mature vendors exist and your needs are not special. Build the thin layer of opinionated templates, standards, and integrations that encode *your* organization's governance and data model, because that is your actual source of leverage and no vendor will do it for you. Partner (or use managed frontier services) for capabilities moving too fast to own, notably large foundation models, where building your own is a way to be permanently behind.
The mistake is inverting this: teams lovingly build a bespoke feature storefeature storeA centralised repository managing ML features, ensuring consistency between training and serving environments.View full definition → (plumbing) and then buy a generic governance product that does not fit their model lifecycle (paved road). You end up maintaining infrastructure that adds no differentiation while your standards remain unenforced.
Knowledge check
1. According to the lesson, what is the key question that distinguishes CDOs who successfully scale AI from those who accumulate technical debt?
2. The European bank example primarily illustrates which underlying concept?
3. Why does a fully centralized Center of Excellence model tend to fail, according to the lesson?
4. Select ALL correct answers. What characterizes an 'orphan' model as described in the lesson?
Select all the correct answers.
5. Select ALL correct answers. Which failure patterns does the lesson attribute to a fully embedded operating model (data scientists reporting into business units)?
Select all the correct answers.
The org chart and the platform are necessary but not sufficient. Two design choices determine whether the operating model actually functions on Monday morning: how you fund it, and how you run it.
Fund the platform as a product, not a project. The most common way to kill a nascent AI platform is to fund it as a capital project tied to a specific use case. The use case ships, the funding ends, the platform decays, and the next team rebuilds. Instead, fund the hub as a persistent internal product with its own roadmap, its own product manager, and its own success metrics—adoption, time-to-production, reuse rate. The spokes are funded by the business units against P&L outcomes. This split matters: it means the platform survives the completion of any single project, and it means the business pays for value while the enterprise pays for leverage.
A concrete mechanism: charge spokes a light internal consumption fee for platform services (compute, feature-store queries, serving). This does three things simultaneously—it creates cost discipline in the spokes, it gives the platform team a demand signal for what to build next, and it kills the "free internal resource" dynamic that leads to waste. Keep the fee low enough that it never pushes a team toward shadow IT; the goal is a price signal, not a profit center.
Design the talent model around a rotation, not a wall. The dotted-line-to-the-hub arrangement fails if it is only an org-chart line. Make it real with a rotation: applied data scientists in the spokes spend a defined stint on the platform team early in their tenure. They learn the paved road, they build a reusable component, and they carry those standards back into the business. This is how you propagate the operating model culturally rather than by decree—the standards become "how we work" because the people who work in the spokes helped build them.
Reserve your scarcest talent—senior ML engineers, responsible-AI specialists—in the hub, where their leverage is highest across all use cases. Distribute the applied talent to the spokes, where domain proximity is what makes them effective.
The mechanism that makes hub-and-spoke run is an intake and stage-gate process owned by the hub but governed jointly. Every proposed use case passes through gates that are deliberately lightweight early and rigorous late:
1. Intake. A one-page problem statement with a named business owner and a quantified value hypothesis. No owner, no value number—no entry. This single gate eliminates most sprawl.
2. Feasibility. A time-boxed spike (two to four weeks) against real data on the platform. The output is a go/no-go, and critically, most POCs should die here on purpose. A high POC-to-production conversion rateconversion rateThe percentage of visitors or prospects who complete a desired action (purchase, sign-up, contact form), calculated as conversions divided by total opportunities.View full definition → is a warning sign, not a badge of honor—it means you are not being selective enough at intake, or not killing weak ideas fast enough.
3. Productionization. The model enters the registry, gets its monitoring contract, passes governance review scaled to its risk tier, and deploys via the paved road.
4. Operate. Live monitoring, scheduled review, and a retirement plan. Every model has an owner and an expiry date for re-evaluation.
The stage gate is where the operating model becomes visible and enforceable. It is also where the CDO's judgment is most valuable: your job at the gates is not to approve everything that is technically sound, but to protect the platform and the portfolio from low-value work that will accumulate as maintenance debt. Saying no to a plausible-but-marginal model is one of the highest-return decisions you make, because every model you deploy is a liability you carry for years.
The bank in our opening rebuilt around exactly this. Within eighteen months, model deployment time dropped from months to weeks, every production model had a registry entry and a named owner, and—counterintuitively—the *total number* of models in production fell, because the intake gate stopped the sprawl at the source. Fewer models, more value, dramatically less risk. That is what a working operating model produces.