Data FinOps: controlling cloud data cost

# Data FinOps: controlling cloud data cost

In 2021, a mid-market fintech's data team celebrated a milestone: their Snowflake bill had crossed $3M annually. Not a milestone they *wanted*, one they discovered when finance escalated to the board. The platform was technically excellent. Query latency was low, adoption was high, the analysts loved it. And it was quietly consuming 40% of the entire technology budget with no line-of-business owner able to explain a single dollar of it. The CDO had built a Ferrari and handed the keys to anyone who asked, with the fuel invoice going to a mailbox nobody read.

This is the trap of cloud data platforms. The elasticity that makes them powerful, infinite storage, on-demand compute, self-service access, is the same elasticity that makes cost invisible until it's structural. You already know how to build the catalog and enforce governance. This lesson is about making the *bill* a governed, attributable, optimizable asset, and doing it without turning the platform back into the bottleneck you spent two years removing.

AttributionAttributionA framework for assigning credit to the touchpoints that contributed to a conversion, so you can measure which channels and interactions actually drive results.Voir la définition complète → is the precondition for everything

You cannot optimize what you cannot attribute. The single biggest failure in data cost management is treating the platform as one undifferentiated cost center. When the bill is a single number, the only lever available is a crude one: freeze spend, and watch adoption collapse.

The FinOps discipline borrowed from cloud infrastructure gives you three operating phases, Inform, Optimize, Operate, but for a data platform specifically, the leverage all sits upstream in attributionattributionA framework for assigning credit to the touchpoints that contributed to a conversion, so you can measure which channels and interactions actually drive results.Voir la définition complète →. Your first job is to make every dollar of compute and storage traceable to a team, a workload, and a business outcome.

The tagging and warehouse-isolation strategy

AttributionAttributionA framework for assigning credit to the touchpoints that contributed to a conversion, so you can measure which channels and interactions actually drive results.Voir la définition complète → in a data warehousedata warehouseA central repository that consolidates data from many source systems into a structured, query-optimized store designed for analytics, reporting, and business intelligence.Voir la définition complète → happens two ways, and you need both.

Compute isolation means separating workloads onto distinct virtual warehouses or compute pools so their spend is measured independently. This is your primary attributionattributionA framework for assigning credit to the touchpoints that contributed to a conversion, so you can measure which channels and interactions actually drive results.Voir la définition complète → mechanism because compute is usually 70-85% of the bill. A single shared warehouse where marketing's dashboards, finance's reconciliation jobs, and a data scientist's runaway notebook all compete gives you one number and no accountability. Split them.

Tagging carries the business context that isolation alone can't. Every warehouse, every scheduled job, every materialized dataset should carry tags for cost center, domain, environment, and criticality.

-- Snowflake: object-level tags that flow into cost views
ALTER WAREHOUSE mktg_analytics_wh SET TAG
  cost_center = 'MKT-204',
  data_domain = 'campaign',
  env = 'prod',
  criticality = 'tier2';

The discipline here is not the syntax, it's the *enforcement*. Untagged resources are the entropy that destroys every FinOps program by month four. Make tagging a deployment gate: no warehouse or scheduled task ships to production without required tags, enforced in CI, not in a policy PDF. This is where your existing governance muscle pays off, you already control the deployment pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.Voir la définition complète →; extend the same policy engine to cost tags.

Showback before chargeback

Once spend is attributed, you face a political choice: showback (tell teams what they cost) or chargeback (bill it to their P&L). Do not jump to chargeback. Chargeback without a mature showback period produces gaming, teams shift workloads to escape allocation, or hoard capacity to protect it. Run showback for at least two quarters. Publish a monthly cost-per-domain report that names owners. The mere act of visibility typically cuts 15-20% of spend, because most waste is nobody-was-looking waste, not malicious waste. Move to chargeback only once the numbers are trusted and the unit economics are stable.

The metric that matters most here isn't total spend, it's unit cost: cost per query, cost per pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.Voir la définition complète → run, cost per active user, cost per dashboard load. Total spend rising while unit cost falls is a *healthy* platform scaling with the business. Total spend flat while unit cost rises is a platform rotting from the inside. Report unit economics to your leadership, not gross dollars, or you will lose the narrative to a finance team that only sees the invoice going up.

Compute: where the money actually burns

With attributionattributionA framework for assigning credit to the touchpoints that contributed to a conversion, so you can measure which channels and interactions actually drive results.Voir la définition complète → in place, compute is where you'll find the fastest, largest savings. Three levers, in order of impact.

Query optimization is a governance problem disguised as a technical one

The worst offenders on any platform follow a power law: a handful of queries consume a disproportionate share of compute. The classic pattern is the SELECT * on a multi-terabyte table that a BIBITechnologies and processes that turn raw data into actionable insights via reporting, dashboards and analysis, so teams can decide based on facts rather than intuition.Voir la définition complète → tool re-runs every fifteen minutes because someone left auto-refresh on. Or an unpartitioned scan that reads the full history of a table to answer a question about last week.

Your query engine already logs this. Snowflake's QUERY_HISTORY, BigQuery's INFORMATION_SCHEMA.JOBS, Databricks' system tables, each exposes bytes scanned and compute consumed per query. The move is to build a recurring most-expensive-queries report and route it to the owning domain, not to your central team. Centralizing the fix recreates the bottleneck; the domain that wrote the query owns the fix, and your platform team owns making the cost visible and providing the pattern library.

The highest-leverage technical patterns to enforce:

• Partition and cluster pruning: ensure tables are partitioned on the columns queries actually filter on. A well-clustered table can cut bytes scanned by an order of magnitude.
• Materialization discipline: recomputing the same aggregate hundreds of times a day is a tax. Materialize it once. But watch the inverse, materialized views that refresh constantly for data nobody reads are pure waste.
• Result caching: identical queries should hit cache, not recompute. This is often a configuration teams have disabled without realizing the cost.

Right-sizing and auto-suspend

Idle compute is the most embarrassing line on a data bill because it buys literally nothing. A warehouse configured to stay warm "for responsiveness" while idle 80% of the day is a space heater. Set aggressive auto-suspend, 60 seconds is defensible for most interactive warehouses. The trade-off is cold-start latency versus idle cost; measure it rather than defaulting to always-on out of fear.

Right-sizing is the harder judgment. Bigger warehouses run faster but cost proportionally more; the question is whether the *total* cost (size × runtime) drops. Frequently a larger warehouse finishes a heavy transformation so much faster that it's cheaper overall, this is counterintuitive and worth measuring per workload. Don't assume smaller is cheaper.

Commitment purchasing

Once your baseline consumption is predictable, and attributionattributionA framework for assigning credit to the touchpoints that contributed to a conversion, so you can measure which channels and interactions actually drive results.Voir la définition complète → is what makes it predictable, negotiate committed-use discounts. Reserved capacity and committed-spend contracts routinely deliver 20-40% off on-demand rates. The risk is over-committing to capacity you won't use. The rule: commit to your *floor*, the consumption you're confident about, and let the peaks flex on on-demand. Never commit to your peak.

Storage: the slow leak

Storage is usually a smaller fraction of the bill than compute, but it's the leak that grows silently and compounds. The core discipline is tiering: matching data's storage cost to its access frequency and value.

The tiering framework

Think in three tiers, defined by access pattern, not by age alone:

• Hot: queried frequently, latency-sensitive, kept in the warehouse's native performant storage. Expensive per terabyte, but the access justifies it.
• Warm: queried occasionally, tolerant of slightly higher latency. Candidate for cheaper object storage queried via external tables.
• Cold: retained for compliance or rare audit, essentially never queried. Belongs in archival tiers (S3 Glacier, equivalent) at a fraction of the cost.

The mistake is tiering by age. "Everything older than 90 days goes cold" fails when a machine-learning team needs three years of history for training. Tier by *actual access*, which your query logs already tell you. A table not scanned in 90 days is a tiering candidate regardless of its age; a two-year-old table queried daily stays hot.

The three storage killers

Three patterns account for most storage waste, and none of them are the data your business actually uses:

1. Time-travel and failsafe retention set too long. These features let you recover deleted or changed data, and they're wonderful, until you're paying to retain 90 days of change history on tables that get fully rebuilt every night. Set retention per table criticality, not one global default.

2. Orphaned datasets. The output of an experiment that ended eighteen months ago, the "temp" table someone created in 2022, the full copy of production a data scientist made to "be safe." Your catalog can find these, cross-reference the asset inventory against access logs and flag zero-read datasets with no downstream lineage.

3. Uncompacted small files. In data-lake architectures, streaming and micro-batch writes produce thousands of tiny files that inflate both storage and query cost. Scheduled compaction is unglamorous and essential.

This is where lineage, the subject of this module, becomes a *cost* instrument, not just a governance one. A dataset with no downstream lineage and no reads is a deletion candidate. Lineage tells you what breaks if you archive something; without it, teams hoard everything because deleting feels dangerous. The catalog you built for trust is also the mapmapUsing software to automate repetitive marketing tasks and campaigns, enabling personalisation at scale across channels like email, web, and social.Voir la définition complète → for safe cost reduction.

Operating the discipline: making it stick

A cost cleanup is a project. Data FinOps is an operating model. The difference between the two is whether the savings survive contact with the next quarter's growth. Most cost initiatives deliver a spectacular one-time reduction and then the bill climbs right back, because the incentives and rituals never changed.

The cross-functional operating rhythm

FinOps only works as a shared responsibility between the CDO's platform team, finance, and the consuming domains. Establish a lightweight FinOps working group with a monthly cadence. The platform team brings the attributed spend and unit economics. Finance brings the budget context and forecast. The domains own their consumption and their optimization backlog. Your role as CDO is to chair the trade-offs, not to police every query, which would make you the bottleneck again, but to arbitrate when a domain's cost growth outpaces its value.

The key artifact is a budget-versus-actual view per domain with anomaly alerting. A domain whose spend jumps 3x week-over-week should trigger an alert to *that domain's owner*, automatically, before finance ever notices. Waiting for the monthly invoice to detect a runaway pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.Voir la définition complète → is how the $3M surprise happens.

Guardrails, not gates

The instinct after a cost scare is to lock everything down, approval workflows for every new warehouse, quotas that block work. This destroys the self-service culture you built and pushes teams to shadow infrastructure. Prefer guardrails: automated limits that prevent catastrophe while permitting autonomy.

Concretely: set resource monitors that suspend a warehouse when it exceeds a credit threshold, so a runaway job caps out instead of running all weekend. Set per-query timeout limits. Set default warehouse sizes that teams can override with justification, rather than requiring approval to provision anything at all. The philosophy: make the safe path the default and the expensive path visible, but never make people ask permission to do their jobs.

Tie efficiency to the value narrative

The trap to avoid is letting FinOps become a pure cost-cutting exercise. A data platform's job is to generate value, and there are workloads worth their high cost. A fraud-detection model that runs expensive real-time inference and saves millions in losses is not a candidate for optimization, it's a candidate for *investment*. The unit-economics framing protects you here: when you report cost against the value each domain generates, you can defend the expensive workloads that earn their keep and cut the cheap ones that produce nothing. That's the conversation that keeps you at the strategy table rather than the budget-defense table.

Key Takeaways

• Attribution is the whole game. Isolate compute by workload and enforce cost tags as a deployment gate. Untagged resources are the entropy that kills every FinOps program, make untagged the exception that fails CI, not the norm.
• Report unit economics, not gross spend. Cost-per-query, cost-per-user, and cost-per-outcome tell you whether the platform is scaling healthily or rotting. Rising total spend with falling unit cost is success; defend that narrative to your board.
• Attack compute first, storage second. The power-law of expensive queries and idle warehouses is where the fast money is. Route the expensive-query report to the owning domain, you provide visibility, they own the fix.
• Tier storage by access, not by age, and use your lineage graph to identify orphaned, zero-read datasets as safe deletion candidates. The catalog you built for trust is also your mapmapUsing software to automate repetitive marketing tasks and campaigns, enabling personalisation at scale across channels like email, web, and social.Voir la définition complète → for safe cost reduction.
• Build guardrails, not gates. Resource monitors, query timeouts, and anomaly alerts to domain owners preserve self-service while capping disaster. A monthly cross-functional rhythm turns a one-time cleanup into a durable operating model.