DataData Architecture

Cutting cloud data costs without breaking analytics

Cloud bills for data infrastructure have become one of the fastest-growing line items in enterprise IT budgets, and most organizations are overpaying without realizing it. This playbook gives CDOs a concrete sequence of moves to reduce spend significantly while keeping analytical capability intact.

🎙️

Listen to the podcast

4 min

The pattern is familiar. A data platform gets built under pressure, teams pile on compute, storage sprawls across three clouds, and somewhere around the 18-month mark the CFO calls. Cloud data costs have a way of compounding quietly: Snowflake query costs, BigQuery slot reservations, S3 storage for data nobody queries anymore, Databricks clusters running idle overnight. According to Gartner (as of their 2024 reporting, still widely cited in 2026 budgeting conversations), cloud waste across enterprise workloads runs between 30 and 35 percent of total spend. For data teams specifically, the figure is often higher because data infrastructure is less monitored than application infrastructure.

The reason this bites harder now is that many organizations signed multi-year cloud contracts during the 2021-2023 expansion period and are now renegotiating or renewing them with much less leverage than they expected. At the same time, LLM-driven analytics workloads are being layered on top of existing infrastructure, creating new cost vectors before the old ones are controlled. CDOs who do not act on this deliberately will find their budgets consumed by infrastructure before any value is demonstrated.

A concrete cost-reduction sequence

Step 1: Get an honest inventory before touching anything

Most cost-reduction efforts fail because teams jump to optimization before they understand what they are actually running. Spend two weeks pulling a complete catalog of active compute clusters, storage buckets, data pipelines, and query patterns. In Snowflake, the ACCOUNT_USAGE schema gives you warehouse credit consumption by user and by query over the past year. In BigQuery, the INFORMATION_SCHEMA.JOBS view shows slot consumption and bytes billed per query. Export this into a simple dashboard. You are looking for three things: idle or near-idle compute, storage that has not been read in 90-plus days, and expensive queries running on unoptimized tables.

Step 2: Attack compute first, it moves the needle fastest

Idle compute is the single largest source of waste. Auto-suspend settings on Snowflake warehouses are frequently left at defaults (10 minutes) or disabled entirely. Set them to 60 seconds for development environments and review whether XL and larger warehouses are justified by actual concurrency data, not by assumption. On Databricks, audit cluster policies to ensure jobs clusters are used for batch workloads rather than all-purpose clusters left running. One large European retailer reduced its Databricks spend by 40 percent in a single quarter simply by enforcing cluster auto-termination at 15 minutes and migrating overnight batch jobs from interactive clusters to jobs clusters.

Step 3: Tier your storage properly

Not all data should live in the same storage class. Hot analytical tables that are queried daily belong in performance storage. Data older than 18 months that exists for compliance or audit purposes belongs in cold storage, which on AWS S3 is Glacier Instant Retrieval or Glacier Flexible Retrieval depending on your access tolerance. The cost difference between S3 Standard and S3 Glacier Flexible Retrieval is roughly 80 percent per GB per month. Run an automated lifecycle policy that moves data through tiers based on last-access timestamps. This is unglamorous work but consistently delivers 20 to 30 percent reductions in storage costs for organizations that have never done it.

Step 4: Fix the query patterns that are bleeding money

In columnar systems like BigQuery and Redshift, full-table scans on wide tables are an expensive habit. Partition your large tables by date or by a high-cardinality filter column that your analysts actually use. In BigQuery, partitioned tables combined with clustering can reduce bytes billed per query by 60 to 90 percent on common analytical patterns, according to Google's own documentation (Google being the vendor here, so treat this range as optimistic and validate against your actual workloads). Work with your data engineering team to add a query cost gate in your BI tool or notebook environment that warns users when a query will scan more than a defined threshold.

Step 5: Revisit your reserved capacity commitments

If you are on Snowflake, BigQuery, or Redshift, you almost certainly have the option to purchase reserved capacity at a discount versus on-demand pricing. The problem is that most organizations bought reservations based on peak usage projections that were too high. Run 90 days of actual consumption data before renewing or expanding any commitment. Right-sizing a Snowflake enterprise contract from a 200-credit reservation to 120 credits based on measured usage can represent hundreds of thousands of dollars in annual savings for mid-to-large deployments.

Pitfalls that will trip you up

The most common failure mode is optimizing development and staging environments aggressively while leaving production untouched because "we can't risk breaking anything." Production is where most of the money is. Engage your data engineering leads and establish a change-control process for production optimization rather than avoiding it.

A second trap is treating cost reduction as a one-time project. Cloud costs drift upward continuously as new pipelines are added and query volumes grow. Build cost metrics into your data team's sprint reviews the same way application teams track latency. Assign a named owner, not a committee.

Watch out for the "FinOps team handles it" deflection. FinOps teams are excellent at application and infrastructure costs, but data platform optimization requires domain-specific knowledge: understanding why a particular dbt model runs daily when it only needs to run weekly, or why a Fivetran connector is syncing a table at 15-minute intervals when downstream consumption is once per day. CDOs need to own this problem, not delegate it entirely to a finance-adjacent function.

Finally, do not cut costs by reducing data availability to analytical teams. That is not optimization, it is just degradation. Every cost measure should be evaluated against whether it affects query SLAs or data freshness for production dashboards.

Quick wins to start this week

  • Pull your top 50 most expensive queries from the past 30 days and flag any that do full-table scans on tables larger than 100GB
  • Set auto-suspend to 60 seconds on all non-production Snowflake warehouses today
  • Identify storage buckets or datasets with no read activity in the past 90 days and schedule a review meeting with data owners
  • Check whether your Fivetran or Airbyte sync schedules match actual downstream consumption frequency, not the default settings

The organizations that manage cloud data costs well in 2026 are not necessarily spending less than their peers, they are spending deliberately. Knowing exactly what each pipeline and query costs, and tying that cost to demonstrated business value, is what gives a CDO credibility when asking for budget to build something new.

Finished reading?

Validate your read to earn XP and feed your radar.