Building a business case for an AI pilot
The scene: monday morning with the CFO
Your team wants to pilot a policy-renewal chatbot. It answers questions like "why did my premium go up" and "can I change my payment date" without a human agent. You've got a vendor demo, an enthusiastic product owner, and a Slack channel full of excitement.
Then the CFO asks four questions: What does this cost, fully loaded? What's the benefit, in dollars, not "efficiency"? How long until we see it? And what breaks if it doesn't work?
If you can't answer those in the first five minutes, the pilot doesn't get funded. This lesson builds the case the CFO will actually sign.
Start with the real cost stack, not the license fee
Vendors quote a subscription price. That's the smallest line item. A realistic cost stack for a policy-renewal chatbot pilot includes:
- Software/API costs: per-conversation or per-tokentokenA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition → fees if using a large language modellarge language modelA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition → (LLM, a model trained on text to generate human-like responses) APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → from a provider like OpenAI, Anthropic, or Google.
- Integration engineering: connecting the bot to the policy administration system, billing platform, and CRMCRMCustomer Relationship Management: software and strategy to manage and analyse customer interactions throughout their lifecycle.View full definition → (customer relationship management software). This is usually the biggest hidden cost, often 2 to 4 times the software fee for a mid-sized carrier.
- Data preparation: cleaning FAQ libraries, call transcripts, and policy wording so the bot has accurate source material. Insurers often underestimate this by half.
- Compliance and legal review: state insurance departments in the US, and regulators like the UK's FCA (Financial Conduct Authority) or the EU's forthcoming AI Act obligations, may require disclosure that a customer is speaking with an AI, plus human escalation paths.
- Human-in-the-loop staffing: someone must review flagged conversations, especially anything touching coverage advice or claims, during the pilot period.
- Change management: training contact center staff on new escalation workflows.
Worked example (illustrative estimate, not a real vendor quote):
| Cost item | Estimated range (90-day pilot, mid-sized carrier) |
|---|---|
| LLM API usage | $8,000 to $15,000 |
| Integration engineering | $40,000 to $90,000 |
| Data prep | $15,000 to $30,000 |
| Compliance review | $10,000 to $20,000 |
| Human oversight (0.5 FTE for 3 months) | $20,000 to $25,000 |
| Total | roughly $93,000 to $180,000 |
These are illustrative bands based on typical mid-market implementation patterns, not a quote for any specific vendor or carrier. Your actual numbers depend heavily on how messy your legacy systems are.
Define the benefit in terms the CFO trusts
"Improves customer experiencecustomer experienceThe overall perception a customer forms of your brand across every interaction, from first touch to post-purchase support.View full definition →" doesn't survive budget season. Convert benefits into three measurable buckets.
1. Cost deflection
Each chatbot conversation that resolves a query without a live agent saves an estimated $3 to $8 in contact center labor cost (industry estimate, varies by market and wage structure). If the bot handles 10,000 renewal queries a quarter and successfully resolves 40% without escalation, that's 4,000 deflected contacts.
Simple calculation:
4,000 deflected contacts × $5 average savings = $20,000 per quarter, or $80,000 annualized.
Compare that to your cost stack above. In year one, you may not break even. That's normal and the CFO knows it. The pilot's job is to validate the deflection rate assumption, not to prove profitability on day one.
2. Retention lift
Renewal friction is a real churn driver. If a chatbot reduces average response time on renewal questions from 48 hours (email queue) to under a minute, some carriers report modest retention improvements, commonly cited in the 1 to 3 percentage point range for the segment touched, though this varies widely and should be treated as a hypothesis to test, not a promise.
3. Agent capacity redeployment
Deflected volume doesn't just save money, it frees agent time for retention calls, cross-sell conversations, or complex claims. This benefit is real but easy to double-count. Be explicit: are you cutting headcount, or reallocating it? The CFO will ask which.
Time-to-value: what a CFO actually wants to see
CFOs don't fund vague roadmaps. They fund milestones with dates and exit criteria.
A defensible timeline for a renewal chatbot pilot:
- Weeks 1 to 4: data integration and content grounding (feeding the bot accurate, current policy and billing information).
- Weeks 5 to 8: limited release to 5 to 10% of renewal traffic, with human fallback on every interaction.
- Weeks 9 to 12: measure deflection rate, containment rate (percentage of chats that don't need a human), customer satisfactioncustomer satisfactionCustomer Satisfaction Score, a direct measure of satisfaction captured right after a specific interaction or experience, usually on a short rating scale.View full definition →, and error rate on regulated content (e.g., misstating a coverage term).
- Week 12 decision gate: go/no-go based on pre-agreed thresholds, not vibes.
Set the thresholds before you start. For example: proceed to scale-up only if containment rate exceeds 35% and zero critical compliance errors occur. This protects you from sunk-cost pressure to keep funding a pilot that isn't working.
What breaks: the risk register the CFO wants to see
- Hallucination risk: the bot confidently states something false about coverage terms. This is the single biggest reason insurers keep humans in the loop during pilots. The NAIC (National Association of Insurance Commissioners) has issued model bulletins on AI use in insurance that many US states have adopted, generally requiring documented testing and human oversight for consumer-facing AI.
- Regulatory disclosure: several jurisdictions require telling customers they're interacting with an AI system, and some, like Colorado's algorithm and predictive model regulations for insurers, require documentation of testing for unfair discrimination.
- Data privacy: renewal chatbots touch personal and financial data. In the EU, GDPRGDPREU regulation governing how organizations collect, store and use personal data, with fines tied to global revenue for breaches.View full definition → (General Data Protection Regulation) governs this; in the US, it's a patchwork of state privacy laws.
- Vendor lock-in: switching LLM providers mid-pilot can mean re-testing everything. Build this into your exit-cost assumptions.
A one-page risk register with owner names next to each item is often what gets a skeptical CFO to "yes."
Knowledge check
1. Why does the lesson insist on building a 'real cost stack' rather than relying on the vendor's license quote?
2. A product team estimates the pilot's data preparation cost by counting hours to clean the existing FAQ library. What conceptual mistake are they most likely making?
3. Why does the lesson frame the CFO's four questions (cost, dollar benefit, time-to-value, and failure risk) as the real test of whether a pilot gets funded?
4. Select ALL correct answers about why integration engineering is often the biggest hidden cost in an AI pilot like a policy-renewal chatbot.
Select all the correct answers.
5. Select ALL correct answers about compliance and human-in-the-loop considerations for an AI pilot in insurance.
Select all the correct answers.
Building the one-page business case
Put these five things on a single page:
- Problem statement: renewal queries take 48 hours to resolve by email; agents spend 30% of time on repetitive renewal questions (use your own contact center data here, don't invent a number).
- Cost: fully loaded pilot cost, with a range, not a false-precision point estimate.
- Benefit hypothesis: deflection rate target, cost-per-contact saved, retention hypothesis, each labeled as an estimate to be tested.
- Timeline with decision gates: 90 days, explicit go/no-go thresholds.
- Risk register: top 3 to 5 risks with named owners and mitigations.
This format works because it matches how CFOs evaluate any capital request: cost, benefit, time, and downside. It also builds your credibility for the next pilot, since you're establishing a track record of honest forecasting rather than hype.
For a broader look at how to structure AI pilot evaluation frameworks, the OECD's AI policy observatory has useful sector-neutral material on risk-tiered AI evaluation that maps well onto insurance use cases.
🎬 [VIDEO: "How Insurers Are Using AI Chatbots for Customer Service" - youtube.com - search for recent industry panel discussions (2024 to 2025) featuring carrier executives discussing real deployment lessons and ROIROIReturn on Investment: the ratio of net profit to the cost of an investment. A 300% ROI means each dollar invested returns $3.View full definition → timelines]
Key Takeaways
- Cost a pilot on the full stack (integration, data prep, compliance, human oversight), not just the vendor license. Integration is usually the largest hidden line item.
- Convert benefits into deflection rate, cost-per-contact saved, and retention hypotheses, each explicitly labeled as an estimate to validate, not a guaranteed return.
- Set go/no-go thresholds before the pilot starts (for example, containment rate and zero critical compliance errors) so the decision at week 12 is mechanical, not political.
- Name real regulatory obligations relevant to your market (NAIC model bulletins, state AI regulations like Colorado's, GDPR in the EU) in your risk register, with an owner assigned to each.
- A CFO funds a one-page case with cost, benefit hypothesis, timeline, and risk register far more readily than a long deck full of "efficiency gains."