# Automating customer support in regulated finance
A customer messages your bank at 2 a.m.: "Someone charged $340 to my card at a store I've never visited. I want my money back." That single message triggers a legal clock. Under Regulation E (Reg E, the federal rule governing electronic fund transfers and consumer error resolution in the U.S.), the bank has strict deadlines to investigate and, in many cases, must provide provisional credit within 10 business days.
Now imagine an LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.Voir la définition complète → (large language modellarge language modelA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions., the AI behind chatbots like ChatGPT) fielding that message. Get it right and you resolve disputes faster, cheaper, and around the clock. Get it wrong and you have missed a regulatory deadline, given unauthorized advice, or promised a refund you cannot honor.
This lesson shows how to build a support agent that stays inside the lines.
In most industries, a chatbot mistake means an annoyed customer. In regulated finance, a mistake can mean a compliance violation (breaking a rule enforced by regulators like the CFPB, the Consumer Financial Protection Bureau).
Three boundaries matter most:
The core design principle: the LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.Voir la définition complète → is a conversation layer, not a decision maker. It explains, gathers, and routes. It does not adjudicate disputes or improvise policy.
The winning pattern here is RAG (retrieval-augmented generation), where the model answers using text pulled from your approved documents rather than its own memory.
Why RAG matters: a raw LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.Voir la définition complète → will confidently state a "45 day" deadline it half-remembers from training data. That is called a hallucinationhallucinationA hallucination is when an AI model generates output that is fluent and confident but factually wrong, fabricated, or unsupported by its source data.Voir la définition complète → (a confident but fabricated answer). RAG forces the model to quote from your actual, current policy.
Here is the flow:
1. Customer message arrives.
2. System retrieves relevant approved text: your Reg E procedures, dispute FAQ, and required disclosures.
3. The LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.Voir la définition complète → drafts a reply grounded only in that retrieved text.
4. Guardrails (automated checks) inspect the draft before it reaches the customer.
5. High-risk cases route to a human.
Guardrails are the difference between a demo and a deployable system. Build them in layers:
A simple version of the output check:
BLOCKED_PATTERNS = [
r"you should (invest|buy|sell|move your money)",
r"(guarantee|guaranteed|you will definitely get)",
r"this is (financial|legal|investment) advice",
]
def passes_guardrails(draft, retrieved_docs):
# Must be grounded in retrieved policy
if not retrieved_docs:
return False, "no_source"
# Must not contain advice or guarantee language
for pattern in BLOCKED_PATTERNS:
if re.search(pattern, draft, re.IGNORECASE):
return False, "advice_or_guarantee"
return True, "ok"This is deliberately blunt. Regex checks catch obvious violations cheaply. Pair them with a second LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.Voir la définition complète → acting as a classifier ("Does this reply give investment advice? Yes/No") for nuance.
Let's walk the 2 a.m. message through the system.
Step 1: Classify. The agent recognizes an unauthorized transaction claim. This is a formal dispute, not a general question.
Step 2: Gather structured facts. The agent asks scripted, compliance-approved questions: transaction date, amount, merchant, whether the card is still in the customer's possession. It does not ad-lib.
Step 3: State the process, not the outcome. The agent explains what happens next using retrieved, approved language:
> "I've opened a dispute for the $340 charge. We will investigate and follow the timelines required under federal Regulation E. You may be entitled to provisional credit while we investigate. A specialist will follow up."
Notice what it did not say: it did not promise a refund, quote a specific outcome, or invent a deadline.
Step 4: Log everything. Every message, timestamp, and retrieved source is stored. In finance, if it is not logged, it did not happen. Regulators expect an audit trail.
Step 5: Route. The formal investigation belongs to a trained human or a separate rules-based system. The LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.Voir la définition complète →'s job ends at intake and explanation.
For the underlying rules, the CFPB publishes plain-language guidance on electronic fund transfers and error resolution. Ground your retrieval corpus in the actual regulatory text, not a blog summary.
🎬 [VIDEO: "How Retrieval-Augmented Generation (RAG) Works" — youtube.com — a clear, non-technical explainer of grounding LLMs in your own documents]
You cannot ship this and walk away. Regulated support agents need continuous evaluation.
Red-team before launch. Deliberately try to break the agent. Ask it for stock tips. Claim a dispute and then try to get a refund promise. Use hostile phrasing. Log every failure and add a guardrail.
Build an evaluation set. Collect real (anonymized) support transcripts. Label the correct behavior. Run the agent against them after every change. This catches regression (a fix that quietly breaks something else).
Monitor in production. Track the human handoff rate, the guardrail block rate, and any customer complaints about the bot. A sudden drop in handoffs might mean the agent is overstepping.
Keep a human in the loop for the gray zone. The goal is not zero humans. It is having humans handle the 15 percent of cases that carry 85 percent of the risk.
Vérification des acquis
1. The lesson describes the LLM as a 'conversation layer, not a decision maker.' What is the primary reason for this design principle in regulated finance?
2. Why is a chatbot error in regulated finance treated as fundamentally different from a chatbot error in most other industries?
3. A customer reports an unauthorized $340 charge at 2 a.m. Why does this single message create a 'legal clock' the automated agent must respect?
4. Select ALL correct answers. Which of the following would be appropriate roles or behaviors for the LLM conversation layer as described in the lesson?
Sélectionnez toutes les réponses correctes.
5. Select ALL correct answers. Which of the following are boundaries the lesson identifies as most important for a finance support agent?
Sélectionnez toutes les réponses correctes.
The eager refunder. The model, trying to be helpful, tells the customer their money is on the way. Fix: block guarantee language and never let the LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.Voir la définition complète → state outcomes.
The stale policy. Your Reg E procedure changed, but the retrieval corpus still has last year's version. Fix: treat the document store as a controlled, versioned asset with an owner and update schedule.
The confident guesser. Retrieval returns nothing relevant, so the model fills the gap from memory. Fix: force a handoff when grounding fails. Silence plus a human is safer than a fluent wrong answer.
The advice creep. A customer asks, "What should I do with the refund?" and the bot suggests a product. Fix: classify any forward-looking money question as advice and decline politely.
The disclosure paraphraser. The model "improves" a required disclosure into something clearer but legally different. Fix: inject required text verbatim and forbid the model from editing it.
Done right, an LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.Voir la définition complète → support agent handles the high-volume, low-complexity work: explaining processes, gathering dispute details, answering "where is my statement." Human specialists focus on judgment calls and edge cases.
The value is not just cost. It is consistency. A well-grounded agent gives the same compliant answer at 2 a.m. as at 2 p.m., every time, with a full audit log. That consistency is itself a compliance asset.