AI for fraud detection and eligibility determination

The critical insight: these two jobs have opposite failure modes, and confusing them is how MiDAS went wrong.

• A wrong eligibility denial hurts a legitimate person who needs help.
• A missed fraud case wastes public money.

You cannot maximize both at once. Every model design choice trades one against the other.

The confusion matrix, in plain terms

Every fraud or eligibility model produces four outcomes. Understanding them is non-negotiable for anyone overseeing these systems.

| | Model says fraud/ineligible | Model says legitimate/eligible |

|---|---|---|

| Actually fraud/ineligible | True positive (caught) | False negative (missed) |

| Actually legitimate/eligible | False positive (wrongful denial) | True negative (correct) |

MiDAS was catastrophically tuned toward the bottom-left cell: false positives, wrongful accusations of innocent people.

Two metrics matter here:

Precision: Of everyone the model flagged, what fraction were actually fraudulent? Low precision means many innocent people get caught.

Recall: Of all actual fraud, what fraction did the model catch? Low recall means real fraud slips through.

MiDAS effectively pushed recall up (flag everything suspicious) at the expense of precision (accuracy of those flags). The result was tens of thousands of false accusations.

Why "recover the money" is the wrong objective

If you tell an engineering team "recover as much misspent money as possible," they will build a system that over-flags. Flagging more claims recovers more dollars in the short term, even if most flags are wrong, because penalties and clawbacks generate revenue.

This is a misaligned objective function. The objective function is the single quantity a model is trained to optimize. If that quantity is "dollars recovered," the model has no reason to care about wrongful denials.

A better objective weighs the real cost of each error type. In public benefits, the cost of a false positive is usually much higher than a missed fraud case, because:

• A wrongful denial can cause eviction, hunger, or bankruptcy.
• It falls disproportionately on vulnerable people.
• It creates legal liability (Michigan paid a settlement estimated at around 20 million dollars).
• It erodes public trust in the entire program.

You encode this by assigning an explicit cost to each cell of the confusion matrix, then choosing the decision threshold that minimizes total expected cost, not maximizes dollars recovered.

The point of the snippet is not the code. It is that someone must consciously decide those two cost numbers. In MiDAS, no one did, so the system implicitly set the cost of a wrongful denial near zero.

Design principles that would have stopped MiDAS

1. Keep a human in the loop for adverse actions

MiDAS auto-adjudicated fraud with no human review. Any decision that harms a person (denial, clawback, penalty) should require human sign-off. AI can prioritize cases; it should not pronounce guilt.

2. Flag for investigation, do not conclude fraud

Reframe the model's output. It does not detect fraud. It identifies cases that merit a human look. A data mismatch (an employer reports different wages than the claimant) is a question, not proof.

3. Match the burden of proof to the harm

Higher penalties should require higher confidence and more evidence. MiDAS applied quadruple penalties on thin automated signals. That is backwards.

4. Test for disparate impact

A model can be accurate overall yet wrong far more often for one group. In benefits programs this often tracks race, disability, or language. Measure error rates by subgroup before deployment, not after a lawsuit.

5. Give people a real appeals path

Many MiDAS victims could not reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → a human or understand why they were flagged. A right to explanation and a fast, functioning appeals process is a design requirement, not a courtesy.

Building the eligibility side responsibly

Eligibility is often simpler than fraud, because the rules are written in law. That is an advantage: you can build much of it as transparent, auditable logic rather than an opaque model.

Prefer rules where the law is clear. If SNAP eligibility depends on income below a defined threshold for a given household size, encode that rule directly. It is explainable, testable, and defensible in an appeal. Do not use a black-box model to reproduce a rule you can write in one line.

Use models to assist, not replace, caseworkers. Machine learning is useful for triage (which applications look complete, which need follow-up documents) and for catching data entry errors. It should surface information to a caseworker, not lock a person out.

Design for missing and messy data. Many wrongful denials come from bad data, not fraud: a mismatched Social Security number, an employer's late filing, a name spelled two ways. Treat a data conflict as a prompt to ask the applicant, not as a verdict.

Verify against authoritative sources carefully. Cross-matching against wage databases or death records is common. But these sources contain errors too. The U.S. GAO has documented recurring accuracy problems in the very databases agencies rely on for verification. Never treat a single match as ground truth.

A practical governance checklist

Before any public benefits AI goes live, an oversight body should confirm:

• The objective function explicitly weighs wrongful denials, not just dollars recovered.
• Every adverse action requires human review.
• Subgroup error rates have been measured and disclosed.
• A plain-language explanation is available to every affected person.
• An appeals process exists and has been load-tested for volume.
• The system is monitored after launch, with a kill switch if error rates spike.

Michigan lacked nearly all of these. The technology was not the root failure. The absence of governance around it was.

Key takeaways

• Fraud detection and eligibility are different problems with opposite failure modes. A wrongful denial harms a person; a missed fraud case wastes money. Design for both, and know which one your objective is quietly favoring.

• "Recover the money" is a dangerous objective function. It rewards over-flagging. Assign an explicit, higher cost to wrongful denials and tune the decision threshold to minimize total harm.

• AI should flag cases for humans, not adjudicate them. Every action that harms a person needs human review and a burden of proof that scales with the penalty.

• Bad data, not fraud, causes many false accusations. Treat data mismatches as questions to resolve with the applicant, not as proof of wrongdoing.

• Governance is the real safeguard. Subgroup error testing, plain-language explanations, working appeals, and post-launch monitoring are what separate a useful triage tool from the next MiDAS.