# Why manufacturing AI projects stall after the pilot
A computer-vision system for detecting weld defects hit 98% accuracy in a pilot plant in Ohio. Line supervisors loved it. Six months later, the same model deployed to a sister plant in Poland failed so badly that operators started bypassing it, manually approving parts the system flagged as defective just to keep the line moving. Same vendor, same model, same defect type. What went wrong?
Nothing about the algorithm changed. What changed was the camera angle, the ambient lighting, the metal alloy's surface finish, and the fact that the Polish plant's quality team had never been consulted on how the tool would change their job. This is the single most common failure pattern in manufacturing AI, and it explains why so many projects never make it past a lonely pilot line.
Estimates vary, but multiple industry surveys (Gartner, McKinsey, BCG, circa 2023 to 2025) consistently put the share of AI pilots that scale into full production deployments somewhere between 20% and 30%. That means roughly 7 in 10 manufacturing AI pilots that "work" never become operational systems used across multiple sites.
This isn't unique to manufacturing, but manufacturing has features that make the gap worse:
The Ohio-to-Poland failure is a data drift problem: the statistical properties of new data no longer match what the model was trained on. In computer vision this shows up as lighting, camera angle, part orientation, or surface finish differences.
Good evaluation practice catches this before rollout, not after:
Pilot validation checklist (vision QC example):
1. Test model on data from 2+ physical sites, not just the pilot site
2. Vary lighting conditions and camera angles in test set
3. Include seasonal variation if outdoor or unconditioned space
4. Report accuracy PER SITE, not blended average
5. Define minimum acceptable accuracy before deployment, in writingThe blended average is the trap. A model that's 98% accurate in Ohio and 81% in Poland might report as "91% average accuracy," which sounds fine and hides the real problem.
Pilots are almost always run on borrowed infrastructure: a laptop, a spare camera, a data scientist's laptop running a Jupyter notebook. Scaling means connecting to the MES, the PLC network, and often an Enterprise Resource Planning (ERP) system that was configured a decade ago by a vendor who no longer exists.
A rough rule of thumb from manufacturing AI implementers: the pilot itself is often 10 to 20% of total project cost. Integration, change management, and multi-site rollout account for the rest. Projects that budget for the pilot but not the scaling phase run out of money and executive patience at exactly the point where the real value would appear.
Simple worked example. Suppose a pilot for defect detection costs $150,000 (estimate, illustrative) and shows a projected annual saving of $400,000 per line from reduced scrap. That looks like a strong return. But scaling to 12 plants requires per-site camera installation, network upgrades, and retraining local staff, estimated at $80,000 per site. That's $960,000 in rollout cost alone, before any software licensing. The per-line saving may also shrink outside the best-case pilot site. The pilot's ROIROIReturn on Investment: the ratio of net profit to the cost of an investment. A 300% ROI means each dollar invested returns $3.View full definition → and the program's ROIROIReturn on Investment: the ratio of net profit to the cost of an investment. A 300% ROI means each dollar invested returns $3.View full definition → are different calculations, and only the second one matters to a CFO.
Operators who fear being replaced, or who were never asked how a tool should work, will find ways to defeat it. This is well documented in adoption research (see MIT Sloan Management Review's ongoing coverage of AI and workforce adoption) and is not unique to manufacturing, but the shop floor makes resistance easy: mark the sensor as faulty, override the flag, or simply stop entering data consistently.
Successful rollouts treat the frontline worker as a stakeholder in system design, not an end user who receives a finished tool. Practical measures that show up in successful programs:
Pilots are usually run by a data science team, an innovation group, or a vendor's solutions engineer. Once the pilot proves feasibility, ownership needs to shift to plant operations and IT. This handoff frequently doesn't happen, and the system becomes an orphan: no one is responsible for retraining the model when a new part number is introduced, no one owns the maintenance contract, no one escalates when accuracy degrades.
A useful governance question for any manufacturing AI deployment: who is responsible for this model's accuracy six months after go-live, and what's their monitoring cadence? If there's no named answer, the project is not ready to scale.
Knowledge check
1. In the Ohio-to-Poland weld defect example, what was the root cause of the model's failure at the second plant?
2. Why does manufacturing tend to have a worse pilot-to-scale gap than many other industries adopting AI?
3. What does the operators' behavior of bypassing the flagged parts at the Polish plant most directly illustrate?
4. Select ALL correct answers about why a manufacturing AI pilot that achieves high accuracy in one plant might fail at another plant.
Select all the correct answers.
5. Select ALL correct answers about the statistic that only 20-30% of manufacturing AI pilots scale into full production.
Select all the correct answers.
Before committing capital to a multi-site rollout, a credible evaluation covers:
1. Multi-site data validation. Test on data the model has never seen, from at least one additional physical location.
2. Total cost of scale, not just pilot cost. Integration, per-site hardware, retraining, and ongoing monitoring.
3. A defined accuracy floor with a rollback plan. What accuracy triggers a pause? Who decides?
4. Workforce engagement plan. Who was consulted, and what changes to their daily workflow are expected?
5. Named post-pilot owner. A person or team accountable for the system after the pilot team moves on.
Manufacturers who apply this rigor tend to run fewer, larger pilots with cross-functional buy-in from the start, rather than many small pilots that each need to be re-sold to skeptical plant managers. Siemens, Bosch, and Schneider Electric have all published case material (via their own investor and innovation communications) emphasizing multi-site validation as a precondition for scaling vision and predictive-maintenance systems, a signal that this lesson has been learned expensively elsewhere.
🎬 [VIDEO: "Why AI Projects Fail in Manufacturing" - youtube.com - Search for recent (2024-2025) manufacturing-AI conference talks from MESA International or industry analysts covering pilot-to-scale failure patterns, a good visual complement to this lesson's failure taxonomy.]
AI vendors often price and pitch based on pilot-site performance. Treat any vendor's headline accuracy number as a ceiling, not a guarantee. Ask specifically: "What was accuracy on sites you did not use for training or tuning?" If the vendor doesn't have an answer, that itself is diagnostic.