# Evaluating AI vendors and build-versus-buy decisions
A fleet operator signs a three-year deal for a connected-car analytics platform. Year two, they want to move predictive-maintenance alerts to a cheaper provider. They discover the vendor owns the trained model, keeps the telematics history, and charges a six-figure fee to export raw sensor data. The analytics were good. The contract was the trap.
This lesson gives you a scoring method to avoid that trap: how to benchmark a connected-car AI offering, read the data ownership terms, spot model lock-in, and decide whether to license, partner, or build.
License (buy): You pay a vendor for a finished AI product. Fast, low upfront cost, least control. Example: subscribing to a driver-drowsiness detection APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → from a supplier.
Partner: You co-develop with a vendor or Tier 1 supplier (a company that sells complete systems directly to automakers, like Bosch or Continental). Shared IP, shared risk. Example: an OEM (Original Equipment Manufacturer, the carmaker itself) working with a mapping firm to fine-tune an autonomous-parking model on its own fleet data.
Build (in-house): You hire the team and own everything. Highest cost and slowest, but full control. Example: Tesla training its own vision models on fleet video.
Most decisions are not binary. You often license the commodity layer and build the differentiating layer.
Before scoring a vendor, confirm the use case is real, not a demo. AI genuinely earns its keep in:
If a vendor pitches "AI" for something a simple rules engine already solves (for example, "alert me when oil pressure drops below X"), that is not a build-versus-buy question. That is off-the-shelf software.
Vendors quote accuracy figures. Force them into your context.
Ask for performance on your data distribution, not their marketing set. A defect-detection model at 99 percent accuracy on a clean dataset may collapse on your specific paint color under your factory lighting.
Three questions that separate real from theater:
1. What is the false-positive rate? A predictive-maintenance model that cries wolf will get ignored by technicians within weeks. Ask for precision and recall, not just accuracy.
2. How does it degrade? Models drift as vehicles age and driving patterns shift. Ask how often they retrain and who pays for it.
3. Can we run a shadow pilot? Run the vendor model alongside your current process for 60 to 90 days before committing. No pilot, no deal.
Say a vendor claims their battery-degradation model flags at-risk EV (electric vehicle) packs. You run a 1,000-vehicle shadow pilot. Results:
Precision = true positives / all flagged = 40 / (40 + 160) = 20 percent.
Recall = true positives / all actual failures = 40 / (40 + 10) = 80 percent.
So the model catches most failures (good recall) but 80 percent of its alerts are noise (poor precision). For a warranty-cost use case where a missed failure is expensive, that may be acceptable. For a customer-facing alert, it would erode trust fast. The number only means something against your cost of a false alarm versus a miss.
This is where most automotive AI deals go wrong. Connected vehicles generate enormous telematics volumes, and whoever controls that data controls the future model.
Check the contract for:
🎬 [VIDEO: "The EU Data Act explained" - youtube.com - short overview of how the Data Act changes rights over connected-device and vehicle data]
Lock-in is the switching cost you cannot see until you try to leave.
| Lock-in factor | Low risk | High risk |
|---|---|---|
| Data ownership | You own raw data | Vendor owns it |
| Model portability | Standard formats, exportable | Proprietary, non-exportable |
| Integration depth | APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition →-based, swappable | Hardwired into your systems |
| Retraining control | You can retrain or move | Vendor-only, black box |
Score each row. If most of your answers land in the right column, you are not buying a tool, you are renting a dependency.
Rate each dimension 1 to 5, then weight by what matters for the use case.
Vendor Score = (Performance x 0.30)
+ (Data ownership x 0.25)
+ (Lock-in resistance x 0.20)
+ (Integration effort x 0.15)
+ (Total cost of ownership x 0.10)
Decision guide (illustrative, not a hard rule):
>= 4.0 -> License with confidence
3.0-3.9 -> Partner and negotiate ownership terms
< 3.0 -> Consider building the differentiating layerAdjust the weights to your context. For a core ADAS capability that defines your brand, performance and ownership dominate. For back-office fleet reporting, cost and integration effort matter more.
Knowledge check
1. The fleet operator's connected-car story illustrates which core lesson about vendor evaluation?
2. An OEM wants full control over a vision capability that differentiates its cars from competitors, and has the budget and talent to sustain a team. Which option best fits this situation?
3. Why does the lesson say most build-versus-buy decisions are 'not binary'?
4. Select ALL correct answers about the distinction between licensing and partnering.
Select all the correct answers.
5. Select ALL correct answers about contract terms a buyer should scrutinize before signing an AI vendor deal.
Select all the correct answers.
Building AI in-house is expensive and slow. It makes sense only when three conditions hold together:
1. The capability is a differentiator. If it is core to how you compete (Tesla and its vision stack, or an insurer's proprietary risk model), owning it matters.
2. You have the data advantage. Building perception models is pointless without labeled fleet data at scale. Data, not talent, is usually the bottleneck.
3. You can retain the team. Automotive firms compete with big tech for machine-learning engineers and often lose. A half-staffed build is worse than a good license.
The honest default for most non-tech automotive players is: license the commodity (speech, standard object detection), partner on the semi-custom (fleet-specific maintenance models), and build only the crown jewel.
Do not model AI ROI as instant. Expect:
A predictive-maintenance program that reduces unplanned fleet downtime is measurable: fewer roadside failures, lower warranty claims. Insist the vendor agree upfront on the metric you will judge success by, and the baseline you are measuring against. "It felt better" is not ROIROIReturn on Investment: the ratio of net profit to the cost of an investment. A 300% ROI means each dollar invested returns $3.View full definition →.