Using data for competitive intelligence at scale
Your competitors are telling you more than they realize.
Every job posting they publish reveals their strategic priorities. Every patent they file maps their innovation roadmap. Their pricing tells you their margin strategy. Their web traffic trends predict their revenue before their earnings call.
Competitive intelligence has always existed. What's changed is that data has made it systematic, real-time, and scalable.
The CI data sources that actually matter
Job posting analysis
Job postings are one of the most underused competitive intelligence sources. When a competitor posts 50 data science roles and 20 ML engineer positions, they're signaling a strategic shift toward AI. When they post 30 customer success roles, they're investing in retention. When they post 0 engineering roles for a quarter, they may be in financial distress.
Platforms like LinkedIn Talent Insights, Lightcast (formerly Burning Glass Technologies, which merged with Emsi in 2021 and rebranded in 2022), and Revelio Labs aggregate and analyze job posting data at scale. Hedge funds use this data to predict company performance before earnings, and so should your strategy team.
Web traffic intelligence
SimilarWeb, Semrush, and Ahrefs provide estimated web traffic, traffic sources, engagement metrics, and keyword rankings for any domain. Track whether a competitor's direct traffic is growing (brand strength) or declining (customer dissatisfaction). See which channels they're investing in. Identify which content is driving their growth.
Share of searchShare of searchThe proportion of searches in your category that mention your brand, a leading indicator of market share.View full definition → as market sharemarket shareThe percentage of total industry sales your company captures in a given period. It measures competitive position relative to rivals in a defined market.View full definition → predictor
Les Binet's research (adam&eveDDB) demonstrated a statistically significant correlation between "share of search", the percentage of searches in a category going to your brand, and market share. The relationship has a 3-6 month lag: share of search moves first, market share follows.
This gives you a leading indicator for competitive position that traditional market research, which lags by months, can't provide.
Patent and trademark filings
Patent filings reveal R&D priorities months or years before products launch. USPTO, EPO, and WIPO databases are publicly searchable. Companies like PatSnap and Derwent Innovation have built analysis tools on top of these databases. Amazon's patent portfolio for drone delivery predicted their logistics automation strategy years before it became public.
Knowledge check
1. According to the lesson, what has fundamentally changed about competitive intelligence in the data era?
2. Why is 'share of search' considered a valuable competitive metric?
3. A competitor suddenly posts dozens of data science and ML engineer roles. What does this most likely signal?
4. Select ALL of the following that are described as legitimate, publicly available competitive intelligence data sources.
Select all the correct answers.
5. Select ALL insights that patent and trademark filings can provide to a strategy team.
Select all the correct answers.
Building a systematic CI program
A systematic competitive intelligence data program involves:
Data collection layer: APIs from SimilarWeb, LinkedIn Talent Insights, Brandwatch (social listening), and pricing monitoring tools (Prisync, Wiser). Most have API access. Build connectors into your existing data infrastructure.
Integration layer: A simple data pipelinedata pipelineETL (Extract, Transform, Load) is a data integration process that pulls data from sources, reshapes it into a consistent format, and writes it into a target system.View full definition → pulling from these sources daily or weekly into your data warehousedata warehouseA central repository that consolidates data from many source systems into a structured, query-optimized store designed for analytics, reporting, and business intelligence.View full definition →. This is a relatively straightforward ELT job, 2-4 weeks of engineering.
Analysis layer: Dashboards tracking key competitive metrics, updated automatically, accessible to strategy and leadership teams.
Intelligence layer: Weekly or monthly reports synthesizing signals into actionable intelligence. What is the competitor doing? What does it mean? What should we do?
Total build cost for a basic CI data program: $30-80K in tool licensing plus 1-2 months of engineering time. The ROIROIReturn on Investment: the ratio of net profit to the cost of an investment. A 300% ROI means each dollar invested returns $3.View full definition → can be immediate: if competitive intelligence informs a single pricing decision or product launch timing, the payback is instantaneous.
The ethical boundaries
CI is legal when drawn from public sources. It becomes legally and ethically problematic when it involves:
- Accessing systems without authorization (Computer Fraud and Abuse Act in the US)
- Using confidential information from former competitor employees
- Scraping in violation of Terms of Service
- Privacy violations: collecting personal data without consent
The distinction: analyzing a competitor's published job postings = competitive intelligence. Bribing their employee for internal data = corporate espionage. The CDO's role includes building the ethical guardrailsguardrailsRules and controls that keep an AI system inside safe, legal and on-brand boundaries, blocking outputs and actions that cross the line.View full definition → around CI programs.
Key Takeaways
- Public data (job postings, web traffic, patents, share of search) reveals competitor strategy weeks or months before it becomes obvious.
- Job posting analysis runs through platforms like LinkedIn Talent Insights, Lightcast (the former Burning Glass) and Revelio Labs.
- A basic CI data program costs $30-80K in licensing plus 1-2 months of engineering, and can pay back on a single pricing or timing decision.
- The line between CI and espionage is the source: published data is fair game, unauthorized access and bribery are not. The CDO owns those guardrails.