# Thinking before building: framing an AI problem
A support team at a mid-sized software company spent three months building an AI chatbot. They launched it, and within two weeks customers were furious. The bot gave vague answers, looped in circles, and made people angrier than before. When the team finally looked at the data, they found something embarrassing: 80% of incoming questions were the same 15 questions, all answered in their existing help docs. Customers just couldn't *find* those docs.
They didn't need a chatbot. They needed better search.
This is the most common way AI projects fail. Not bad technology. Bad framing. Someone says "let's add AI," and the team starts building before anyone asks "what problem are we actually solving?"
"Let's add a chatbot" is a solution. It is not a problem.
When you start with the solution, you skip the most important step: understanding what is broken and for whom. You end up building something impressive that nobody needed.
Solution-first sounds like this:
Problem-first sounds like this:
Notice the difference. The second list describes pain, who feels it, and how much. That gives you something to measure and solve. The first list is just a tool looking for a job.
Before you build anything, write down four things. This takes 20 minutes and saves months.
What is the specific task someone is trying to get done? Be concrete.
Bad: "Improve customer support."
Good: "Help a customer find the answer to a common question without waiting for a human."
Name the actual person. A frustrated customer at 11pm? An overworked support agent? A manager drowning in tickets? Each one points to a different solution.
We cover how to define a measurable success criterion and audit your data in detail in the lesson "Scoping: Data and Success Criteria."
This is the big one. Force yourself to ask: what is the *least* amount of technology that solves the problem? Often the answer involves no AI at all, or a tiny slice of it.
For the support team, the simplest fix was a smart search bar over their existing docs. Cheaper, faster, and it actually worked.
Let's redo the failed chatbot project the right way.
The Job: A customer wants the answer to a routine question (password reset, billing date, export limits) right now.
Who: Customers, often outside business hours, who would rather self-serve than wait.
Success metric: 50% of common questions resolved without a human, measured by ticket volume dropping.
Simplest thing that could work: A search box that understands plain-language questions and returns the right help article. Not a conversation. A search.
This is where a technique called semantic search fits perfectly. Semantic search means matching by *meaning*, not just keywords. A customer types "I can't log in" and it finds the article titled "Resetting your password," even though none of those words match. Traditional keyword search would miss it.
You can build a basic version of this with an embedding model (a tool that turns text into a list of numbers representing its meaning, so similar meanings sit close together). Here is a minimal example using OpenAI's APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → in Python:
from openai import OpenAI
client = OpenAI()
# Your existing help articles
docs = [
"Resetting your password: click Forgot Password on the login screen.",
"Your billing date is the day you first subscribed.",
"You can export up to 10,000 rows on the free plan.",
]
def embed(text):
return client.embeddings.create(
model="text-embedding-3-small",
input=text
).data[0].embedding
# A customer's question
question = "I can't log in to my account"
q_vec = embed(question)
doc_vecs = [embed(d) for d in docs]
# Find the closest article by meaning
import numpy as np
scores = [np.dot(q_vec, d) for d in doc_vecs]
best = docs[int(np.argmax(scores))]
print("Best match:", best)This returns the password article, even though the customer never said "password." That is the entire problem solved, with no chatbot, no conversation flow, and far less risk of the AI inventing wrong answers.
For a clear, free walkthrough of how embeddings and semantic search work, see OpenAI's embeddings guide.
Problem-first thinking does not mean "never build AI." It means build the right thing.
If your analysis showed that most questions were *unique*, involved back-and-forth ("my export failed, here's my error, what do I do?"), and required reasoning across multiple documents, then a conversational AI assistant genuinely earns its place.
The test is simple. Ask: does the problem require a conversation, or just an answer?
Most teams assume "conversation" by default. Most problems only need "answer."
Run any AI idea through these questions before building:
1. Can I state the problem without naming a technology? If you can only describe it as "an AI thing," you don't understand it yet.
2. Who feels the pain, and how often? Rare pain rarely justifies a project.
3. What's the dumbest solution that might work? Try that first. A spreadsheet, a better search bar, a saved prompt template.
4. What does success look like as a number? If you can't measure it, you can't improve it.
5. What happens when the AI gets it wrong? A wrong FAQ answer is annoying. A wrong medical or legal answer is dangerous. Higher stakes mean simpler, more controlled designs.
Knowledge check
1. In the opening story, why did the support team's chatbot fail despite three months of work?
2. According to the lesson, what is the key distinction between solution-first and problem-first statements?
3. The lesson mentions writing down a 'Job' as the first framing step. Which example best reflects a well-defined Job?
4. Select ALL correct answers. Which of the following are examples of problem-first thinking as described in the lesson?
Sélectionnez toutes les réponses correctes.
5. Select ALL correct answers. Based on the lesson, what makes the simple framing method valuable before building?
Sélectionnez toutes les réponses correctes.
Once you know the problem, match the tool to it. Going overboard wastes money and adds failure points. Going too small leaves the pain unsolved. Here is a rough ladder, from simplest to most complex:
A better-organized FAQ page. A search filter. A template. Sometimes the real problem is that information is hidden, not that it's missing. Always check this first.
One prompt, one answer. "Summarize this document." "Classify this email as urgent or not." No memory, no conversation. Cheap, predictable, easy to test.
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "Classify the support email as 'billing', 'technical', or 'other'. Reply with one word."},
{"role": "user", "content": "My card was charged twice this month."}
]
)
print(response.choices[0].message.content) # billingThis one snippet could automatically route every incoming email to the right team. That alone might solve the support team's real bottleneck.
This is the semantic search example from earlier, often called RAG (Retrieval-Augmented Generation: the AI looks up relevant info from your documents, then answers using it). Good when answers must come from *your* content, not the model's general knowledge.
Multi-turn chat, memory of the conversation, maybe the ability to take actions. The most powerful and the most expensive to build and maintain. Reserve it for problems that truly need dialogue.
The lesson: start at the lowest level that solves the problem. You can always climb up. Climbing down (ripping out an over-built chatbot) is painful and public.
Framing first feels slow. It is actually the fastest path. The support team lost three months building the wrong thing. The right framing would have pointed them at a search box they could have shipped in two weeks.
Every hour spent on the four framing questions (the job, who, the metric, the simplest fix) saves you weeks of building something nobody asked for.