# What llms are great at (and what they are not)
Ask ChatGPT, Claude, or Gemini how many times the letter "r" appears in "strawberry," and there's a real chance it answers "two." The correct answer is three. This is a model that can draft a legal summary, explain quantum tunneling, and write working code, yet it stumbles on something a six-year-old gets right.
That contradiction is the whole lesson. Once you understand *why* it happens, you can predict where these tools shine and where they quietly fail.
LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions. stands for "." It's the engine behind ChatGPT, Claude, and Gemini. In plain terms: it's a system trained to predict the most likely next chunk of text, based on patterns in a huge amount of writing.
It is not a calculator. It is not a database. It is a very sophisticated pattern-completer for language.
Keep that one idea in your head and most of its behavior stops being surprising.
LLMs don't read letters the way you do. Before processing your text, they break it into tokenstokensA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition →: chunks of text, often whole words or word-pieces. "Strawberry" might become one or two tokenstokensA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition →, not seven individual letters.
So when you ask it to count "r"s, it's working with a blurry, pre-chunked version of the word. It never really saw the letters one by one. It's guessing based on patterns, and guessing is exactly the wrong tool for an exact count.
The same blind spot shows up with:
48,271 × 9,330)Try this and watch it fumble:
> "How many words are in this sentence, and how many start with a vowel?"
It will often give a confident, wrong number.
Modern versions of ChatGPT, Claude, and Gemini can run code or call a calculator behind the scenes. When they do, accuracy jumps, because the actual math is handled by a real computer, not by pattern-guessing.
If you have a coding-capable model, this prompt forces the reliable path:
Count the letter "r" in "strawberry" by writing and running
Python code, then show me the result.word = "strawberry"
print(word.count("r")) # 3The lesson: when you need an exact number, ask the model to *use a tool*, or do that step yourself.
Now the good news. The same pattern-completion that fails at counting is *brilliant* at language work. These are the wins you should lean on every day.
This is arguably the single most useful thing LLMs do. Paste in clumsy text and ask for a specific tone.
> "Rewrite this email to be warmer and shorter, and remove the apology at the start."
There's no single "correct" answer here, which is exactly why the model thrives. It's remixing language, not computing a fact.
Drop in a long article, transcript, or thread and ask for the gist.
> "Summarize this 2,000-word report into 5 bullet points a busy manager would care about."
It's very good at this. Just remember: it can occasionally drop or distort a detail, so verify any number or name that matters.
Blank-page problems are where LLMs feel like magic. Names, outlines, angles, first drafts.
> "Give me 10 subject lines for a webinar invite about AI for accountants. Mix curious, direct, and slightly playful."
You're not asking for *the* right answer. You're asking for many plausible options to react to. That's the model's home turf.
Here's the contrast that maps the whole landscape.
The failure (precision task):
> "How many r's in strawberry?" → "There are two r's." (Wrong.)
The win (language task):
> "Turn these rough notes into a polite 3-sentence client update: project delayed, new date March 14, will send a revised plan Friday."
> → "Hi Sarah, I wanted to flag that the project timeline has shifted, with our new target date now March 14. We're putting together a revised plan and will have it to you by Friday. Thanks for your patience, and please reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → out with any questions."
Same tool, same minute. One is a guess pretending to be a fact. The other is exactly what these systems were built for.
There's a simple test for whether a task plays to an LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition →'s strengths.
Plays to strengths: the task has many acceptable answers, and "good enough and fluent" is the goal. Rewriting, summarizing, explaining, brainstorming, translating, drafting.
Plays to weaknesses: the task has exactly one correct answer that depends on precise counting, calculation, or current facts. Math, character counts, live data, exact citations.
When you're in "one correct answer" territory, either give the model a tool or verify the output yourself.
Knowledge check
1. According to the lesson, what is the most accurate way to describe what an LLM fundamentally does?
2. Why does an LLM often miscount the number of 'r's in 'strawberry'?
3. The lesson mentions that modern LLMs can improve accuracy on math by 'giving it tools.' What does this primarily refer to?
4. Select ALL correct answers. Based on the lesson, which tasks are LLMs prone to fumbling without external tools?
Sélectionnez toutes les réponses correctes.
5. Select ALL correct answers. Which statements accurately reflect the lesson's framing of LLM capabilities?
Sélectionnez toutes les réponses correctes.
We'll go deep on this in the hallucinations lesson, but it belongs here too. A hallucinationhallucinationA hallucination is when an AI model generates output that is fluent and confident but factually wrong, fabricated, or unsupported by its source data.View full definition → is when an LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition → states something false with full confidence: a fake quote, a made-up statistic, a court case that never existed.
This isn't lying. The model is doing its job, predicting plausible-sounding text, and "plausible" is not the same as "true."
The riskiest tasks combine two traits:
1. There's a precise, checkable answer.
2. The model has no tool to check it.
That's why "What's the population of Lyon in 2026?" or "Cite three peer-reviewed studies on X" are danger zones. The model will happily invent a clean-looking answer.
Use the model for the *shape* of the work, and verify the *facts*.
> "Draft a paragraph about the benefits of remote work. Mark any specific statistic with [VERIFY] so I know to check it."
Most current models will flag where they're unsure when you ask them to. You can also enable web search (available in ChatGPT, Claude, and Gemini) so factual claims get grounded in real sources you can click.
For a deeper, free read on how these systems work and where they go wrong, the Elements of AI course is an excellent, jargon-light starting point.
You don't need to memorize a list. Just sort each task before you prompt.
Step 1: Name the task type. Is this a *language* job (rewrite, summarize, brainstorm) or a *precision* job (count, calculate, fact-find)?
Step 2: Match the approach.
Step 3: Verify what's checkable. Numbers, names, dates, and quotes get a quick second look. Tone and phrasing don't need it.
A combined prompt that uses each strength correctly:
Summarize the attached sales notes into 4 bullets for my boss.
For any total or percentage, calculate it using code and double
check it. Flag anything you're unsure about with [VERIFY].This hands the language work to the model and the math to a tool, in one go. That's the mindset: let the LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition → do language, and never trust it as a calculator or an encyclopedia by default.
[VERIFY] flags on anything checkable.