What an AI agent actually is, stripped of the hype

The word "agent" has been applied to so many products in the past two years that it has nearly lost meaning. Marketing teams at Salesforce, Microsoft, and dozens of startups have attached it to anything that does more than answer a single question. The result is genuine confusion among the professionals who have to decide whether to build with these systems, buy them, or ignore them. Before making any of those decisions, it helps to have a working definition that is precise enough to be useful.

An AI agent, in plain terms, is a software system that perceives some state of the world, decides on an action, executes it, observes the result, and then repeats. The loop is what separates an agent from a standard LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition → call. When you type a question into ChatGPT and it answers, no loop happens. The model receives input and produces output, once. An agent, by contrast, keeps going until it reaches a goal or runs out of steps. That distinction sounds simple. The implications are not.

Why it matters for professionals who make decisions about software

The practical stakes of understanding this correctly are real. If your team buys a product described as "agentic" and it is actually a glorified chatbot with a few pre-written prompts, you will be disappointed and probably burned. Conversely, deploying a genuine agent on a task that requires precision and auditability, say, a regulated financial workflow, without understanding that the system makes intermediate decisions you cannot always predict, creates compliance and liability exposure.

Microsoft's Copilot Studio, Salesforce's Agentforce, and ServiceNow's Now Assist all describe their offerings using agent language. Each involves a different architecture and a different degree of autonomy. Without a clear mental model of what an agent actually is, comparing them is essentially guesswork.

Beyond procurement, there is a management question. Agents introduce a different failure mode than static software. A conventional application does what it is programmed to do, and bugs are reproducible. An agent operating in a multi-step loop can produce errors that compound, drift from its original objective, or succeed at the letter of its instructions while violating the spirit of them. Managing that requires different oversight, not more trust in the technology.

How it actually works: the mechanics

The internal logic of an agent typically follows a pattern called the ReAct loop (Reasoning and Acting), formalized in a 2022 paper from researchers at Princeton and Google. The system is given a goal, a set of tools it can call, and a context windowcontext windowThe context window is the maximum amount of text (measured in tokens) a language model can process at once, including both the input prompt and the generated output.View full definition → that accumulates results. At each step it reasons about what to do next, selects a tool, runs it, reads the output, and decides whether the goal is met.

A concrete example: suppose you ask an agent to find the cheapest direct flight from London to Tokyo on a specific date, book it under a given name, and send a confirmation email. The agent does not execute this in one pass. It calls a flight-search APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition →, reads the results, identifies the cheapest option, calls a booking API with the relevant parameters, checks that the booking succeeded, and then calls an email API. Each step is a discrete action. If the booking API returns an error, the agent should detect that and either retry or escalate.

The tools available to an agent are whatever the developer connects to it: web search, databases, code interpreters, calendar systems, internal APIs. The intelligence determining which tool to use and in what order comes from the underlying LLM, typically GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro as of mid-2026. The LLM is the brain; the tools are the hands. Neither works without the other in this architecture.

Memory is a separate component. Agents can have short-term memory (the context window of the current run), long-term memory (a database retrieved via search), and in some architectures, shared memory across multiple agents working in parallel. This is where multi-agent systems come in: rather than one agent doing everything sequentially, you split tasks across specialists that communicate results to each other. AutoGen, a framework released by Microsoft Research, and CrewAI, a popular open-source alternative, both implement this pattern.

Where the loop breaks

The failure points are predictable once you know the structure. LLMs are probabilistic, so reasoning steps are not guaranteed to be correct. Tool calls can return unexpected formats. The agent can get stuck in a loop if it does not detect that its actions are not progressing toward the goal. Context windows have limits, so very long tasks risk the agent losing track of earlier steps. None of this is hypothetical. Any team that has run production agents for more than a few weeks has hit at least two of these.

When to use it, and when not to

Agents earn their complexity when a task has multiple sequential steps, requires external data or action in the world, and has some tolerance for occasional errors that a human can catch and correct. Research summarization across dozens of documents, draft-to-send email workflows in lower-stakes contexts, code generation with automated testing, IT ticket triage: these are areas where teams are seeing genuine productivity gains in 2026.

The pattern holds up less well in four situations: when the task requires perfect accuracy on every step (agents hallucinate and tools fail), when there is no feedback mechanism to catch errors before they matter, when the chain of decisions needs to be fully auditable for legal or regulatory reasons, and when the task is simple enough that a single well-crafted LLM prompt would do the job in half a second.

A common mistake is to reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → for an agent because it sounds more sophisticated. A single prompt with good instructions and access to one retrieval tool will outperform a poorly configured five-step agent on almost any bounded question-answering task. Complexity should follow from the problem, not from enthusiasm for the technology.

The definition that matters operationally is this: an agent is a loop that reasons, acts, and observes, powered by an LLM and connected to external tools. Once you see it that way, it becomes possible to evaluate specific products honestly, design appropriate guardrails, and set realistic expectations with stakeholders. That is the work, and it starts with knowing what you are actually dealing with.