What an AI agent actually is, minus the hype

The word "agent" has been attached to so many products in the past two years that it has started to lose meaning. Vendors describe their newest chatbot upgrade as an "agentic system." Consultants pitch "agent orchestration" as if the term explains itself. Meanwhile, the professionals who need to make decisions about these tools, whether to buy, build, or integrate them, are left working with a concept that feels slippery. That slipperiness is worth fixing, because the underlying idea is genuinely useful once you separate it from the marketing.

Why it matters for managers and decision-makers

If you are responsible for a function, a team, or a P&L, the distinction between an AI agent and a regular AI tool affects what you can realistically automate, what you need to supervise, and what can go wrong without warning.

A standard LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition → integration, say a tool that drafts emails or summarises documents, takes input and returns output. You review it. You act. The model does nothing else until you ask again. That is a tool, not an agent.

An agent is different because it takes actions in the world, not just in a text box. It calls external systems, makes decisions about what to do next, and in many configurations, keeps going until a goal is met or it hits an error. That shift from "produces output" to "takes action" changes the risk profile entirely. A bad email draft is embarrassing. An agent with access to your CRMCRMCustomer Relationship Management: software and strategy to manage and analyse customer interactions throughout their lifecycle.View full definition →, your email system, and your calendar that misinterprets an instruction can create compounding errors across all three before anyone notices.

Understanding this is not academic. In 2025, several organisations that deployed early agentic systems from vendors including Salesforce (Agentforce) and ServiceNow reported that the harder problem was not building the agent but scoping what it was allowed to do and monitoring what it actually did. Those lessons are now informing how the category is being designed and sold in 2026, with guardrails and approval checkpoints becoming table-stakes features rather than optional add-ons.

How it actually works: the mechanics in plain language

An AI agent, at its core, is a loop. That is the simplest accurate description.

The loop works like this. The agent receives a goal or task. It reasons about what action to take next. It takes that action, which typically means calling a tool: a web search, an APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition →, a database query, a code executor. It observes the result. Then it reasons again about what to do next, based on what it just learned. This continues until the agent decides the goal is met, or until a stopping condition is triggered.

The reasoning step is usually handled by a large language modellarge language modelA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition →. The tools are external functions the model has been given access to, described in its system instructions. The memory is either held in the context windowcontext windowThe context window is the maximum amount of text (measured in tokens) a language model can process at once, including both the input prompt and the generated output.View full definition → (short-term) or written to an external store the agent can retrieve from (long-term).

A concrete example makes this tangible. Imagine you ask an agent: "Find the three highest-value enterprise accounts that haven't had any outreach in 90 days, and draft a personalised email for each." A standard LLM cannot do this. It does not have access to your CRM data, and it cannot take the additional step of drafting and preparing the emails based on what it finds. An agent can, if it has been given the right tools. It queries your CRM, identifies the accounts, pulls contact history, retrieves relevant context, and generates the drafts. Each step depends on the previous one. The model is not just completing a prompt; it is executing a plan it is constructing as it goes.

The most widely used architectural pattern for this right now is called ReAct (Reasoning and Acting), published by Google researchers in 2022. Most production agentic systems, including those built on frameworks like LangGraph, AutoGen, and Anthropic's agent tooling, use some variation of this reasoning-then-action loop, though with different implementations around memory, parallelism, and error handling.

When to use it and when not to: the honest tradeoffs

Agents earn their cost when three conditions hold: the task requires multiple steps that each depend on prior results, at least some of those steps require calling external systems, and the task recurs often enough that automation pays off relative to the overhead of building and maintaining the agent.

Competitive intelligence gathering fits this description. So does monitoring regulatory filings, triaging and routing incoming support cases, and reconciling data across systems that do not share a native integration. These are tasks where the value comes from chaining steps together reliably, not just from the quality of a single output.

Agents are a poor fit when the task is better handled by a single well-constructed prompt, when the stakes of an error are high and recovery is difficult, or when the goal is ambiguous enough that a human needs to stay in the reasoning loop throughout. The failure mode of agents in production is not dramatic collapse; it is quiet drift, where the agent completes a plausible-looking sequence of actions that turns out to be wrong, and no one notices until downstream.

There is also a cost question that vendor materials tend to understate. Each step in an agentic loop calls the LLM again, sometimes multiple times per step. A task that requires fifteen reasoning cycles costs roughly fifteen times more in inference than a single prompt. For high-frequency workflows, that adds up fast. Benchmarking the actual tokentokenA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition → consumption of an agent against the time it saves is work that most teams skip and later regret.

The honest summary is that agents are a specific tool for a specific class of problem. They are not a general upgrade to how AI works. When the conditions are right, they genuinely extend what automation can do. When they are not, they add complexity without proportional return.

Build the minimum scoping document before you build or buy an agent: what exactly is the goal, what tools does it need, what happens when it gets a result it did not expect, and who reviews its output before it has consequences. That document will tell you faster than any demo whether you actually need an agent.