An agent that acts does not fail like an assistant that answers
AI agents carry high expectations: entrusting a system no longer with a question, but with a goal. Handling a request end to end, reconciling invoices with purchase orders, preparing a file by querying several applications. The promise is real. The step to climb is higher than it looks.
An assistant that makes a mistake produces incorrect text, which a person can reread before using it. An agent that makes a mistake executes: it sends a message, modifies a record, triggers an order. The error has taken effect in the information system before anyone has seen it.
Three properties make the subject difficult. Errors accumulate: a ten-step task, each step of which usually succeeds, fails far more often than a one-step task. Behaviour is not deterministic: the same goal can be handled by two different paths. Finally, the agent reads content you do not control (emails, web pages, attachments), and that content may contain instructions intended to hijack it.
Loop, tools, memory: what makes an agent, and how far to let it decide
An AI agent is a language model placed in a loop. It receives a goal, decides on an action, calls a tool (a programming interface, a database query, a search), observes the result, then decides on the next action. The loop stops when the goal is achieved or a stopping condition is triggered. Tools define what the agent can do. Memory defines what it retains from one step or session to the next.
So-called “agentic” systems do not all have the same level of autonomy. This choice is the first design decision.
| Form | Who decides the sequence | Typical use | Level of control |
|---|---|---|---|
| Orchestrated workflow | The code: the steps are fixed, the model handles each one | Stable, well-described process | High, predictable behaviour |
| Tool-using agent | The model chooses the tools and their order, within a bounded scope | Variable tasks, cases impossible to list in advance | Depends on the guardrails |
| Multi-agent system | An orchestrator distributes the work among specialised agents | Long tasks combining several skills | Harder to test and trace |
Our rule is to choose the lowest level of autonomy that delivers the expected service. Many needs presented as agent projects are better handled by an orchestrated workflow, which is simpler to test. And when the need is to answer from documents, without acting, a generative AI assistant is enough.
Designing AI agents starting from their permissions
Our consultants work within your teams; decisions remain yours.
Choosing a suitable process. A good candidate is a frequent task whose result can be checked and whose errors can be corrected. A process that nobody can describe will not be carried out any better by an agent.
Defining tools as contracts. Each tool exposed to the agent has a precise description, typed parameters and explicit error messages. We separate read tools from write tools. The agent has its own technical identity and rights limited to its task: it does not borrow a user’s account. This assumes that your applications expose clean interfaces, which brings us back to information system architecture. Open protocols such as the Model Context Protocol (MCP) standardise this connection between agents and tools.
Building guardrails into the system, not into the instruction. Asking a model to “never” do something is not a control. The limits are technical: a closed list of authorised actions, amount and volume caps, a maximum number of steps, a budget per task, code execution in an isolated environment, filtering of incoming content.
Grading human oversight. Three regimes coexist. Prior approval: the agent prepares, a person approves before any irreversible action. After-the-fact checking: the agent acts, a person checks a sample. Escalation: the agent hands back control as soon as it leaves its scope or a doubt rule is triggered. The regime is chosen action by action, according to the severity of an error.
Tracing and testing trajectories. Every decision, every tool call and every result is logged, so that what the agent did, and in what order, can be explained afterwards. Tests focus on the final state obtained in a simulated environment, and not only on the text produced. They include hijacking attempts by instruction injection.
When the agent constitutes a high-risk AI system within the meaning of Article 6 of Regulation (EU) 2024/1689, human oversight is a requirement of the text (Article 14). The oversight arrangements then form part of your AI governance.
What derails an agent project
- Starting with several agents. A multi-agent system multiplies the exchanges to trace and the causes of failure. It is justified after a single agent has been stabilised, rarely before.
- Granting broad rights “so that it works”. An agent that can do everything can damage everything. Each additional right is justified by a specific step of the task.
- Removing human approval too early. Autonomy is extended on the basis of measurements, for example the share of the agent’s proposals accepted without modification, and not on an impression.
- Going live without a shutdown procedure. An agent must be able to be suspended immediately, and its recent actions must be retrievable.
- Forgetting those whose work changes. Supervising an agent is not the same job as carrying out the task. Validation criteria, the checking workload and responsibilities are defined with the teams concerned.
Success indicators for an agent in operation
An agent is managed like a service, with measurements taken continuously:
- the share of tasks completed without human intervention, and whose result is correct;
- the frequency of escalations and their reasons, to distinguish useful caution from blockage;
- the share of actions corrected or cancelled by a supervisor;
- the number of steps and the cost per task, which signal loops and drift;
- attempts at out-of-scope actions blocked by the guardrails;
- end-to-end processing time, compared with that of the previous process.
Reading these measurements together guides decisions. A high completion rate accompanied by many after-the-fact corrections indicates an agent too autonomous for its actual reliability. Very frequent escalations indicate a poorly defined scope or inadequate tools.