Building Effective AI Agents: Lessons from Anthropic’s Barry Zhang
Barry Zhang’s advice is refreshingly practical: do not build an agent when a predictable workflow will do, keep the architecture simple, and design from the agent’s limited point of view. Here is a clear breakdown of the talk, plus what still matters for teams building agentic systems in 2026.
Three ideas worth keeping
- Use agents selectively. They are best suited to valuable, ambiguous tasks where the system must decide what to do next.
- Start with the smallest workable architecture. A model, a clear environment, well-described tools, and a strong system prompt are often enough to begin.
- Think from inside the agent’s context. The agent only knows what you expose through prompts, tools, state, screenshots, and feedback.
- Build evaluation and budget controls early. Autonomy without observability can quickly become expensive, slow, and unreliable.
Barry Zhang: How We Build Effective Agents
Recorded at the Agent Engineering Session Day during AI Engineer Summit 2025 in New York.
Why agentic systems matter
Agentic systems have moved AI beyond single prompt-and-response interactions. Instead of producing one answer, an agent can examine a goal, choose tools, take actions, inspect the results, and adjust its approach across multiple steps.
That flexibility is useful, but it comes with a trade-off. The more freedom a system has, the less predictable its path becomes. Teams must decide where autonomy creates meaningful value and where a conventional workflow would be safer, faster, and easier to maintain.
At the February 2025 AI Engineer Summit in New York, Anthropic Member of Technical Staff Barry Zhang presented How We Build Effective Agents. The talk expanded on Anthropic’s earlier engineering guide, which Zhang co-authored with Erik Schluntz, and focused on three practical principles: do not build agents for everything, keep them simple, and think like the agent.
Predetermined paths
The application follows steps defined in code. A model may be used inside the workflow, but the sequence and branching logic remain largely controlled by the developer.
Dynamic decisions
The model decides how to proceed, which tools to use, and when to revise its approach. This flexibility helps with open-ended work, but increases the need for evaluation and guardrails.
Do not build agents for everything
An agent is not automatically better because it is more autonomous. The right question is whether the task genuinely benefits from flexible reasoning and tool use.
A fixed workflow is usually easier to test, cheaper to operate, and more predictable. An agent earns its complexity when the task cannot be fully mapped in advance and when the value of solving it justifies the extra cost and risk.
Choose the simplest system that can reliably solve the problem.
Practical interpretation of Barry Zhang’s guidanceA practical decision framework
- Task ambiguity: Does the system need to interpret changing context and decide its next action, or can the steps be written down in advance?
- Business value: Is the task valuable enough to justify longer development cycles, model usage, evaluation, and monitoring?
- Model and tool capability: Can the model complete the critical actions with acceptable accuracy, latency, and cost?
- Verifiability: Can the output be checked through tests, rules, human review, or another dependable feedback mechanism?
- Cost of failure: What happens when the agent chooses the wrong action, stops too early, loops, or misunderstands the goal?
Coding and research tasks
- The work can involve many possible paths.
- Outputs can often be checked with tests, source review, or structured evaluation.
- The value of completing a difficult task can outweigh the added model cost.
Routine, high-volume processes
- The steps are stable and predictable.
- Low latency and low cost matter more than flexible reasoning.
- Errors are easier to prevent with rules, forms, routing, and human escalation.
Keep the architecture simple
Teams often add frameworks, planners, memory systems, orchestration layers, and multiple agents before proving that the basic loop can solve the task. Barry’s recommendation was to begin with the core pieces and add complexity only when evidence demands it.
Where the agent operates
This might be a browser, code repository, CRM, simulated workspace, document collection, or another controlled interface. The environment defines what the agent can observe.
How the agent acts
Tools expose useful actions through APIs or interfaces. Clear names, descriptions, parameters, permissions, and return values make it easier for the model to use them correctly.
What success looks like
The system prompt defines the goal, operating rules, boundaries, and expected behaviour. It should explain what the agent is trying to accomplish without hiding essential context.
Anthropic’s broader guidance makes a similar point: simple, composable patterns are often more successful than highly abstracted frameworks. A direct model-and-tool loop is easier to inspect, debug, and improve because the developer can see exactly what the system received and why it acted.
What to add only after the basic loop works
- Persistent memory for information that must survive beyond the current context window.
- Planner or orchestrator components when one model loop cannot manage the task cleanly.
- Parallel workers when the work can be safely divided and recombined.
- Evaluator-optimizer loops when quality can be measured and repeated refinement is worth the added cost.
- Human approval points for consequential or irreversible actions.
Think like your agent
Developers see the whole product. The agent does not. It sees only the information placed inside its context at that moment.
This difference creates many failures that appear mysterious from the outside. A person may understand the meaning of a screen, a vague instruction, or an incomplete tool description because they bring years of background knowledge. The model may be deciding from a static screenshot, partial history, and a few lines of text.
The agent may be acting almost blind
A computer-use agent might receive a screenshot rather than a continuous understanding of the interface. It may not know the screen resolution, whether a page changed between actions, which controls are safe to use, or what the user considers an acceptable result unless those details are explicitly provided.
How to test from the agent’s point of view
- Review the exact prompt, tool definitions, state, and observations available at the moment a decision is made.
- Remove any assumption that exists in the developer’s head but not in the agent’s context.
- Inspect whether tool names and descriptions distinguish similar actions clearly.
- Test interrupted, incomplete, and contradictory states instead of only the ideal path.
- Ask whether the agent has a reliable signal for completion, failure, and when to request human help.
Barry’s predictions and the 2026 update
Barry closed the 2025 talk with several ideas about where agent engineering was heading. Rather than treating them as guaranteed outcomes, they are useful design pressures that teams should plan for.
Autonomy needs limits
Agent runs can vary in length and cost. Production systems need explicit limits for time, tokens, tool calls, retries, and money, plus a clear fallback when the budget is reached.
Interfaces will evolve
Agents may help identify weak tool ergonomics, propose improved interfaces, or assemble reusable procedural capabilities. Human review remains important before new tools receive meaningful permissions.
Coordination adds overhead
Specialized agents can divide complex work, but communication, duplicated context, conflicting decisions, and evaluation can make a multi-agent design harder to operate than a strong single agent.
The core message has aged well
Anthropic’s more recent engineering material continues to emphasize rigorous evaluations, clear interfaces, context management, and the need to question assumptions built into an agent harness. The practical lesson is not to chase autonomy for its own sake. Build the minimum reliable system, measure it, and expand only when the evidence supports the next layer of complexity.
| Design area | Weak approach | Stronger approach |
|---|---|---|
| Architecture | Start with multiple agents and a large framework. | Prove a simple model-and-tool loop before adding orchestration. |
| Tools | Expose vague functions with overlapping responsibilities. | Use narrow, well-described tools with predictable responses and permissions. |
| Evaluation | Judge quality from a few impressive demonstrations. | Build repeatable task sets, failure categories, and production monitoring. |
| Budgets | Allow open-ended retries and tool calls. | Set caps for time, cost, steps, and escalation. |
| Human control | Automate irreversible actions immediately. | Add approval gates until reliability and accountability are established. |
A simple implementation checklist
Before adding another agent, framework, or memory layer, use this checklist to determine whether the foundation is ready.
- The task has a clear business outcome and a defined reason for needing flexible decision-making.
- The agent can observe the information it needs without relying on hidden human assumptions.
- Each tool has a distinct purpose, clear parameters, controlled permissions, and useful error responses.
- The system prompt defines the goal, boundaries, completion conditions, and escalation path.
- Representative evaluations cover successful runs, common errors, ambiguous inputs, and edge cases.
- Budgets limit time, tokens, retries, tool calls, and overall cost.
- Logs make it possible to reconstruct what the agent saw, decided, and did.
- Human approval is required before high-impact, sensitive, or irreversible actions.
AI agent FAQs
01 What is an agentic system in AI?
An agentic system uses a language model to direct at least part of its own process, such as deciding which tool to use, what step to take next, or how to respond to intermediate results. Some agentic systems are tightly constrained, while others operate with more autonomy across longer tasks.
02 When should a team use an agent instead of automation?
Use an agent when the task is valuable, difficult to map as a fixed sequence, and benefits from interpreting context or choosing among multiple actions. Use conventional automation when the process is stable, predictable, high-volume, and easier to express with rules.
03 What are the core components of an AI agent?
A practical starting point includes a capable model, an environment the agent can observe, tools it can use to act, a system prompt that defines its goal and boundaries, and a feedback loop that shows the result of each action.
04 Why is tool design so important for agents?
Tools are the agent’s interface to the outside world. Vague names, overlapping functions, unclear parameters, or inconsistent responses make correct tool selection harder. Clear tool design improves reliability and makes failures easier to diagnose.
05 Are multi-agent systems better than a single agent?
Not automatically. Multiple agents can help when work divides naturally into specialized tasks, but they also introduce coordination costs, duplicated context, more model calls, and additional failure modes. A strong single agent or structured workflow is often the better starting point.
06 How should AI agents be evaluated?
Evaluate the full trajectory, not only the final answer. Track whether the agent selected appropriate tools, followed constraints, recovered from errors, used a reasonable number of steps, stayed within budget, and produced an outcome that can be verified.