The word agent is often used for almost any application that combines an LLM with tools. I used to think that the more steps and tools a system had, the more "agentic" it was. What matters more, however, is who decides what happens next: predefined code or the model itself.
I wrote this note to clarify that distinction and to give myself a practical way to choose an architecture. The goal is not to build the most autonomous system possible, but to use the simplest pattern that can complete the task reliably.
What are workflows and agents?
- Workflows are systems where LLMs and tools are orchestrated through predefined code paths.
- Agents are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.
When building applications with LLMs, find the simplest solution possible, and only increase complexity when needed. Agentic systems often trade latency and cost for better task performance.
Common patterns for agentic systems from simple to complex
Basic Building Block: The augmented LLM
The basic building block of agentic systems is an LLM enhanced with augmentations such as retrieval, tools, and memory.
Workflow: Prompt chaining
Prompt chaining decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one. Add programmatic checks (gates) on any intermediate steps to ensure that the process is still on track.
Use examples:
- Generating marketing copy, then translating it into a different language.
- Writing a document outline, checking that outline, then writing the document based on the outline.
Workflow: Routing
Routing classifies an input and directs it to a specialized follow-up task. This workflow allows for separation of concerns and building more specialized prompts.
Routing works well for complex tasks where there are distinct categories that are better handled separately, and where classification can be handled accurately, either by an LLM or a more traditional classification model/algorithm.
Use examples:
- Directing different types of customer service queries (general questions, refund requests, technical support) into different downstream processes, prompts, and tools.
- Routing easy/common questions to smaller, cost-efficient models and hard/unusual questions to more capable models for best performance.
Workflow: Parallelization
Parallelization is effective when the divided subtasks can be parallelized for speed, or when multiple perspectives or attempts are needed for higher-confidence results.
Use examples:
- Sectioning: Breaking a task into independent subtasks run in parallel.
- Implementing guardrails where one model instance processes user queries while another screens them for inappropriate content or requests. This tends to perform better than having the same LLM call handle both guardrails and the core response.
- Automating evals for evaluating LLM performance, where each LLM call evaluates a different aspect of the model's performance on a given prompt.
- Voting: Running the same task multiple times to get diverse outputs.
- Reviewing a piece of code for vulnerabilities, where several different prompts review and flag the code if they find a problem.
- Evaluating whether a given piece of content is inappropriate, with multiple prompts evaluating different aspects or requiring different vote thresholds to balance false positives and negatives.
Workflow: Orchestrator-workers
A central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results.
This workflow is well-suited for complex tasks where you can't predict the subtasks needed. The key difference from parallelization is its flexibility -- subtasks aren't predefined, but determined by the orchestrator based on the specific input.
Use examples:
- Coding products/agents that make complex changes to multiple files each time.
- Search tasks that involve gathering and analyzing information from multiple sources for possibly relevant information.
Workflow: Evaluator-optimizer
One LLM call generates a response while another provides evaluation and feedback in a loop.
This workflow is particularly effective when we have clear evaluation criteria and when iterative refinement provides measurable value.
Use examples:
- Literary translation where there are nuances that the translator LLM might not capture initially, but where an evaluator LLM can provide useful critiques.
- A complex search task that requires multiple rounds of searching and analysis to gather comprehensive information, where the evaluator decides whether further searches are warranted.
Agent
Agents begin their work with either a command from or an interactive discussion with the human user. Once the task is clear, agents plan and operate independently, potentially returning to the human for further information or judgement. During execution, it's crucial for the agents to gain "ground truth" from the environment at each step (such as tool call results or code execution) to assess their progress. Agents can then pause for human feedback at checkpoints or when encountering blockers. The task often terminates upon completion, but it's also common to include stopping conditions (such as a maximum number of iterations) to maintain control.
Agents can be used for open-ended problems where it's difficult or impossible to predict the required number of steps, and where you can't hardcode a fixed path. The autonomous nature of agents means higher costs and the potential for compounding errors. This autonomy requires extensive testing in sandboxed environments, along with the appropriate guardrails.
Use examples:
- Coding agents to resolve SWE-bench tasks that involve edits to many files based on a task description
- Agents use a computer to accomplish tasks.
How I choose a pattern
These patterns are not a required progression from simple to advanced, and they can be combined. I choose between them by asking how predictable the task is and how clearly I can evaluate its result:
- A single augmented LLM is enough when one call, supported by retrieval, tools, or memory, can reliably complete the task.
- Prompt chaining or routing fits a task with predefined stages or clearly distinguishable input categories.
- Parallelization fits fixed, independent subtasks or cases where multiple perspectives can improve confidence.
- Orchestrator-workers fit work whose subtasks cannot be known in advance, such as a code change that may involve an unpredictable set of files.
- Evaluator-optimizer fits work with explicit quality criteria and results that measurably improve through feedback.
- An agent is justified when the number of steps is unknown, and progress depends on repeatedly acting, observing environmental feedback, and deciding what to do next.
The more autonomy I add, the more important evaluation, stopping conditions, sandboxing, and human checkpoints become. If I cannot describe what success looks like, giving the system more autonomy will not solve that problem.
Best practices and principles
Success isn't about building a sophisticated system but building the right system for needs. Start with simple prompts, optimize them with comprehensive evaluation, and add multi-step agentic systems only when simpler solutions fall short.
Follow three core principles when implementing agents:
- Maintain simplicity
- Prioritize transparency by explicitly showing the agent's planning steps.
- Carefully craft the agent-computer interface (ACI) through thorough tool documentation and testing.
My takeaway
The main lesson is not that agents are always better, but that agentic complexity should be earned. I should start with a simple LLM call, add retrieval, tools, or memory only when they address a demonstrated limitation, and use workflows when the sequence can be defined in advance. Autonomous agents should be reserved for tasks that genuinely require flexible planning, tool use, and feedback from the environment.
For example, producing an article in a fixed format does not automatically require an agent. Prompt chaining may be enough to create an outline, check it against known criteria, and draft the article. By contrast, changing an unfamiliar codebase may require an agent because it must inspect files, decide what to edit, run tests, interpret failures, and revise its approach. The difference is not the apparent sophistication of the task; it is whether the path can be predetermined.
This also changes how I judge an agent. A convincing response is not sufficient evidence that it worked. The system needs ground truth from tool results, tests, or other environmental feedback, plus a clear completion condition. Reliability comes from the loop around the model as much as from the model itself.
Source
Based on: Building effective agents