Share:
Start with a single agent embedded as a product feature: one reasoning model, a small explicit tool set, permission boundaries around anything irreversible, human review gates, and logging from day one. Add memory systems, orchestration, and multi-agent patterns only when customer growth forces them — premature agent platforms are the most common failure mode. This guide comes from running AI coding teams and OpenClaw agents in production, including the costs and failures.
The Architecture Ladder
Most agent architecture advice starts at the top of the ladder. Start at the bottom.
Level 1 — The feature agent
One model, one job, a handful of tools, embedded in your product like any other feature. It has a system prompt you version-control, tools you enumerate explicitly, and an owner on your team. The overwhelming majority of "we need agents" conversations I have as a fractional CTO end here, and should.
Stay here until: a single agent's context or tool list has grown past what one person can reason about, or you have distinct workflows fighting over one prompt.
Level 2 — Orchestrated workflows
Deterministic code decides when agents run; agents decide how to do their step. Fan-out for coverage, pipelines for stages, and code — not a model — owning control flow. This is where my own development practice sits: autonomous coding agents planning and shipping alongside humans, with the orchestration boring on purpose.
Stay here until: you have evidence (not vibes) that agent-to-agent negotiation outperforms code-owned control flow for your workload.
Level 3 — Multi-agent systems
Agents delegating to agents. Genuinely useful for a narrow set of problems — broad research sweeps, adversarial verification — and the most common way startups set money on fire. As I argued after Ai4, benchmarks don't ship; demo-driven architecture at this level is how you end up with an impressive system nobody can debug.
The Five Decisions That Matter
1. Model choice
Pick per-task, not per-company. Frontier model for judgment-heavy steps, small fast models for mechanical ones. Revisit quarterly — the ground moves.
2. Tool and permission design
Enumerate tools explicitly; never hand an agent a general-purpose shell where a scoped API will do. Anything irreversible — sending, deleting, paying — sits behind a permission gate. In my own systems this mirrors the merge policy I use for code: reversible internal work is agent-autonomous, anything customer-visible gets human review, and money, credentials, and data deletion stay human-only.
3. Context and memory strategy
Start with no memory beyond the task. Add retrieval when users repeat themselves, and durable memory only when the product's value depends on it. Memory is a data liability as much as a capability.
4. Human-in-the-loop gates
Gates scale with blast radius: agent-autonomous for reversible internal work, human-review for anything customer-visible, human-only for money, auth, and data deletion. (The merge-policy version of this for code is its own post: Should AI agents write production code?)
5. Observability and evals
Log every tool call and token from day one. You cannot improve an agent you can't replay. Before scaling a workflow, write evals for it — even ten hand-checked cases beat none.
What It Actually Costs
Agent teams scale cost linearly with the work: more parallel agents, more tokens, and (unlike humans) zero marginal management overhead — the token bill IS the headcount conversation. That linearity cuts both ways. It means you can price a workflow precisely: log tokens per task from day one, and within a week you know what each agent-run costs and whether it beats the human alternative. It also means a workflow that's mispriced loses money on every single run, silently, at whatever scale you operate.
Three cost rules that hold up in practice:
- Price the workflow, not the platform. "Our agents cost $N/month" hides which workflows earn their tokens and which burn them.
- Route by task value. Frontier models for judgment-heavy steps; small, fast models for the mechanical majority. Most cost bloat is a frontier model doing intern work.
- Watch retries. The expensive failure mode isn't one big run — it's an agent quietly re-attempting a failing task in a loop. Retry budgets belong in the orchestration layer, not in hope.
Where Agent Systems Break
The failure classes I watch for, having operated these systems daily — each maps to a guardrail from the five decisions above:
- Confident wrong ambiguity resolution. Given an underspecified task, an agent doesn't ask — it picks an interpretation and executes it well. The guardrail is spec review before execution, not output review after.
- The runaway fix loop. An agent "fixing" its own failing change digs deeper with each attempt — what should have been one failed task becomes an afternoon of token spend and a tangled diff. Retry limits and a kill switch, always.
- Stale-context actions. An agent acting on cached state — republishing an old draft, re-running a completed migration — because nothing forced a fresh read before an irreversible step. Gate irreversible actions on current state, not remembered state.
- Tool-boundary creep. The permission set that was tight at launch accumulates "just this once" additions until the agent effectively has a shell. Audit the tool list on a schedule, like dependencies.
Anti-Patterns
- Overbuilding for the demo. Multi-agent orchestration to impress investors, single-agent economics underneath.
- Unbounded tools. A shell and a browser is not a tool design; it's a liability with a chat interface.
- No kill switch. If you can't stop every agent in one action, you don't operate the system — it operates you.
- Memory as a default. Storing everything an agent sees, then discovering you've built a compliance problem.
- Skipping evals because "we review everything". Review doesn't scale; evals do.
Where Identity Fits
Once agents transact with the outside world, "which agent, operated by whom, allowed to do what" becomes an identity problem — verifiable credentials and delegation, not API keys. I wrote about where the identity industry is heading on exactly this in my notes from Identity Week America 2026, and a dedicated deep-dive on agent identity verification is coming.
FAQ
Single agent or multi-agent to start?
Single, embedded as a product feature. Multi-agent only when you have evidence code-owned orchestration is the bottleneck.
How do I keep agent costs under control?
Log tokens per task from day one, price each workflow, and use small models for mechanical steps. Cost problems are almost always architecture problems.
What should never be delegated to an agent?
Irreversible actions without a human gate: payments, credential changes, data deletion, and anything legal or safety-relevant.
How do I evaluate agent output quality?
Hand-built evals per workflow — even ten cases — plus production sampling. Never rely on the agent grading itself.
Which framework should a startup pick?
The one your team can debug. Framework choice matters far less than tool boundaries, logging, and evals — the parts you own regardless.
Lance Ennen is a technology founder and fractional CTO specializing in AI-agent systems, digital identity, and payments infrastructure. He runs AI coding teams in his own companies and advises startups on agent architecture. Related: Rethinking Agile for AI Engineering Teams, The Rise of the Autonomous Company, Autonomous AI Software Teams.
Written by
Lance Ennen
CTO & Technical Advisor helping startups and Fortune 100 companies build innovative digital products. Passionate about blockchain, AI, and scalable architecture.
Enjoyed this article? Share it with others