Our analysts spend the first hour of each day collecting the same kinds of information about the crypto markets they cover: news about specific projects, on-chain metrics, exchange announcements, regulatory updates, and how prices moved overnight. We built an internal assistant that prepares a first draft of that briefing. It researches and summarises; it does not recommend trades, size positions or touch any trading system. Analysts read, check and decide.

A single agent with all the tools worked for simple questions and fell apart for the full briefing: its context filled with raw search results, it forgot earlier findings, and costs per run varied tenfold. The design that worked uses a planner and specialised sub-agents. This post covers the engineering decisions, which apply to any research-style agent system.

The architecture

Analyst request
      |
  Planner agent  ---- writes plan.md, assigns tasks
      |
  +---+--------------+-------------------+
  |                  |                   |
News sub-agent   On-chain sub-agent   Regulatory sub-agent
  |                  |                   |
  writes findings/news.md  findings/onchain.md  findings/regulatory.md
      |
  Writer agent  ---- reads findings/*.md, writes briefing.md with sources
      |
  Checker agent ---- verifies claims against sources, flags issues

Each box is an LLM call loop with its own system prompt and its own small tool set. They communicate through files in a per-run working directory, not through one shared conversation.

Lesson 1: isolate sub-agent context

The single-agent version accumulated everything in one context: every search result, every page fetched, every intermediate thought. By the time it wrote the briefing, the important findings from the first ten minutes were buried under 150,000 tokens of raw material, and quality dropped.

Sub-agents fix this by design. Each one starts with a short, focused context: its task description, its tools, and the relevant part of the plan. It may read 50 pages, but it returns only a compact summary file. The planner and writer never see the raw pages.

Rules we follow:

  • A sub-agent's output is a file with a fixed structure, not free text in the conversation.
  • Sub-agents do not see each other's working context; they can read each other's output files if the plan says so.
  • The planner's context contains the plan and short status updates, nothing else.

Lesson 2: files as memory

Instead of keeping state in the conversation, agents read and write files in a working directory:

runs/2026-10-02T06-00Z/
  request.md
  plan.md
  findings/
    news.md
    onchain.md
    regulatory.md
  sources.jsonl          # every URL fetched, with timestamp and hash of content
  briefing.md
  check-report.md

Benefits:

  • The writer reads only the findings it needs, so its context stays small.
  • Runs are inspectable and reproducible: when an analyst questions a sentence, we can see exactly which source and which sub-agent produced it.
  • Retrying one failed sub-agent does not require rerunning the others.

The tools exposed to agents are simple: read_file, write_file, list_files, scoped to the run directory, plus domain tools (search, fetch page, query an on-chain data API).

Lesson 3: structure the findings

Free-form summaries were hard to combine and check. Each findings file follows a template:

## Finding
Claim: <one sentence>
Evidence: <quote or data point>
Source: <URL> (retrieved <timestamp>)
Confidence: high | medium | low

The checker agent verifies each claim against its source by refetching and comparing. Claims it cannot verify are marked in the final briefing. This step caught most hallucinated numbers in testing.

Lesson 4: cap everything

Agent loops can run away: a search returns nothing useful, the agent searches again with a slightly different query, forty times. We enforce limits in code, not in prompts:

Limit Value per run Enforced where
Tool calls per sub-agent 25 Agent loop
Total tokens per run 600k Token accounting middleware
Wall-clock time per sub-agent 6 minutes Task timeout
Fetched page size 40k characters, truncated Fetch tool
Total cost per run Fixed budget, alert at 80% Accounting middleware

When a sub-agent hits its limit, it must write whatever it has with a note that it stopped early. The writer reports gaps rather than inventing coverage.

Lesson 5: choose models per role

Not every role needs the strongest model. The planner and checker, which need careful reasoning, use a larger model. Sub-agents that mostly read and summarise use a smaller, faster model. That cut cost per run by about 60% with no measurable quality loss in our evals.

Lesson 6: evaluate like a product

We maintain an evaluation set of 40 past mornings with briefings written by analysts. For each change to prompts, tools or models, we run the system on the set and score:

  • Coverage: did the briefing include the items analysts considered important?
  • Accuracy: are claims supported by the cited sources (checked automatically, then sampled by humans)?
  • Faithfulness of numbers: every number in the briefing must appear in a source.
  • Cost and latency per run.

Changes ship only if accuracy does not drop. Coverage improvements are welcome; accuracy regressions are not.

Guard rails for a finance context

  • The system prompt and the tool set exclude any trading capability; agents have no credentials for exchanges or internal trading systems.
  • Output is a research briefing with sources, clearly labelled as machine-generated, reviewed by an analyst before it is shared.
  • The writer is instructed, and the checker verifies, that the briefing contains no buy or sell recommendations, price targets or position advice. A simple classifier flags such language as a backstop.
  • Fetched web content is treated as untrusted data: instructions inside pages are not followed, and tool access is limited so a malicious page cannot make agents do anything harmful.

Observability for agent runs

Every model call and tool call is traced with the run id, agent role, tokens and latency, using the OpenTelemetry generative AI conventions. When a briefing is slow or expensive, the trace shows which sub-agent spent the budget and on which tool. The two most common findings in the first month were a news sub-agent refetching the same pages under slightly different URLs, fixed by normalising URLs and caching fetches per run, and a checker that re-verified every claim twice because of a retry bug.

Rolling it out to analysts

We ran the assistant in shadow mode for three weeks: it produced briefings that analysts compared with their own, without using them. Their feedback became new evaluation cases. Only after accuracy on those cases was consistently high did the briefing become the starting point of their morning, and analysts still edit every one before it is shared.

Common mistakes

Mistake Effect Do this instead
One agent with every tool Context overload, inconsistent quality Planner plus focused sub-agents
Passing raw pages between agents Large, noisy contexts Structured findings files
Limits only in prompts Runaway loops and costs Enforce limits in code
Same large model for every role High cost Match model size to the task
No evals Changes degrade quality silently Fixed evaluation set and scores

Key takeaways

  • Split research work across a planner, focused sub-agents, a writer and a checker.
  • Keep each agent's context small; pass structured files, not conversations.
  • Use a working directory as shared memory for inspectability and retries.
  • Enforce tool, token, time and cost limits in code.
  • Evaluate coverage and accuracy on a fixed set before shipping changes, and keep research separate from decisions.

References