Your agent traces are a corpus

Work in progress. The date and the scope are set, the material isn’t finished. Expect details to move between now and the 25th.

Register here. It’s a lightning lesson in RAG: Retrieval Augmented Gathering, the free series Doug Turnbull runs on Maven, at 1pm EDT on Friday September 25th. Free to attend, recorded afterward, and it may contain a promotional CTA for hevlayer.com.

What you’ll learn

How to capture agent traces without a vendor in the path. A daemon that watches local agent sessions and writes them to object storage you control.

Where OpenTelemetry fits, and where it stops. Emitting spans through OTel is what keeps the capture side neutral, because any collector and any backend a team already runs can read it. The GenAI semantic conventions give agent spans names that mean the same thing across tools, which is what makes two agents comparable at all. I’ll also show where the model runs out: a span is built for a request that finished in 200ms, and an agent session is hours of prose you want to keep and search for years.

A storage layout that survives the tool that wrote it. The archive keeps the raw capture alongside versioned Parquet, and that split is the contract every downstream reader depends on.

What it costs to make them searchable. I’ll price the index per gigabyte per month across a couple of choices, with write amplification included.

Live CLI and UI. I’ll query a real trace archive, including the part where the interesting question turns out to be a retrieval question.

Why this topic matters

A coding agent produces an enormous amount of text about your codebase. It records every file it read, every command it ran, and every approach it abandoned before the one that worked. Almost all of that is discarded when the session closes.

Most tools that do keep it treat the stream as telemetry, which gets you run counts, token spend, and latency percentiles. That’s enough to bill against and too thin to engineer against. The archive is more useful as a corpus, because then you can ask it what the agent read before it broke the build, which of three tools did better work on the same task, or why it chose a library the repo already had.

OpenTelemetry is what makes that archive worth building rather than a private format nobody else can read. The agent tools are converging on it, the GenAI semantic conventions are settling, and a team instrumenting agents today already has a collector running for everything else. So the capture side should speak OTel and stay out of the way. The part OTel does not answer is what happens after the collector: where an hour of agent prose lands, in what layout, and how you get a specific moment back out of six months of it. That question is left to the backend, and the backends were designed for metrics and short request traces.

Those are retrieval questions, and they arrive with the shape that makes retrieval hard: the archive is large, cold, and mostly prose, while the part you want back is small and hot. That makes it a storage decision before it is a model decision, and the bill for getting it wrong arrives once at ingest and then every month after. I run mine on turbopuffer. The layout underneath doesn’t care, and I’ll show what changes when you swap it.

The capture side goes open source before the talk.