SuperWitness: every run, on the record
When an agent finishes a piece of work, three different things are true at once. The agent did a series of things. The system it ran on behaved a certain way while it did them. And the result was either good or it wasn’t. Usually those three live in three places: a transcript, a tracing tool and somebody’s opinion in a chat thread.
SuperWitness puts them in one record. It is the last step in the suite: you see proof.
One document per run
Ask SuperWitness for a run and it builds a single document at the moment you ask. It brings together the SuperPipeline run, each AgentPod attempt (with a fingerprint of the configuration it ran with), the trace spans and the logs and errors.
It is careful about what it doesn’t know. If one of those sources is down, the document says so, and the numbers that depended on it read unknown, never zero. A missing measurement and a measurement of nothing are different facts.
Judgements you can’t quietly edit
A verdict is a judgement of a run against a versioned rubric. Verdicts are append-only, and the database itself refuses to update or delete them. Changing your mind means recording a new verdict that supersedes the old one, so the history stays.
Two rules make the record worth trusting. The kind of judge, a person or an agent, comes from who is signed in, not from what they claim. And an agent cannot judge a run it executed.
People record verdicts in the web app, which groups runs by day with views for needs verdict, failed and waiting, a span waterfall and the verdict history. Agents can do the same through an MCP server with six tools, from reading a run to recording a verdict.
It reads, it doesn’t collect
SuperWitness doesn’t take in telemetry of its own. Products send their traces and logs to the OpenTelemetry Collector as usual, and SuperWitness queries VictoriaTraces and VictoriaLogs when it needs them. Nothing waits on SuperWitness to do its work.
Where it is today
SuperWitness is early, and honest about it:
- A run document currently means a SuperPipeline run, and a real setup needs SuperPipeline, an AgentPod hub and the trace and log stores.
- It ships as one self-hosted Go binary for Linux (amd64 and arm64) with its own Postgres, under the MIT licence. There is no hosted service.
- The web app is for an allow-listed operator, not a multi-user service.
- Working with any agent stack, offline evaluations, calibration and comparing configurations are where it’s going, not what it does yet. It stores scores; it doesn’t run graders.
There is a fake mode that lets you try a development run on your own machine without the other products, though it still needs Postgres.
Read more on superwitness.dev or in the docs.