Avg individual fidelity
92%
What every per-agent dashboard reports. All seven stages look healthy on their own.
Observability & control plane for AI agents
Every agent passed. The pipeline failed.
Verdict watches every agent execution across the whole chain — capturing traces, scoring each output against the evidence it cited, and surfacing the compounding decay no single agent will ever report. The unified observability and control plane for multi-agent systems.
Worked example · national-phase patent pipeline · 7 agents
Each agent is competent. The chain is not.
Delivered pipeline fidelity
54%
The product of the chain. This is what actually reaches the client.
The invisible gap
38 pts
Lost across handoffs, reported by no single agent. Verdict is the layer that can see it.
The blind spots
Per-agent dashboards are green. The pipeline is broken.
Five failure modes hide in the gaps between agents — and conventional, per-component monitoring is structurally blind to every one of them. Verdict was built to watch the seams, not just the nodes.
01 · COMPOUNDING ERROR
92% per stage becomes 54% delivered
Each agent loses a little fidelity. Multiplied across seven stages, "healthy" silently compounds into failure — and every agent's own metrics stay green the entire time.
Per-stage and cumulative fidelity, scored continuously.
02 · VALIDATOR COVERAGE GAPS
An unscored stage is a stage you trust on faith
Some outputs have a rubric watching them. Others don't. The gap is where regressions ship unnoticed, because nothing was ever checking.
Reconciles which outputs actually have a judge attached.
03 · SILENT FAILURES
200 OK, empty body, work gone
An agent returns a well-formed nothing. The status code is healthy, the pipeline keeps moving, and the deliverable quietly evaporates. A 2xx is not a result.
Treats an empty success as the failure it is.
04 · LOSSY HANDOFFS
Fields one agent emits, the next never reads
Schema drift between adjacent stages drops data at the seam. No error fires; the information is simply absent downstream.
Diffs every handoff and flags the dropped fields.
05 · SINGLE-SUB-AGENT WRAPPING
An "orchestrator" that calls exactly one agent
That isn't a team — it's overhead pretending to be architecture: latency, cost, and a failure surface for no added capability. The declared topology lies.
Sees the real topology, not the one on paper.
↳ THE THROUGH-LINE
Watch the chain, not the components
Conventional monitoring asks "is this agent up?" Verdict asks "is the work still true by the time it reaches the client?" — across every hop, on every trace.
What Verdict does
It captures, scores, defends, and reports — on every execution.
Captures
trace observability
Every execution, end to end — input, output, the full span waterfall, tool calls, retrievals, and downstream invocations. Queryable across millions of traces with millisecond drill-down.
Scores
evaluation & quality
Per-agent rubrics graded continuously by an isolated judge. Retrieval faithfulness is first-class: every claim in an output is checked against the evidence the agent actually retrieved.
Defends
security & isolation
PII redacted before anything is persisted. Prompt injection caught at the door. Tenant isolation enforced at the lowest data layer — never by middleware that can be bypassed.
Reports
audit & outbound
Signed provenance for every causal chain. Scheduled reports, configurable alerts, and outbound webhooks into PagerDuty, Slack, and the workflows your teams already run.
Capability index · 16 functions
A complete observability surface, in five movements.
From the registry that knows every agent exists, to the signed report that proves what happened. Select a movement to see its capabilities.
Capture
Establish ground truth — what agents exist, what they did, and whether the two agree.Agent Registry
One source of truth for every deployed agent: identity, version, team membership, dependencies, ownership, lifecycle state — solo agents distinguished from orchestrators and members.
Trace Observability
Every execution end to end — input, output, span waterfall, tool calls, RAG retrievals, downstream invocations — queryable across millions of traces with millisecond drill-down.
Coverage & Health
Continuous reconciliation between registered agents and observed traces. Surfaces silent agents, ghost agents, and ingestion lag before they become incidents.
Evaluate
Decide whether the work is actually good — by machine, by human, and over time.Evaluation & Quality Scoring
Per-agent rubrics scored continuously by an isolated judge model. Retrieval faithfulness is first-class — every claim checked against the retrieved evidence.
Human Feedback
Reviewers attach thumbs-up/down with comments to individual traces. Negative signals promote into rubric refinement, with attribution and the active rubric version preserved.
Anomaly & Drift Detection
Detects regressions in score, cost, and latency over time, plus output-embedding drift per version — surfacing diverging examples for inspection.
Operate
Run the fleet well — understand cost, compare versions, test before you ship, and ship safely.Cost & Performance
Token-level cost by agent, tenant, and solution. MTD spend, daily run-rate, end-of-month forecast vs budget, and latency percentiles against per-agent baselines.
Comparison & Trend
Side-by-side comparison of two versions across any metric, audited cross-tenant comparison for admins, and a custom trend explorer with saved views.
Testing Tools
Curated simulation and replay sets. Sandbox a single agent, simulate full teams, or replay frozen production traffic with version and RAG-config overrides.
Deployment & Change Control
Pre-deploy regression validation, manual approval gates with attribution, atomic rollback within retention, and a signed report on every deploy.
Experimentation
A/B framework for rubric variants, RAG configs, and prompt changes — per-arm metrics with statistical confidence and one-click winner promotion.
Defend
Keep tenant data private, provable, and strictly isolated — at the data layer, not the perimeter.Security & Privacy
PII redaction before persistence (Protect AI Recon + GCP DLP), prompt-injection detection (Lakera Guard), and encryption at rest with customer-managed keys for dedicated tenants.
Compliance & Audit
Append-only audit log signed at write time with Cloud KMS. Signed provenance exports make every trace's causal chain verifiable end to end.
Multi-Tenancy & Access
Tenant scoping enforced at the lowest data layer, not by middleware. Built-in plus tenant-scoped custom roles; cross-tenant access requires admin and emits an audit entry.
Report
Turn observation into action and into artifacts — alerts that page the right person, reports that prove the month.Alerting & On-Call
Configurable rules for threshold, anomaly, coverage, SLA, and security events — routed to PagerDuty, Slack, or email per tenant policy, with evidence and related traces attached.
Reporting & Integrations
Scheduled daily / weekly / monthly reports as PDF, CSV, or signed bundle, plus outbound webhooks and native integrations for PagerDuty, Slack, and SendGrid.
Architecture & principles
A query, judgment, and coordination layer — not another system of record.
Verdict is deliberately narrow about what it owns. That restraint is what lets it stay trustworthy as the single lens over every agent, every tenant, every product.
01
The judge is never the agent by contract
A model never grades its own output. Every rubric's judge model must differ from the agent's model — self-grading is a hard, CI-enforced violation, not a guideline.
02
Isolation at the data layer
Tenant scoping is enforced at the lowest data layer, not by middleware that can be misconfigured or bypassed. Cross-tenant access requires an admin role and always emits an audit entry.
03
Modular monolith on Google Cloud
Hexagonal architecture exposing a REST API consumed by a React frontend. Simple to operate, hard to desynchronize — one deployable, clean ports and adapters at every edge.
04
Append-only, signed at write
The audit log is signed at write time with Cloud KMS. Provenance exports let any trace's causal chain be verified end to end — which filing, from which inputs, judged by which rubric version.
What Verdict owns — and deliberately doesn't
- ▸The operational registry of agent identity, version, lifecycle, and topology.
- ▸The query & visualization layer over trace data.
- ▸The evaluation pipeline and judgment surface.
- ▸Coverage, drift & anomaly detection.
- ▸Deploy gates, regression checks & change management.
- ▸The multi-tenant access & isolation model.
Owned elsewhere — on purpose
- ·Trace storage — Langfuse
- ·Model inference — providers
- ·Agent code & definitions — the git registry
- ·RAG corpora — Spanner Graph
- ·Billing & invoicing — surfaced, not stored
Anchor deployment
Built for the deployments that can't afford to guess.
Premium tenants run on dedicated single-tenant deployments — customer-managed encryption keys, EU data residency, isolation enforced at the data layer. The flagship is a high-stakes legal pipeline where a dropped field is a defective filing.
It's the pipeline in the chart above. Seven agents carry a filing from raw document to packaged handoff — each one competent on its own, every per-agent metric green.
doc_parser → asr_extractor → decision_engine → graph_builder → blueprint_gen → docs_renderer → handoff_pkg
Across the chain, fidelity quietly compounds downward — and the decision_engine → graph_builder handoff drops fields no error ever flags. To per-agent monitoring, nothing is wrong. To the client, the filing is degraded.
Verdict makes the decay visible stage by stage, scores every output against the evidence it cited, and turns the lossy handoff into something impossible to miss — with a signed provenance trail behind every filing.
See the gap your dashboards can't.
Bring a multi-agent pipeline. We'll walk it through Verdict and show you the per-stage fidelity, the lossy handoffs, and the silent failures your current monitoring is structurally unable to surface.