Skip to main content
Evals run an AI judge over your agent’s traces and tell you, row by row in Logs, whether each one passed. You turn a judge on once, and BotGauge scores new data as it arrives.

​
The three ideas

​
How it works

1

Traces arrive

Your agent sends traces as usual. Nothing in your code changes. See Tracing overview.
2

A rule picks items

For each judge you turned on, the rule keeps the items that match its filter, then scores the sampling share of them.
3

A worker scores them

Span judges score soon after a span arrives. Trace and session judges wait until the item has been quiet for the wait time. Failures are retried.
4

You read results

Each judge is a column in Logs, and the same results are available through the API.

​
Levels: what the judge reads

Roll-ups go upward only. A span result also shows on its trace and session. A session-level judge such as Agent Flow shows only on the Sessions tab.

​
Terms used in these docs

​
Who can do what

Turning judges on, changing settings, and creating custom judges needs the workspace Admin or Member role. Viewers can read results but can’t change anything. See Roles and permissions.

Turn on your first judge

Open the Judges page in BotGauge to start scoring your agent traces today.

Try BotGauge →

​
Next steps

Evals quickstart

From first trace to first result in four steps.

Built-in judges

The 18 judges you can turn on today.

Custom judges

Write a judge for your own rule.

Reading results

What every cell in Logs can say.