> ## Documentation Index
> Fetch the complete documentation index at: https://www.botgauge.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Evals overview

> Evals run an AI judge over your agent's traces and show pass or fail for every row in Logs. Learn judges, rules, results, and what each level reads.

Evals run an AI judge over your agent's traces and tell you, row by row in Logs, whether each one passed. You turn a judge on once, and BotGauge scores new data as it arrives.

## The three ideas

| Idea | What it is |
| - | - |
| **Judge** | A grading recipe, such as Correctness or Toxicity. Also called an evaluator. Judges are listed on the **Judges** page. |
| **Rule** | "Run this judge at this level on this share of data." Turning a judge on creates a rule. Its settings live on the judge's Configure tab. |
| **Result** | One verdict per judge and item: pass, fail, a score, or a category. It appears as a column in Logs. |

## How it works

**1. Traces arrive**

Your agent sends traces as usual. Nothing in your code changes. See [Tracing overview](https://www.botgauge.com/docs/tracing/overview.md).

**2. A rule picks items**

For each judge you turned on, the rule keeps the items that match its filter, then scores the sampling share of them.

**3. A worker scores them**

Span judges score soon after a span arrives. Trace and session judges wait until the item has been quiet for the wait time. Failures are retried.

**4. You read results**

Each judge is a column in Logs, and the same results are available through the API.

## Levels: what the judge reads

| Level | What it reads | When it is scored |
| - | - | - |
| **Span** | One step: a model call, tool call, or retrieval. | Within minutes of arriving. Text judges only look at spans that carry a question and its answer (agent, chain, and model spans), not tool calls or retrievals. |
| **Trace** | All steps of one request. | After the trace has been quiet for a while. The default is 10 minutes. |
| **Session** | A whole conversation made of many traces. | After the session has been quiet. The default is 30 minutes. |

> [!NOTE]
>
> Roll-ups go upward only. A span result also shows on its trace and session. A session-level judge such as Agent Flow shows only on the Sessions tab.

## Terms used in these docs

| Term | Meaning |
| - | - |
| Judge, evaluator | A grading recipe, usually an AI model with a prompt. Built-in or custom. |
| Rule | A judge turned on at one level, with its sampling, filter, and wait time. |
| Live scoring | Scoring data as it arrives. This is what happens after you turn a judge on. |
| Past data, backfill | A one-time run over older data, chosen when you first turn a judge on. |
| Sampling | The share of items a rule scores, from 0 to 100%. |
| Settle time | The quiet period a trace or session needs before it is scored. The setting is called **Wait before scoring**. |
| Score now | Skips the quiet period for one trace or session. |
| Quota | The monthly number of judge verdicts your plan allows. |
| Sample data | The demo traces you can load during onboarding. |

## Who can do what

Turning judges on, changing settings, and creating custom judges needs the workspace **Admin** or **Member** role. **Viewers** can read results but can't change anything. See [Roles and permissions](https://www.botgauge.com/docs/platform/roles-and-permissions.md).

Turn on your first judgeOpen the Judges page in [BotGauge](https://cloud.botgauge.com) to start scoring your agent traces today.

[Try BotGauge →](https://cloud.botgauge.com)

## Next steps

> **[Evals quickstart](https://www.botgauge.com/docs/evals/quickstart.md)**
>
> From first trace to first result in four steps.

> **[Built-in judges](https://www.botgauge.com/docs/evals/built-in-judges.md)**
>
> The 18 judges you can turn on today.

> **[Custom judges](https://www.botgauge.com/docs/evals/custom-judges.md)**
>
> Write a judge for your own rule.

> **[Reading results](https://www.botgauge.com/docs/evals/reading-results.md)**
>
> What every cell in Logs can say.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.