> ## Documentation Index
> Fetch the complete documentation index at: https://www.botgauge.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Rules and settings

> Control what BotGauge scores and when: sampling rate, wait before scoring, past data runs, filters, turning a judge off, and testing a judge on one example.

Open a judge from the **Judges** page to reach its settings.

![The Configure tab of the Correctness judge with numbered callouts for state, duplicate, scope, semantic kind, prompt, sampling, wait, past data, and output](https://mintcdn.com/botguage/TsL7gBlR3XTR25B3/images/evals/judge-configure-tab.png?fit=max\&auto=format\&n=TsL7gBlR3XTR25B3\&q=85\&s=ef1496840beb3414685b142891e76021)

The Configure tab of a judge (Correctness).

1. **Eval state.** Pause or resume the rule without losing its settings.
2. **Duplicate** to create an editable copy.
3. **Select scope.** Span, trace, or session. Fixed for built-in judges.
4. **Semantic kind.** The span types the judge applies to, here LLM, Agent, Chain, and Model.
5. **Judge prompt.** Read-only for built-ins. Editable on your own judges.
6. **Sampling rate.** Score every item (100%) or only a share of them.
7. **Wait before scoring.** How long a trace or session stays quiet before it is judged. A hint says when you are on the default.
8. **Past data is locked.** After a judge has been on, this choice is fixed. To include past data, duplicate the judge.
9. **Output.** Boolean (pass or fail), score (0 to 1), or category.

## Sampling rate

Sampling scores a share of items instead of all of them. The choice is made once per item, using a stable hash. Raising the rate later does not score items that were skipped earlier. To catch up, duplicate the judge and turn the copy on with **Include past spans**.

Items that were not picked read **Not sampled**.

## Wait before scoring

Traces and sessions are judged only after they stop receiving new spans. In the console you can choose 2, 5, 10 (the default), 15, or 30 minutes for traces, or a custom time from 1 minute to 7 days. Sessions default to 30 minutes. Under the chips, a hint tells you when you are on the default, for example "Using the default (10 min)".

A short wait gives results sooner, but a conversation that keeps going may be scored before it ends.

> [!WARNING]
>
> The first result is final. Span judges have no wait.

## Include past data

Past data scores older data from a start date, then continues with new data. You can choose it only when you first turn the judge on, and the **Backfill** column on the Judges page shows its progress.

The start date means the start of your local day. The API can tell you the exact count first, in spans, traces, or sessions depending on the judge's level. If your monthly quota runs out, the run pauses and resumes by itself.

| | Live scoring | Past data run |
| - | - | - |
| **Covers** | Everything that arrives after you turn the judge on | Older data from the date you pick, up to the moment you turned it on |
| **When you choose it** | Always on | Only at first turn-on |
| **Shown in** | Logs cells | Backfill column and Logs cells |
| **Cost** | Counts toward quota | Counts toward quota, and pauses if it runs out |

## Turn a judge off

Turning a judge off pauses scoring. Results already produced are kept. Data that arrives while it is off is not scored.

## Filters

A rule can be limited to matching data with a filter. Filters are set through the API today. Use one operator per filter, either all `AND` or all `OR`.

```http request theme={null}
PATCH /v1/eval-rules/{id}
{ "filter_text": "\"attributes.model_id\" = 'gpt-5.6-luna' AND \"span.duration_ms\" > 200" }
```

> [!NOTE]
>
> Filters apply before sampling. A rule first picks the items that match its filter, then scores the sampling share of them.

## Test a judge on one real example

Open a judge and choose the **Test** tab. The list defaults to the last 14 days and shows only the span types the judge applies to. Search or filter, scroll to load more, and pick one. The panel on the right shows the selected span. Choose **Test eval** to see the result, a score and the judge's reasoning. Nothing is saved.

![The Test tab of a judge with a list of spans on the left and the selected span and result on the right](https://mintcdn.com/botguage/TsL7gBlR3XTR25B3/images/evals/judge-test-tab.png?fit=max\&auto=format\&n=TsL7gBlR3XTR25B3\&q=85\&s=6bb59aacbe9555fa3d69f0ecf7f0cd9d)

The Test tab. Pick a span on the left, read the verdict on the right.

1. **Time window.** Defaults to the last 14 days. More spans load as you scroll.
2. **Eligible spans.** Only the span types this judge applies to are listed.
3. **Pick one example.** Search or filter real spans, then select one.
4. **Test eval, or Test again.** Runs the judge once on the selected span. Nothing is saved.
5. **Result, score, and reasoning.** The reasoning shown here comes from a local stand-in judge. A real judge writes its own explanation.

## Next steps

[Custom judges](https://www.botgauge.com/docs/evals/custom-judges.md) shows how to write your own, and [Reading results](https://www.botgauge.com/docs/evals/reading-results.md) explains the cells in Logs.

Adjust scoring rulesOpen a judge in [BotGauge](https://cloud.botgauge.com) to change sampling, wait time, and filters or test on a real example.

[Try BotGauge →](https://cloud.botgauge.com)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.