> ## Documentation Index
> Fetch the complete documentation index at: https://www.botgauge.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# API and limits

> Run BotGauge evals from code: authentication, every eval endpoint, example requests for enabling, score-now, and results, plus monthly quota and size limits.

Everything in the console uses these endpoints.

## Authentication

Send an API key for your project, or a token exchanged from one, as a bearer token. See [API keys](https://www.botgauge.com/docs/platform/api-keys.md). The project comes from the credential, so no project ID goes in the path. Writes need the Admin or Member role.

Errors look like this.

```json error theme={null}
{ "error": { "code": "...", "message": "...", "request_id": "..." } }
```

Other details to know:

* The `since` date for past data can be a date or a full date and time. The console sends the start of the local day.
* Judge keys in results look like `builtin:correctness`. When turning a judge on, you can send the bare key (`correctness`).

## Endpoints

| Method and path | Purpose |
| - | - |
| `GET /v1/evaluators/catalog` | List judges with their enabled state, rule, and backfill status. Filters: `search`, `source=builtin\|custom`. |
| `POST /v1/evaluators/custom` | Create a custom judge. |
| `PATCH /v1/evaluators/{key}` and `DELETE` | Edit or retire a custom judge. Built-ins are read-only. Disable a judge before deleting it. |
| `GET /v1/evaluators/evidence-keys?scope=` | Which fields a prompt may use at each scope. |
| `GET /v1/evaluators/filter-options?scope=` | Fields available to a rule filter. |
| `POST /v1/eval-rules/enable` | Turn a judge on at a scope. |
| `GET` and `PATCH /v1/eval-rules/{id}` | Read a rule. Change `sampling`, `filter_text`, or `settle_seconds`. |
| `POST /v1/eval-rules/{id}/enable` and `/disable` | Resume or pause a rule. |
| `POST /v1/eval-rules/{id}/estimate-backfill` | Exact count of what a past-data run would score. |
| `POST` and `GET /v1/eval-rules/{id}/backfill` | Start a past-data run, or read its progress. |
| `POST /v1/eval-rules/test` | Run a judge once on text you supply. Nothing is stored. |
| `POST /v1/eval-results/query` | Read results for up to 500 traces, sessions, or spans. |
| `POST /v1/eval-results/score-now` | Skip the quiet period for one trace or session. |
| `GET /v1/eval-usage` | Evaluations used this month and the limit. |

## Turn a judge on, with past data

```http request theme={null}
POST /v1/eval-rules/enable
{
  "evaluator_key": "coherence",
  "subject_kind": "trace",
  "mode": "backfill",
  "since": "2026-10-14",
  "sampling": 0.5
}
```

```json response theme={null}
{
  "id": "orr_01a15e8807a97e91a9efccfc5e9449d5",
  "name": "builtin:coherence:trace",
  "subject_kind": "trace",
  "sampling": 0.5,
  "enabled": true,
  "backfill_status": "queued",
  "backfill_since": "2026-10-14T00:00:00+00:00",
  "effective_settle_seconds": 600
}
```

## Skip the wait for one trace

```http request theme={null}
POST /v1/eval-results/score-now
{
  "subject_kind": "trace",
  "subject_id": "2d9aafe472de92e59cf7701f28d110fd",
  "evaluator_key": "builtin:correctness"
}
```

```json response theme={null}
202 Accepted
{ "request_id": "esr_d85bcacd80bb4d2da0ad", "status": "queued", "subject_kind": "trace" }
```

Asking twice returns the same request. Score now works for traces and sessions, needs an enabled rule at that level, and respects sampling and filters. If the worker fails, the cell shows **Score now failed** and the button stays.

## Read results

```http request theme={null}
POST /v1/eval-results/query
{ "subject_kind": "trace", "subject_ids": ["2d9aafe472de92e59cf7701f28d110fd"] }
```

```json response theme={null}
{
  "results": {
    "2d9aafe472de92e59cf7701f28d110fd": [
      {
        "evaluator_key": "builtin:correctness",
        "display_name": "Correctness",
        "evaluation_state": "scored",
        "single_outcome": "PASS",
        "explanation": "Stub judge: looks fine.",
        "count": 1, "passed_count": 1, "failed_count": 0,
        "pending_reason": null, "can_score_now": false, "retry": null
      }
    ]
  }
}
```

`evaluation_state` is one of `scored`, `pending`, `retrying`, `failed`, `not_sampled`, `filtered_out`, or `not_backfilled`. `scored` only means a row exists. Read `single_outcome` (`PASS`, `FAIL`, `NOT_EVALUABLE`, or `ERROR`) for the verdict.

## Quota and limits

* Free plans get **5,000 evaluations per month** across the whole organization. Sample data never counts. Each judge verdict counts, including "not evaluable" ones. Paid plans are not capped.
* Turning a judge on never fails because of quota. If you run out, scoring pauses and past-data runs wait.
* Judges read at most 20,000 characters each of a trace's input and output.
* Sampling is 0 to 1. Filters are up to 2,000 characters. A results query takes up to 500 items. A test accepts up to 50,000 characters.
* Past data is limited by how long spans are retained, about 90 days, and can be added only when a judge is first turned on.
* Each project can run up to 5 different judges over its sample data.

```http request theme={null}
GET /v1/eval-usage
```

```json response theme={null}
{ "month": "2026-10", "quota_state": "enforced", "used": 40, "limit": 5000 }
```

## Keep an eye on cost

| Control | How it works |
| - | - |
| **Sampling rate** | Score only a share of items. Items that were not picked read **Not sampled**. |
| **Filters** (API) | Limit a rule to matching data. The filter is applied first, then sampling. |
| **Past-data runs** | They count toward quota. The API can tell you the exact number of items before you start, and a run pauses if quota runs out. |
| **Usage meter** | The Judges page shows evaluations used this month against your limit. |

Run evals from codeOpen your project settings in [BotGauge](https://cloud.botgauge.com) to copy an API key and start calling the eval endpoints.

[Get an API key →](https://cloud.botgauge.com)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.