Failures and troubleshooting

BotGauge retries, then tells you why

If the judge call fails for a temporary reason, BotGauge retries on its own. It makes up to 5 attempts spread over about 45 minutes, roughly 30 seconds, 2 minutes, 8 minutes, then 32 minutes, each with some randomness. A permanent failure is not retried. Spans whose judge never answered are refunded against your quota.

Two failed cells with cause pills and a retrying cell with a retry counter — Cells that have failed for good, and a cell that is retrying.

Cause pill. “Judge not authorised”.
Needs attention. No retry can fix this one.
Cause pill. “Judge timed out”.
Reason. A short detail from the judge call.
Retry line. “Retry 2/5” with the time of the next attempt.

The cause appears in the cell and in its tooltip.

Failure causes

Label in the cell	Meaning	Retried?
Judge timed out	The judge model did not answer in time.	Yes
Judge busy	The model provider is rate limiting.	Yes
Judge unavailable, or unreachable	Provider error or network problem.	Yes
No answer from judge, reply unreadable, or reply invalid	The judge replied with nothing usable.	Yes
Waiting for telemetry	The trace was incomplete when scored.	Yes
Judge not authorised	The provider rejected the judge’s credentials.	No
Judge misconfigured, or not configured	The judge model is not set up correctly or at all.	No
Judge budget used up	The provider account is out of budget.	No
Judge reply cut off	The answer was too long for the judge to finish.	No
Missing input	The row has no data for something the judge needs. This is a quiet pill, not an error.	No

What the API returns

A cell that is waiting returns evaluation_state of pending, and a cell that is retrying returns retrying.

pending cell

{
  "evaluation_state": "pending",
  "pending_reason": "waiting_to_settle",
  "pending_message": "Waiting for trace to settle, scores at 6:33 PM.",
  "scores_at": "2026-10-21T13:03:00+00:00",
  "can_score_now": true
}

retrying cell

{
  "evaluation_state": "retrying",
  "not_evaluable_code": "timeout",
  "retry": { "status": "retrying", "attempt": 2, "max_attempts": 5,
             "next_attempt_at": "2026-10-21T13:05:10+00:00" }
}

Common questions

My traces show 'Settles ...' and nothing else.

That is the quiet period. Wait, choose Score now, or shorten Wait before scoring on the judge.

Everything says 'Not backfilled'.

Those rows are older than the moment you enabled the judge. Past data can only be added when a judge is first turned on. Duplicate the judge and turn the copy on with Include past spans, or wait for new traces.

A column is missing on the Traces or Spans tab.

The judge runs at session level. Open the Sessions tab.

'Judge not authorised' or 'not configured'.

The judge model’s credentials or setup need fixing by whoever runs your BotGauge deployment. Retrying will not help.

The toggle works but my past-data choice is greyed out.

Past data can only be chosen when a judge is first turned on. Duplicate the judge and turn the copy on with Include past spans.

'Not eligible' on spans I expected to be scored.

Text judges only grade agent, chain, LLM, and model spans. Tool calls and retrievals are skipped on purpose. Use a trace judge to include them.

'Sample limit reached' on sample rows.

Five different judges already run over the sample data. The sample is only for trying things out. Run the judge on your own data.

My span judges stay 'Queued' for minutes.

Span judges normally finish in seconds. If the fast path is unavailable, a catch-up job picks them up after about 5 minutes.

I raised the sampling rate but old rows are still 'Not sampled'.

Sampling is decided once per item. To score older rows, duplicate the judge and turn the copy on with Include past spans.

Can I trust the verdict?

Judges are AI models, and their verdicts are advisory. Span text goes into the judge’s prompt, so use Test on real examples and spot-check results.

Inspect and retry failed cells

Open BotGauge to view error causes in Logs and use Score now when a fix is in place.

Try BotGauge →

Get started

Judges

Results

Reference

Failures and troubleshooting

BotGauge retries, then tells you why

Failure causes

What the API returns

Common questions

​BotGauge retries, then tells you why

​Failure causes

​What the API returns

​Common questions

BotGauge retries, then tells you why

Failure causes

What the API returns

Common questions