> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hud.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluations

> Launch an evaluation job on HUD, then read its traces, rewards, errors, and usage.

Pick a taskset and models, then launch the batch. An **evaluation job** is the batch; a **trace**
is one model attempting one task within it. HUD gives every trace its own environment instance and
records the full trajectory. The trace viewer replays it with the model conversation and the
environment state aligned at every step.

## Before you run

A platform evaluation needs:

* a [taskset](/platform/tasksets) with at least one runnable task
* an [environment](/platform/environments) with a successful build for each task
* one or more [models](/platform/models) with an available checkpoint
* any environment variables required by the selected task templates

## Launch a job

Start from **Run Taskset** on a taskset page, or select **Evaluation job** from the add menu in the
platform sidebar.

1. Select the taskset.
2. Select one or more models.
3. Choose the group size, which controls repeated attempts for every task and model pair.
4. Set the maximum number of agent steps per trace.
5. Select **Start Evaluation**.

HUD creates the job and its traces immediately, then schedules each trace as capacity becomes
available. A queued trace stays `pending` until a slot opens.

## Statuses

| Status         | Meaning                                                               |
| -------------- | --------------------------------------------------------------------- |
| `pending`      | The trace exists and is waiting for scheduling.                       |
| `initializing` | HUD has claimed the trace and is preparing its environment.           |
| `running`      | The model is attempting the task.                                     |
| `completed`    | The attempt finished and returned its result.                         |
| `error`        | The attempt failed before completing normally.                        |
| `cancelling`   | Cancellation was requested and infrastructure cleanup is in progress. |
| `cancelled`    | The attempt stopped without completing.                               |

The job status rolls up its traces. A job completes when every trace is terminal. If any trace
errors, the job is marked `error`; otherwise a cancelled trace makes the job `cancelled`.

## Read the job

The job page has three views:

| Tab          | What it answers                                                 |
| ------------ | --------------------------------------------------------------- |
| **Overview** | How did the batch perform across tasks and models?              |
| **Traces**   | What happened in each individual attempt?                       |
| **Usage**    | How much inference and environment runtime did the job consume? |

The overview shows reward distributions, task coverage, errors, and tool usage. Use the task grid
to find weak tasks or inconsistent models, then open the corresponding traces.

A trace can be marked invalid when it should not contribute to the evaluation, such as an
infrastructure failure that produced a misleading reward. Invalidating a trace preserves the
record but excludes it from valid reward aggregates.

## Inspect a trace

The available inspector views depend on the environment:

| View                                         | What it contains                                                     |
| -------------------------------------------- | -------------------------------------------------------------------- |
| **Environment**, **Screen**, or **Robotics** | The observations and interactive surface exposed by the environment. |
| **Workspace**                                | Reconstructed files at the selected point in the trajectory.         |
| **Source**                                   | The immutable build source used by the trace.                        |
| **Terminal**                                 | Shell history when the environment exposes a terminal workflow.      |
| **QA review**                                | Structured review results attached to the trace.                     |
| **Logs** and **Debug**                       | Environment and runner evidence for failed attempts.                 |
| **Info**                                     | Task, model, build, timing, and trace metadata.                      |

The final reward is only the verdict. Read the trajectory and grading result together to determine
whether the agent solved the intended task or found a shortcut. For workspace reconstruction and
redaction behavior, see [file tracking](/platform/file-tracking).

## Cancel, export, and share

* **Cancel** stops pending and active traces, then cleans up their instances.
* **Export** downloads the job's `hud eval` configuration so the run can be reproduced or adapted.
* **Share** grants scoped access or publishes a public job URL.

To launch the same workflow from the terminal or Python, see
[evaluating agents](/v6/guides/running-an-eval). To automate it over HTTP, see the
[rollout endpoints](/platform/rest-api#runs-and-evals).