> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hud.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Run QA

> Run QA agents on a trace, a task, or a whole taskset, turn on automatic QA for every new trace, and run checks from the API or CLI.

Run [QA agents](/platform/qa-agents) wherever you read results: on one trace, on a task, or on a
selection across a taskset. Turn on automatic QA, and every new trace in a
taskset is reviewed when it completes.

To run QA you need:

* **Something to review.** Trace checks run on completed evaluation traces. Task checks run on a
  task's current version.
* **Edit access** to the trace or task.
* **Credit to start work.** QA is refused when your team cannot start new work. See
  [when work can start](/platform/billing#when-work-can-start).

## On a trace

Select **Run QA** in the trace header, or open the inspector's **QA** view and select **Run QA**.
Choose QA agents, then select **Run**. The QA agents that start selected are the ones without a
passing or in-progress result on the trace. Once any check has a result, the header shows the
trace's QA mark instead, and its menu opens the **QA** view.

## On a task

A task page runs both kinds of check:

* **Task checks.** The **Quality assurance** section of the task's **Overview** tab shows each task
  check's latest result on the current version and has **Run QA**. On the other tabs, the task
  header has the same action until a check has a result.
* **Trace checks on the task's traces.** The **Traces** tab's **Run QA** offers **Latest**, the
  newest completed trace on the current version, or **All**, every completed trace on the current
  version that you can edit. You can also select traces in the list and run QA on just those.

## From a taskset

Select tasks in the [task grid](/platform/tasksets#work-in-the-task-grid) and select **Run QA** in
the selection bar, or right-click a task and select **Run QA…**. The **Run QA on selected traces**
dialog runs task checks on the selected tasks and trace checks on the traces you pick from them:

| Setting | What it does |
| - | - |
| **QA agents** | The checks to run, in two groups. **Trace QA** checks run on the picked traces and start all selected. **Task QA** checks run on each selected task's current version and start with the ones some selected task lacks a passing or in-progress result from. **All** selects every check in both groups. |
| **Latest version only** | Only traces of each task's current version. On by default. |
| **Reward** | Only traces with any reward, a reward of 0%, below 100%, or exactly 100%. |
| **Auto-select latest** | Selects each task's newest matching traces, 5 by default. |
| **Run mode** | **Run new only**, the default, skips each trace or task whose latest result of a check is completed or in progress. **Overwrite all** runs every selected check on every trace and task again. |

The trace list starts from the grid's current filters, leaves out invalidated traces, and lets you
add or remove individual traces. The run button counts both groups, such as
**Run on 10 traces × 5 checks + 2 tasks × 2 checks**.

## Automatically

Open the taskset's **QA** tab and turn on the trace checks to run under **Automatic QA**. From then
on, each check runs once on every evaluation trace of the taskset's tasks when the trace completes,
whether it was started from the web app or from code. Turning a check on does not review traces
that already exist; run those from the task grid.

Automatic QA does not run on traces of a cancelled job, and does not run a check again on a trace
that already has a result of it. The review is billed to whoever ran the trace. When their team
cannot start new work, the trace is not reviewed, and you can run QA on it later.

## Running a check again

**Run QA** on a trace, a task, or a job runs every QA agent you select on every subject, including
subjects that already have a result of it, and each analysis is billed. The agents that start
selected are the ones some subject still lacks a passing or in-progress result from. To review only
what is new across a taskset, use the dialog's **Run new only** mode. A new result never deletes an
earlier one: earlier results stay in the subject's history.

## From code

The `hud` CLI waits for the checks to finish and prints each verdict. It skips a check on a subject
whose current result of it is completed or still running, unless you pass `--overwrite`:

```bash theme={"dark"}
hud qa list
hud qa run reward_hacking,false_positive <trace-id> <trace-id>
hud qa run task_reward_exploit,task_failure_analysis <task-id>
hud qa results <trace-id>
```

See the [CLI reference](/v6/reference/cli#qa-checks). Over HTTP, `POST /v2/qa/runs` takes
`check_keys` and either `trace_ids` or `task_ids`, up to 100 subjects per request:

```bash theme={"dark"}
curl -X POST https://api.beta.hud.ai/v2/qa/runs \
  -H "Authorization: Bearer $HUD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"check_keys": ["reward_hacking", "false_positive"], "trace_ids": ["<trace-id>"]}'
```

The request returns at once. Poll `GET /v2/qa/results` until each result is `completed`, `error`,
or `cancelled`. Over HTTP, `overwrite` defaults to `true`; with `"overwrite": false`, a check whose
current result is queued, running, or completed is not run again. See
[QA checks](/platform/rest-api#qa-checks) in the API reference.

## What QA costs

Each analysis is billed from your team's credits like a short run: the model usage of the QA agent,
shown as **HUD QA** on the [Usage](/platform/billing#usage) page, and the runtime of the environment it
runs in. A check that the reward decides passes without an analysis and costs nothing. QA started
from the web app runs in batches of 100 traces; if a batch is refused, for example because credit
ran out, the message says how many traces had started.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.