> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hud.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Tasksets

> Create a reusable evaluation suite on HUD and run it across models.

A **taskset** is a named collection of concrete tasks, used as an evaluation suite. Jobs that run
the same taskset answer the same question, even when the models or the number of attempts change.

If your tasks already live in Python, publish them from the project that defines them:

```bash theme={"dark"}
hud sync tasks my-tasks
```

The taskset then appears under [Tasksets](https://hud.ai/tasksets).

## From template to taskset

| Concept           | What it is                                                                         |
| ----------------- | ---------------------------------------------------------------------------------- |
| **Task template** | A parameterized `@env.template()` definition discovered from an environment build. |
| **Task**          | One concrete set of arguments for a template.                                      |
| **Task version**  | An immutable snapshot of a task. Traces retain the version they ran.               |
| **Taskset**       | A named collection of tasks used as an evaluation suite.                           |

Tasks are mutable containers, but their versions are not. Editing a task creates a new version
without changing the task version attached to an existing trace. This keeps old jobs interpretable
after the suite changes.

## Create a taskset

There are two supported entry points:

1. **Sync from the SDK.** Define tasks in Python and run `hud sync tasks <name>`. HUD uploads only
   what changed.
2. **Create in the platform.** Select **New taskset**, then create tasks individually or upload a
   task file from the taskset page.

A runnable task version must point to a task template from a deployed environment. If no runnable
versions exist, the platform will not start the batch.

For task authoring patterns and graders, see [tasks and tasksets](/v6/reference/tasks) and
[designing tasks](/v6/reference/advice).

## Run the suite

Select **Run Taskset** and configure:

| Setting        | Effect                                                   |
| -------------- | -------------------------------------------------------- |
| **Models**     | One or more gateway models to evaluate.                  |
| **Group size** | Independent attempts for every task and model pair.      |
| **Max steps**  | The maximum number of agent steps allowed in each trace. |

The platform expands the batch as:

```text theme={"dark"}
trace count = tasks x models x group size
```

A taskset with 20 tasks, 2 models, and a group size of 3 creates 120 traces in one
[evaluation job](/platform/evaluations).

Group size is also how evaluation connects to [training](/platform/training): training compares
groups of attempts at the same task, so a run with groups produces its input directly.

## Read the taskset page

| Tab                | What it shows                                                             |
| ------------------ | ------------------------------------------------------------------------- |
| **Overview**       | Description, aggregate results, and top-performing models.                |
| **Tasks**          | Current task versions, their templates, arguments, and task-level traces. |
| **Jobs**           | Evaluation jobs that ran this suite.                                      |
| **Report**         | A generated report for tasksets configured with a due date.               |
| **Settings**       | Name, system prompt, and optional review stages.                          |
| **Access control** | Teams and members with scoped access.                                     |

The available tabs depend on your access. Public viewers see the published summary; editors can
change tasks, launch jobs, and manage access.

## Compare without changing the suite

Keep the taskset fixed when comparing models or repeated runs. Change the models, group size, or
step limit at job launch instead. Results across jobs then stay comparable, because the suite
itself never moved.

When the suite itself changes, the new task versions apply to future jobs. Existing traces continue
to reference the versions they actually ran.

## Share and export

Use **Share** to grant viewer or editor access. Use **Export JSON** to download the runnable task
definitions. Public tasksets can be viewed through their shared URL without signing in.

A runnable suite is the input to an [evaluation job](/platform/evaluations).