> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hud.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Tasksets

> Collect tasks into a suite, run it across models, compare results, configure review and automatic QA, and share or publish it.

A **taskset** is a named collection of tasks: a benchmark, a training set, or your team's regression
suite. It is also where the work on those tasks happens: authoring, review, runs, and QA. Jobs that
run the same taskset are comparable, because the suite stays fixed while the models and repetitions
change.

## Create a taskset

| Way | How |
| - | - |
| **New Taskset** | On [Tasksets](https://hud.ai/tasksets), select **New Taskset**, name it, and select **Create**. Then [create tasks](https://docs.hud.ai/platform/create-tasks.md). |
| **Import from Harbor** | Turn a zip of Harbor tasks and runs into a taskset. See [Harbor import](https://docs.hud.ai/platform/harbor-import.md). |
| **From Python** | `hud sync tasks <name>` uploads the tasks your code defines, and only what changed. See [tasks and tasksets](https://docs.hud.ai/v6/reference/tasks.md). |

Names are saved in lowercase with hyphens (`My Taskset` becomes `my-taskset`) and are unique within
the team. A new taskset belongs to the active [project](https://docs.hud.ai/platform/projects.md).

## Work in the task grid

The **Tasks** tab lists every task with its status, arguments, and results. Select tasks with their
checkboxes, or right-click a task, and act on them:

| Action | What it does |
| - | - |
| **Run task** | Start an [evaluation job](https://docs.hud.ai/platform/evaluations.md) on the selected tasks. |
| **Run QA…** | Run [QA agents](https://docs.hud.ai/platform/run-qa.md#from-a-taskset) on the selected tasks and their traces. |
| **Set status** | Move tasks through the [task lifecycle](https://docs.hud.ai/platform/task-lifecycle.md). |
| **Duplicate**, **Duplicate to taskset…** | Copy tasks into this taskset or another. |
| **Move to taskset** | Move tasks to another taskset. |
| **View Raw** | Show the selected tasks as JSON. |
| **Enable binarized scoring** | Show a task's reward as 1 when it is at full reward, and 0 otherwise. Display only; stored rewards and the leaderboard do not change. |
| **Delete task** | Delete the tasks. |

**Filter** in the header chooses which traces the grid's results count: by model, environment,
earliest environment version, task version, all jobs or only the latest, and how many recent traces
per task to keep. It also shows or hides training and errored runs. Filters and column widths are
saved in your browser; column order and custom columns are saved with the taskset for everyone.

## Run the taskset

Select tasks, then **Run task**, and choose models and **Repetitions**. One job runs tasks × models ×
repetitions traces: 20 tasks, 2 models, and 3 repetitions start 120 traces. See
[evaluations](https://docs.hud.ai/platform/evaluations.md#start-a-job) for every run setting.

Keep the taskset fixed when comparing models; change models, repetitions, or step limits at launch
instead. When the suite itself changes, new task versions apply to future jobs, and existing traces
keep the versions they ran.

## Compare results

**Leaderboard** ranks every model that ran the taskset, one row per model across all of its
environments, from valid traces only:

| Column | Meaning |
| - | - |
| **Average** | Mean reward across the model's traces. |
| **Best\@3**, **Best\@5**, **Best\@10** | For each task with at least k traces, the expected best reward of k attempts, averaged across those tasks. |
| **Pass\@1** | The share of the taskset's tasks with at least one trace at full reward. |
| **Steps** | Mean agent steps per trace. |
| **Runs** | Traces counted for the row. |

A model that ran on more than one environment has a chevron that shows one row per environment
underneath, with the same columns. Select a model row to see its result on each task across all
environments, or an environment row to see it on that environment alone, then select a task to see
its traces. When a model ran on more than one environment version, **Builds** lists a row per
version under its environment and expands models that ran on several environments. **Jobs** lists
every evaluation job that ran the taskset.

## Configure the taskset

| Where | Setting |
| - | - |
| **Settings** → **Details** | **Taskset name**; **Purpose**: **Benchmark** tasksets appear in model results, **Training** tasksets appear when filtering for training data. |
| **Settings** → **Lifecycle stages** | Optional steps tasks pass through before they can be marked ready. See [task lifecycle](https://docs.hud.ai/platform/task-lifecycle.md#add-review-stages). |
| **QA** | **Automatic QA**: QA agents that run on every completed trace in the taskset. See [run QA automatically](https://docs.hud.ai/platform/run-qa.md#automatically). |
| **Settings** → **Danger zone** | **Delete taskset**, after typing its name. |

## Share a taskset

Open **Settings** → **Access**, or **Share** in the **More actions** menu.

| Scope | Grants |
| - | - |
| **View** | View this taskset, its tasks, jobs, and traces. |
| **Create** | View, create, edit, and run tasks and jobs in this taskset. Does not grant access management or task review. |
| **Review** | Everything in Create, plus review tasks and approve work that is ready. Does not grant access management. |

**Add people** grants a scope to members of your team. **Invite by email** reaches anyone, including
people outside your team: they accept from their own HUD workspace within 14 days, and get access to
this taskset without joining your team. **Project access** shows access inherited from the
[project](https://docs.hud.ai/platform/projects.md).

A taskset share covers its tasks, jobs, and traces, and the environments its tasks run on: View
access to the environments for View, edit access for Create and Review. Environments you cannot
edit yourself are left out, with a warning. [Data files](https://docs.hud.ai/platform/data.md) keep their own
permissions; a share never grants access to them.

### Publish a taskset

Turn on **Publicly viewable** to let anyone with the link view the taskset read-only, and copy the
link. **Publish runs** also makes selected jobs public, with all of their traces. A public
leaderboard shows your team's valid results on the taskset, including the names and scores of
private models.

**View as reader** in the **More actions** menu previews the read-only view that people with View
access get.

## Export a taskset

**Export JSON** in the **More actions** menu downloads every task's latest version: its name,
environment, template, arguments, and settings. To upload an export into another taskset, see
[tasksets](https://docs.hud.ai/platform/rest-api.md#tasksets) in the API reference. From the command line,
`hud sync tasks <taskset> --export tasks.json` writes the tasks to a `.json`, `.jsonl`, or `.csv`
file.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.