Create a taskset
Names are saved in lowercase with hyphens (
My Taskset becomes my-taskset) and are unique within
a project; another project can use the same name. A new taskset belongs to the
active project.
Work in the task grid
The Tasks tab lists every task with its status, arguments, and results. Select tasks with their checkboxes, or right-click a task, and act on them:
Filter in the header chooses which traces the grid’s results count: by model, environment,
earliest environment version, task version, all jobs or only the latest, and how many recent traces
per task to keep. It also shows or hides training and errored runs. Filters and column widths are
saved in your browser; column order and custom columns are saved with the taskset for everyone.
Run the taskset
Select tasks, then Run task, and choose models and Repetitions. One job runs tasks × models × repetitions traces: 20 tasks, 2 models, and 3 repetitions start 120 traces. See evaluations for every run setting. Keep the taskset fixed when comparing models; change models, repetitions, or step limits at launch instead. When the suite itself changes, new task versions apply to future jobs, and existing traces keep the versions they ran.Compare results
Leaderboard ranks every model that ran the taskset, one row per model across all of its environments, from valid traces only:
A model that ran on more than one environment has a chevron that shows one row per environment
underneath, with the same columns. Select a model row to see its result on each task across all
environments, or an environment row to see it on that environment alone, then select a task to see
its traces. When a model ran on more than one environment version, Builds lists a row per
version under its environment and expands models that ran on several environments. Jobs lists
every evaluation job that ran the taskset.
Configure the taskset
Share a taskset
Open Settings → Access, or Share in the More actions menu.
Add people grants a scope to members of your team. Invite by email reaches anyone, including
people outside your team: they accept from their own HUD workspace within 14 days, and get access to
this taskset without joining your team. Project access shows access inherited from the
project.
A taskset share covers its tasks, jobs, and traces, and the environments its tasks run on: View
access to the environments for View, edit access for Create and Review. Environments you cannot
edit yourself are left out, with a warning. Data files keep their own
permissions; a share never grants access to them.
Publish a taskset
Turn on Publicly viewable to let anyone with the link view the taskset read-only, and copy the link. Publish runs also makes selected jobs public, with all of their traces. A public leaderboard shows your team’s valid results on the taskset, including the names and scores of private models. View as reader in the More actions menu previews the read-only view that people with View access get.Export a taskset
Export JSON in the More actions menu downloads every task’s latest version: its name, environment, template, arguments, and settings. To upload an export into another taskset, see tasksets in the API reference. From the command line,hud sync tasks <taskset> --export tasks.json writes the tasks to a .json, .jsonl, or .csv
file.