Before you run
- A taskset with at least one task.
- A successful build of every environment the tasks use, and any runtime variables their templates need.
- Credits. Starting a job is not checked, but its environments are billed and its model calls are refused once your team fails the credit check.
Start a job
Select tasks on a taskset’s Tasks tab, then Run task. Or open the + menu in the sidebar and select Evaluation job, then pick the taskset. The Run tasks dialog sets:
The run button shows how many traces will start: tasks × models × repetitions. HUD creates the job
and all of its traces at once, opens the job in a new tab, and starts each trace as
capacity frees up.
Every trace is billed for model usage and environment runtime. See
billing and usage.
Follow a job
The job finishes when every trace has. It ends as
error if any trace errored, otherwise as
cancelled if any trace was cancelled, otherwise as completed.
Read the results
Treat no-score as 0% in the Results menu counts traces without a reward as 0 in the
charts.
Keep bad traces out of the score
Invalidate a trace that should not count, such as one that hit an infrastructure failure. It stays on the job, but the leaderboard and reward summaries skip it.
Show invalidated traces in the Results menu brings them back into the charts, and
Revalidate all makes them count again.
Cancel, export, and share
- Cancel Job, in Details while the job runs, stops its pending and running traces and shuts down their instances.
- Export → Copy JSON or Download JSON gives the job’s
hud evalconfiguration, so you can run it again or adapt it from the terminal. - Share opens Share Job: invite people or teams as Viewer or Editor, or make the job public. Public viewers see Traces and Overview, not Usage.