Skip to main content
For teams with coding tasks in Harbor format: task folders with an instruction, an environment, and a verifier. HUD reads them as they are. Import the tasks and the runs Harbor already recorded, let QA agents find the tasks whose verifiers can’t be trusted, fix them, and run the fixed set on any model on HUD’s hosted infrastructure.

​
Two ways in

Most teams start with an import to see where the benchmark stands, then adapt the tasks to run them on HUD.

​
Find the broken tasks

1

Import tasks and runs

Zip your task folders and job folders, and import them into a taskset. Each task folder becomes a task, each job an evaluation job, and each finished trial a trace with its trajectory and verifier reward. See Harbor import.
2

Run QA on the runs

In the taskset’s task grid, select the tasks and select Run QA to review their traces. See run QA.
3

Read what QA found

Each check points at the steps and files involved. See read results.
4

Fix and import again

Change the task folder and import it again. A task whose content changed moves to a new version, and runs you import later link to the version they ran. See import again.
What each trace check finds in coding tasks:

​
Run the tasks on HUD

The SDK adapts Harbor task folders into HUD environments and tasks, keeping the Harbor environment and verifier:
See adapt Harbor tasks for the full flow. Once deployed, run the taskset on any model from the web app or with hud eval, as many times as you need. Turn on automatic QA so every new trace is reviewed when it completes, and use file tracking to see the code the agent changed at each step.

Harbor tasks

Import, adapt, and export, with the zip layout and every limit.

QA agents

What each check reads and decides.

Harbor interoperability

Adapt, import, and export from the SDK.