HUD Documentation - Evaluations and RL Environments.

Every page in these docs uses the same vocabulary. Read this page once; the rest of the platform is these objects and the actions on them.

Objects

Concept	What it is
Team	Your organization on HUD. It owns every resource, credential, file, and credit balance.
Project	A workspace inside a team that groups resources and scopes who can see and change them. Every team has a Main project. See Projects.
Environment	A containerized world an agent acts in: a workspace with files and a shell, a browser, a desktop, a robot simulator, or any service with tools. See Environments.
Build	One deployed version of an environment’s source and image. Builds are immutable.
Task template	A parameterized task definition that an environment exposes, such as `review_files(prompt, attachments, criteria)`. The SDK calls it a scenario.
Task	One template with its arguments filled in: what the agent is asked, what it works with, and how it is graded.
Task version	An immutable snapshot of a task. Editing a task creates a new version; a trace keeps the version it ran.
Taskset	A named collection of tasks: a benchmark, a training set, or a queue of work for task authors. See Tasksets.
Evaluation job	One run of a set of tasks across models, with a number of repetitions. See Evaluations.
Trace	One attempt by one model at one task: every step, tool call, observation, and the final reward. See Traces.
Reward	The grader’s score for a trace, from 0 to 1.
QA result	A QA agent’s verdict on a trace or a task, with findings. See QA agents.
Model	An entry in the model catalog: a provider model, or a trainable model your team owns. See Models.
Checkpoint	A saved set of weights for a trainable model.
Data file	A file uploaded to the team’s store that tasks can attach as input. See Data.

Environments and tasks

An environment decides how work happens and is graded: the tools the agent gets and the grading logic. A task decides what is asked by filling in a template’s arguments, and each template defines its own: review_files asks for a prompt, files, and a rubric, while the robot gripper’s pick-object asks for a scene, an object, and a lift height. One environment serves many tasks, so people who never write code can create tasks by filling in a template’s form. See create tasks in the web app.

Versions and reproducibility

Nothing a trace depends on changes under it:

A trace records the task version and the environment build it ran.
Editing a task creates a new version. Earlier traces keep pointing at the version they ran.
A new build does not alter earlier builds. Runs started during a build keep the previous successful build until the new one is ready.

Because of this, jobs that run the same taskset stay comparable, and any trace can be explained from the exact inputs it saw.

Task status

Each task version moves through a review lifecycle: it is Pending while authors work on it, Ready when an author submits it, and then Verified, Revision, or Rejected after a reviewer decides. Tasksets can add custom stages before Ready. See task lifecycle and review.

Who can do what

Access works at three levels:

Level	Who it covers	Controls
Team role	Every member: Admin, Member, or External.	Settings, billing, members, and default visibility. See roles.
Project scope	A member or a whole team in one project: Contributor, View, Create, Review, or Manage.	What they can see, create, review, and administer in that project. See Projects.
Taskset scope	A member or team on one taskset: View, Create, or Review.	Work on a single taskset without wider project access. See Tasksets.

An API key acts as the member it belongs to, in that member’s team. See API keys and secrets.

Get started

Create tasks

Evaluate and train

QA agents

Use cases

Collaborate

Manage your team

API

Concepts

Objects

Environments and tasks

Versions and reproducibility

Task status

Who can do what

​Objects

​Environments and tasks

​Versions and reproducibility

​Task status

​Who can do what

Objects

Environments and tasks

Versions and reproducibility

Task status

Who can do what