https://api.hud.ai, every public route under /v2, authenticated with a team API
key. A first call, returning what an agent actually did on a single attempt:
Orientation Authentication · Conventions · Platform modelEndpoint reference Core · Runs and evals · Secondary · Utilities · Secrets · Account · Team · Sign in with HUD · Errors · Deprecations
Authentication
Create a key in Settings → API keys and send it as a bearer token on every request (see above). See API keys and secrets. Missing or invalid credentials return401. A valid key that may not touch the resource returns
403. A valid key whose owner can no longer use the team, because the user or team is banned or
deleted or the user was removed, returns 403 with error membership_unavailable. Removing a
member or deleting an account also revokes its keys, which then return 401.
Two kinds of access appear across the surface. An API key is the default; the groups below call
out where an endpoint differs.
A partner application can instead send its users’ OAuth access tokens to a subset of the API; see
Sign in with HUD.
Conventions
Every example on this page is generated from the
OpenAPI document, so field names match what the server
sends and receives. The values are placeholders. For a console that sends real requests, use the
interactive reference.
Platform model
Nothing runs until three things exist: a place for an agent to act, something to ask it to do, and a model to do it. The API is organized around those objects.
Environments and builds are the stage, tasksets and tasks are the script, jobs and rollouts are the
action, and traces are the recording. The remaining groups cover credentials, models, live
capacity, and cost.
Endpoint reference
Core
The objects you set up before anything runs: the environment an agent will act in, the suite you grade it on, and the individual tasks inside that suite, written in HUD or imported from elsewhere.Environments
An environment is the sandboxed application an agent works in, packaged as a container image that HUD hosts. The API calls the catalog of those images the registry, and one entry is one environment: its name, its owner, and the work it knows how to run. Reading the registry is how a client discovers what it can run against; creating an entry is what builds do. Each environment declares scenarios, the parameterized task definitions baked into its image. Tasks point at a scenario, and one taskset can contain tasks backed by one or more environments.Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Tasksets
A taskset is a named bundle of tasks and the unit an evaluation is defined against: a public benchmark, your team’s regression suite, a customer’s acceptance set. Two jobs are comparable because they ran the same taskset. Four reads answer different questions. Listing discovers the tasksets your team can access. Fetching a taskset returns the card: name, task count, ownership. Listing its tasks pages through the live task records with their IDs, for follow-up per-task reads. Exporting returns the tasks themselves: each task’s latest version with itsname, env, scenario, args, and settings.
The fields follow POST /v2/tasks/upload, except for the environment and the template: upload takes
them as "env": {"name": ...} and task_id. Convert those two to upload an export, for example to
copy a suite into another taskset.
Creating a taskset makes an empty one owned by your team, in the team’s
Main project unless you pass project_id. Names are normalized (My Eval Set becomes my-eval-set) and unique per team,
including tasksets you cannot access; a taken name returns 409. Uploading with taskset_name also
creates the taskset when it is absent.
Set
public_only=true when listing to browse public tasksets across teams. Without it, the list
contains tasksets owned by or shared with your team. featured_only=true lists the tasksets HUD
features. Listing without credentials requires public_only=true or featured_only=true.
UUID export accepts an API key, a signed-in
session, or no credentials for public tasksets. By-name lookup requires an API key and finds only
tasksets of your team you can access; pass project_id to resolve the name in one project. Fetching a public taskset by ID or listing its tasks does not require credentials.Request
Response 200
Request
Response 201
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Tasks
A task is one graded request inside an environment plus the check that decides whether the agent succeeded. Tasks are what a job actually runs. Most of this group exists because tasks are written and reviewed by people before anyone trusts them to grade a model. A task may carry a brief - the prompt and any attachments the idea was captured from - alongside the runnable scenario. Every task carries a status, the review state everyone reads.
While a task is
pending, its taskset can define stages: an optional checklist inside that one
status, such as write, then QA, then polish. A task advances one stage at a time, or jumps back
when someone returns it for more work, and it must reach the last stage before it can become
ready. Tasksets with no stages configured ignore this machinery entirely.
A ready-check evaluates the taskset’s submission requirements against a task: enough graded
traces, accuracy thresholds, particular models, QA checks. It reports each requirement as passed or
failed along with what is missing. It reports; it does not change the status.
Comments and activity are the review trail: human discussion on one side, and an audit
timeline of version and status changes on the other.
Authoring and review state
Request
Response 201
Request
Response 200
Request
Response 200
Request
Response 200
Comments and activity
Request
Response 200
Request
Response 200
Request
Response 200
Request
204 with an empty body.Request
Response 200
Versions and traces
Tasks are versioned. Changing what a task runs - its scenario, arguments, prompt, or grading - creates a new version, and the task’slatest_version_id points at the newest one. Every trace
records the version it ran as task_version_id. Reading a version by ID returns the task alongside
exactly what that version runs: scenario and arguments, validation, the build_id it targets, and
its environment. An environment imported from another system, such as Harbor, carries an
origin.
Listing a task’s traces pages through runs of every version, newest first. Filter by
task_version_id or model_id, or set exclude_training=true to leave out training jobs.
Reading a version requires an API key or a signed-in session. Listing traces also accepts no
credentials; callers without access to the task receive traces from public jobs only.
Request
Response 200
Request
Response 200
Imports
An import brings a bundle made outside HUD into a taskset you can edit. The only format isharbor: a zip of Harbor task folders, which become tasks with one environment each, and job
folders, which become jobs linked to those tasks. Re-importing a bundle is safe; unchanged items
are left alone. Harbor import covers the bundle layout, what an import
creates, and how to read its report.
GET /v2/imports/formatslists the formats on offer and the largest bundle each takes.POST /v2/imports/uploadsreserves a data file for the bundle and returns a presigned PUT. Upload the bytes, then publish the file withPOST /v2/data/{file_id}/complete.POST /v2/importsstarts importing a ready data file into a taskset and returns the import.- Poll
GET /v2/imports/{import_id}until itsstatusis no longerqueued,starting,running, orcancelling. GET /v2/imports/{import_id}/reportreturns what became of every task, job, and run in the bundle, and why, once the import iscompletedorstopped.
completed delivered what it could; its report lists items it skipped or
refused. One that is stopped was refused or stopped partway, and its summary and report say
why. A failed import did not run to an outcome, and error says why. Team admins see every
import on the team; other members see the imports they started.
Request
Response 200
Request
Response 201
Request
Response 200
Request
Response 201
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Runs and evals
What happens once you actually run something. Rollouts start the work, a job groups it, a trace records each attempt, and an instance is the sandbox an attempt is running in right now.Jobs
A job is one batch of work: this taskset, these models, this many attempts, grouped so the results can be read as a single score. Every graded attempt belongs to a job. Jobs come into existence when work is launched through rollouts, when the SDK enters a job as it starts running, or when runs done outside HUD are imported: in a Harbor bundle through imports, or through the job import calls below. Importing a job run elsewhere links each run to a task by thecontent_digest of the build whose
source defines the task, and you drive the job to completion yourself:
POST /v2/jobs/importnames the taskset, the job’sexternal_id, and up to 1,000 runs to add. Theexternal_idis<system>:<id>: the system the job ran in (currently onlyharbor) and the job’s id there. Job, job list, and trace responses report it asimported_from. The first call creates the job; each run becomes a running trace. Every run’s digest must match a task in the taskset, or nothing is created. A run’sname,agent, andmodellabel its trace;modelis kept as reported and also links the HUD model with that id ormodel_name.POST /v2/jobs/import/{job_id}/traces/{trace_id}/trajectorystores the run’s ATIF trajectory as the trace’s spans; a trace keeps the first trajectory it receives. Callers that already have HUD spans upload them toPOST /v2/telemetry/trace/{trace_id}/telemetry-uploadinstead.POST /v2/jobs/import/{job_id}/traces/{trace_id}/finishrecords the run’s status, reward, and evaluation result.POST /v2/jobs/import/{job_id}/finishsettles the job once all of its traces are finished.
Core resource
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Aggregates
Rollups over the job’s attempts, so a client does not have to fetch every trace and add the numbers up itself.Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Attempts in the job
Two views of the same attempts. Traces are the full records, suited to scripts. Thumbnails are the compact tiles the platform’s grid renders, with counts per outcome attached.Request
Response 200
Request
Response 200
Request
Response 200
List projections
Shapes built for dashboards: the jobs still running, and job cards with their scores already rolled up so a list of jobs renders in one round trip.Request
Response 200
Request
Response 200
Request
Response 200
State
Publishing makes a job readable without a key. Invalidating marks every attempt in it as not counting toward scores, which is how a run spoiled by a broken environment or a bad prompt is taken out of the record without deleting it.Request
Response 200
Request
Response 200
Request
Response 200
Rollouts
A rollout is one attempt placed on hosted infrastructure: pick up a task, start a sandbox, run the agent, record the result. This group is the go and stop switch for that work, and nothing else. Reading results happens under jobs and traces. Launching returns immediately with ajob_id and a trace_id. The work itself is queued, so a
client polls the trace, or the job, until it finishes.
The three launch endpoints differ in what they are given. Running one task needs a task version and
a model. A batch takes lists of both and groups the result into a single job. Submitting is the
path the SDK uses when it has already minted its own identifiers and describes the work by
environment and task name instead.
Launch
These routes run the rollout on HUD’s infrastructure. A model whose hosted rollouts an administrator has disabled is rejected with400; it stays available for inference and
for rollouts the SDK runs and reports itself.
Request
Response 201
Request
Response 201
Request
Response 201
Cancel
Cancelling is scoped: one attempt, one job, or everything you have running.Request
Response 200
Request
Response 200
Request
Response 200
Traces
A trace is one attempt at one task: what the agent saw, every tool call it made, what the environment said back, how long it took, what it cost, and the reward it ended with. Traces are the evidence behind every score on the platform. The record itself is small. The interesting read is events, a flat and typed projection of the attempt’s telemetry into messages, tool calls, and results, ordered by a sequence number. Event reads are bounded pages: whilehas_more is true, pass next_seq back as since_seq to continue.
latest_seq is the trace’s current high-water mark; once a reader has caught up, passing that value
as since_seq returns only what is new, which is how a viewer follows a run that is still going.
Most reads use optional authentication, so a trace inside a published job is readable without a
key. Writes and the analysis bundle require a key or a session.
Core resource
Request
Response 200
Request
Response 200
Request
Response 200
Telemetry and logs
Three levels of detail: the projected trajectory, the runner’s own logs, and the raw output of the environment container. The analysis bundle packages all of it for an agent that has been asked why a run failed.Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Related traces and validity
Request
Response 200
Request
Response 200
Instances
An instance is a sandbox that is live right now, running one attempt of one environment. Each one holds real compute, so instances are the surface you reach for when a run has hung, when something is stuck holding capacity, or when you want to see what is currently occupied.Request
Response 200
Request
Response 200
Secondary
Supporting catalogs behind the core loop: how an environment image gets made, which models you can point at it, and how the work is reviewed.Builds
A build turns environment source into the container image that runs attempts. The shape of it is the same as shipping any image: ask where to put the source, upload it, start the build, watch it, then decide whether the result becomes the version new runs use. Uploading is a two-step handshake. Requesting an upload URL returns abuild_id and a one-time
link; your client sends the source archive to that link directly, then triggers the build by ID.
Marking a build as latest is a separate call, which leaves earlier builds in place to roll back to.
Registering the uploaded source instead of triggering it records an UPLOADED build that is not
built yet. Tasks can be uploaded against it by build_id right away; they cannot run until
triggering the same build_id builds it and binds each task to the build’s task of the same name.
A task uploaded with a build_id and no scenario_name is the task that build’s source defines,
so uploading it requires reading that source; when a later registered source becomes the
environment’s next build, each such task in a taskset the caller may edit gets a new version on it.
Registering can take a content_digest (sha256: and 64 hex digits) that identifies the
uploaded source; HUD stores it as given and does not read the archive to check it. A Harbor import
sends the task folder’s Harbor content hash. Registering a digest the environment already has
returns that build and discards the new upload. The response reports outcome: "created" when it
registered a new build and outcome: "existing" when it reused one. POST /v2/builds/lookup
finds builds by content_digest, for example to link imported runs to tasks registered earlier.
Registering with external_id (<system>:<id>, currently only harbor) and a name instead of
a registry ID targets the team’s environment imported from that system under that id, created on
first use. Environment, build lookup, and task responses report it as origin.
A source in a Harbor environment is a Harbor task folder. Triggering it refuses runtime_config
and a runtime_provider other than modal: building it adapts the folder into an environment
with a Compose project and builds that on Modal, and the build’s source stays the folder. Once it
is built, the tasks its source defines run the adapted task, and a task uploaded with its
build_id and no scenario_name gets that task, so it cannot set args, runtime_config,
verifier, or agent_config itself. Each task version in a Harbor environment runs the build
its task came from, even after a newer folder becomes the environment’s latest build.
Lifecycle
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Source and workspace
After a build indexes its context, its files can be browsed: the source that was uploaded, and the workspace the build produced. The platform uses this to answer “what did we actually ship in this image?” without a rebuild.Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
200 with a file body.Models
A model is an entry in the catalog you point evaluations at. Two kinds share the resource: base models the platform provides, and trainable models your team owns. A team model is created by forking a trainable base, which gives it its own line of checkpoints, one of which is the head that inference uses. Names are resolved rather than guessed: pass whatever model string you have, such as a slug from a config file, and resolve turns it into a catalog record with an ID. For signed-in callers, listed base models also carrycapacity: how many hosted rollouts the model can run at once across all teams,
and the share in use. Results are the other end of the loop, reporting how the model scored on the
tasksets it has been run against.
Checkpoint weights themselves are exported from the training service rather than from this API:
see downloading checkpoint weights.
Catalog
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Training lineage
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Results and usage
Request
Response 200
Request
Response 200
Request
Response 200
Checks
A check is a registered quality or readiness evaluation you can run against a resource. The catalog lists what is available; a run records one attempt against a subject you can see.Request
Response 200
Request
Response 200
Request
Response 201
Request
Response 200
Request
Response 200
QA checks
A QA check is a platform-run review of one kind of subject. Trace checks review completed evaluation traces: failure analysis, false positive and false negative analysis, reward hacking, and prompt alignment. Task checks review a task’s current version and its completed rollouts: task reward exploits and task failure analysis. Running a check analyzes each subject and records apassed or failed verdict with its findings, or an error when the run could not complete or
its final result was missing or invalid. A
trace whose reward already decides the check (full reward for failure analysis and false
negatives, zero reward for false positives) passes without analysis, as a skipped result.
Listing returns each available check’s key, the subject_type it judges (trace or task),
its title, the question it answers, and a description of what it catches, in the order the
platform presents them. See QA agents for what each check covers.
A run takes check_keys and exactly one of trace_ids or task_ids, and runs every check on
every subject; each check must judge that subject type. It returns immediately with each subject’s
result of each check; poll the results read until each one is completed, error, or
cancelled. Every trace must have completed, or the run is refused with 409 listing each
trace’s status. A task check judges the task’s current version, so a new version starts with no
results. With "overwrite": false, a subject whose result of a check is queued, running,
cancelling, or completed keeps it, and only the others run. The results one run launches share a
run_id; skipped and manual results have none.
The results read takes trace_ids or task_ids and returns, per subject, the result in effect
for every check: the latest one, which for a task is the latest on its current version. Each
result names its subject with subject_type and either subject_trace_id, or subject_task_id
and subject_task_version_id. With include_history=true it returns every result instead, newest
first, across all of a task’s versions. Reading a task’s results needs access to the task’s team
workspace, even when the task is public. A manual result is a HUD reviewer’s verdict over the
findings it carries. result follows the versioned qa_agent_result.v1 shape; a result recorded
before that shape existed carries its raw output as legacy_result instead, with no verdict.
A check fails on a confirmed problem and otherwise passes. When one of its steps did not finish,
the result keeps its verdict, result.execution_error names the stage and a reason, and
error repeats the reason. Older results can have an unknown verdict.
Request
Response 200
Request
Response 200
Request
Response 200
QA agents
A QA agent is a saved model-based reviewer: one analysis template, one model, and the arguments it always runs with. Running it against a subject launches an analysis rollout that inspects a frozen snapshot of that subject and records a structured verdict. Environment, taskset, and task agents review those resources; completed traces and tasks are reviewed by QA checks, which the QA agents guide covers. HUD provides these agents. The API finds them, runs them, and reads what they concluded. Listing returns your team’s agents plus public ones for onesubject_type; a public agent owned by another
team shows its template and model but not its saved arguments.
Every route in this group is deprecated, with no sunset date set yet. Use
QA checks to review traces and tasks. Environment and taskset review is being
retired.
Agents
Request
Response 200
Request
Response 200
Run
A resource agent reviews the environments, tasksets, or tasks named bysubject_ids; running any
other agent returns 400. Launching returns immediately with results in queued status; poll the
results read until each one is completed, error, or cancelled.
overwrite defaults to true, which always starts a new review. With overwrite: false, a
resource returns its existing result as long as neither the resource nor the agent has changed
since it ran.
Request
Response 200
Results
A completed result carries the reviewer’s raw output asresult, the same payload as
canonical_result when it follows the standard qa_agent_result.v1 shape, and a normalized
canonical_verdict. A result is stale once the agent’s configuration or the subject itself has
changed since it ran.
Request
Response 200
Request
Response 200
Utilities
Shared plumbing that other groups depend on rather than a resource of its own.Data files
A data file is a tenant-scoped blob the platform stores for you: a dataset, an attachment, a file a task or job later names by id. Reserve a row, PUT the bytes to the returned URL, then complete the upload. Listing is scoped; group listing is the browse path by logical collection.Request
Response 200
Request
Response 200
Request
Response 201
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
204 with an empty body.Request
204 with an empty body.Trace storage
After a run, captured screenshots, recordings, and workspace files live on the trace. These reads return the catalog and contents without going through the live instance.Request
Response 200
Request
Response 200
Request
Response 200
Secrets
Hosted runs frequently need credentials: the site the environment drives wants a login, the agent wants a provider key. Three stores cover that, and they differ in who owns the value.
At launch both are merged into the run’s environment, and an environment variable wins if the same
name exists in both.
Environment variables
Variables are attached to an environment and injected whenever it runs. A run that needs a credential which is not set will usually fail or refuse to start, so the check endpoint exists to answer “is this environment ready?” before anyone launches anything. Variable names, including scenariorequired_env_vars metadata, are normalized to uppercase with
hyphens replaced by underscores; other characters are rejected.
Request
Response 200
Request
Response 201
Request
Response 200
Request
204 with an empty body.Request
Response 200
Request
Response 200
Member secrets
Member secrets are personal values injected into rollout sandboxes you launch. Use them for credentials that your workload needs inside the sandbox. For provider keys used by the inference gateway, use BYOK credentials. Member-secret names follow the same normalization and character rules as registry variables.Request
Response 200
Request
Response 201
Request
Response 200
Request
204 with an empty body.BYOK credentials
Bring-your-own-key credentials are provider keys the platform stores and uses for inference billed to your account. Values are never returned in full after create; listing shows a mask.Request
Response 200
Request
Response 201
Request
Response 200
Request
204 with an empty body.Account
What a run costs, and what stops it from costing more.Limits
A limit is a spending ceiling in US dollars over a week or a month, attached to a team, one member across all of their keys, or a single API key. A limit set to alert sends mail when it is passed; a limit set to block refuses new work instead. Listing limits also returns the current period’s spend, which is what makes it useful as a read even when nothing is near the ceiling. Scope and interval are fixed once a limit exists, so changing them means deleting the limit and creating a new one. There is no fetch-one route; listing is the read path.Request
Response 200
Request
Response 201
Request
Response 200
Request
204 with an empty body.Request
Response 200
Completions
One chat completion, proxied through the platform’s inference gateway and billed like any other inference. No environment, no task, no grading: messages in, text out. This route is deprecated. Send chat completions straight to the inference gateway’s OpenAI-compatiblehttps://inference.hud.ai/v1/chat/completions with the same HUD API
key.
Request
Response 200
Usage
Spend has two sources: the sandboxes you run, charged by the time they are alive, and the model calls made inside them, charged by tokens. Both are readable as history. Inference is reported at two grains. Bucketed aggregates for charts live at/v2/usage/inference. The row-level log of individual calls lives at /v2/inference/usage.
Calls made by HUD QA agents are reported with qa_analysis: true and the model HUD QA rather
than the model they ran on, and are left out of results filtered by model_id.
Request
Response 200
Request
Response 200
Request
Response 200
Team
Who is on your team, who is invited, and what each person may do. Admins and members can read the roster, and external members see only themselves; inviting, changing roles, and removing people require an admin’s API key. The platform’s default key, which hosted rollouts carry, is refused here. A member’s role isadmin, member, or external. Admins manage the team; members see the
team’s resources; external members see only their own work and what is shared with them. You cannot change or remove
your own membership, and a change that would leave the team without an admin is refused.
The write endpoints take a list of up to 30 items and apply each one separately. The response is
200 with succeeded listing the items applied and failed explaining the rest. Each failure
carries the same error envelope as a failed request (see Errors):
Removing a member also revokes their API keys immediately. A
502 means the identity provider
could not be reached for the request as a whole; list the team before retrying.
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Request
Response 200
Sign in with HUD
On the Enterprise plan, a partner application can let its users sign in with their HUD account and call this API on their behalf, without handling API keys. HUD registers the application against the partner’s team and provides its OAuth client credentials; contact HUD to set one up. Each registered application belongs to exactly one team. Sign in with HUD walks through the sign-in flow; this section is the API reference. After a user authorizes the application, send their OAuth access token (it starts withoat_) as the
bearer token, exactly like an API key. The token acts as the user’s membership in the application’s
team: it has that member’s access and shares the same rate limit as their API keys.
Revoking the token takes effect within a minute. An address that sends many invalid or revoked tokens
gets 429 for new tokens until a minute passes.
Request the scopes the application needs in its authorization request; the user sees them on the
consent screen:
With
hud:api, OAuth access tokens reach the endpoints for what the platform and hosted runs do:
Core (environments, tasksets, tasks, imports), Runs and evals except
instances, Secondary, Utilities, limits and usage under
Account, and listing members under Team. A token without the scope an endpoint
requires gets 403 with error insufficient_scope. Every other endpoint, including
Secrets, completions, and instances, answers 403. Users who call HUD from the SDK or
CLI create their own API keys in Settings → API keys.
A user who is not yet a member of the team gets 401 with error principal_not_found. Call
POST /v2/oauth/join once with their token to add them, then retry. New members join with the
external role, which a team admin can change under Team; current members keep their role.
A user a team admin removed, or whose account or the team is banned or deleted, gets 403 with
error membership_unavailable, from POST /v2/oauth/join too. A removed user must be invited
again by an admin.
Request
Response 200
Errors
Failures share one envelope, with a machine-readable code and a human-readable message. The status tells you whether to fix the request, the credentials, or the state.Deprecations
A deprecated route or response field still works, but is on its way out: move off it before its sunset date, after which it can be removed without further notice. Until then it stays in the OpenAPI document, flaggeddeprecated. Every successful
response from a deprecated route also carries these headers:
QA agent routes
Every/v2/qa-agents route is deprecated. QA checks review traces and tasks.
Environment and taskset review is being retired, with no replacement.
The last two review traces. They are past their sunset, are no longer in the OpenAPI document,
and are still served only for older
hud CLIs.
Completions route
Response fields
For how the objects fit together beyond the API, see the
platform introduction. For a console that sends real requests, use the
interactive reference.