HUD Documentation - Evaluations and RL Environments.

HUD’s coding example is a complete repository environment: the agent works in a sandboxed ssh workspace while the environment keeps a clean baseline outside it, then applies the authored tests after the agent finishes. hud init --template coding copies the project as a starting point.

Create the environment

hud init my-coding-env --template coding
cd my-coding-env
uv sync

The generated project contains:

Path	Purpose
`env.py`	Repository setup, workspace, and the `coding-task` template.
`tasks.py`	Concrete task rows, test patches, commands, and expected results.
`grader.py`	Example-local `BashGrader` extension for selected JUnit tests.
`flask.bundle`	Prepared repository baselines and reference fixes for the example tasks.
`Dockerfile.hud`	Reproducible dependencies and the packaged repository workspace.

The included flask-4992 and flask-5063 tasks are from SWE-bench Lite. Run them with any coding agent that opens the environment’s shell capability:

hud set HUD_API_KEY=your-key-here
hud eval tasks.py claude --task-ids flask-4992,flask-5063 --runtime local -y

Local non-root runs require usable Linux bwrap isolation. Otherwise, build the packaged environment and attach to its control channel:

docker build -f Dockerfile.hud -t my-coding-env .
HUD_SECCOMP=/path/to/site-packages/hud/eval/docker-seccomp.json
docker run -d --name my-coding-env -p 8765:8765 \
  --security-opt "seccomp=$HUD_SECCOMP" \
  --security-opt systempaths=unconfined \
  --security-opt apparmor=unconfined \
  my-coding-env
hud eval tasks.py claude --task-ids flask-4992,flask-5063 --runtime tcp://127.0.0.1:8765 --max-concurrent 1 -y
docker rm -f my-coding-env

HUD’s seccomp profile and system-path settings allow bubblewrap to create the Workspace namespaces inside Docker without disabling seccomp. The host must allow unprivileged user namespaces; the environment refuses to start if the runtime blocks them.

How the grading boundary works

Task setup clones base_ref into an environment-owned baseline outside the workspace.
The clean worktree is copied into the ssh workspace and initialized as a one-commit repository, so the source history and reference-fix refs are never exposed to the agent.
Grading terminates the agent’s isolated session namespace and discards its Git metadata.
The environment restores test_path to the baseline and applies the hidden patch.
The custom JUnitGrader runs test_command through BashGrader under the workspace’s isolated UID, then scores the selected fail-to-pass and pass-to-pass test IDs.

This keeps the deliverable as repository state rather than a text answer, while preventing the agent from weakening the grader or reading the reference fix.

Define a task

Call coding_task once per repository issue and give the resulting task a stable slug:

tasks.py

from env import coding_task

_fix_parser = coding_task(
    description="Fix the parser without breaking existing inputs.",
    test_command="python -m pytest -q test_parser.py --junitxml={junit_path}",
    test_patch="""diff --git a/test_parser.py b/test_parser.py
--- a/test_parser.py
+++ b/test_parser.py
@@ -1,3 +1,6 @@
 def test_existing_input():
     assert parse("old")
+
+def test_new_input():
+    assert parse("new")
""",
    base_ref="origin/parser_baseline",
    test_path="test_parser.py",
    fail_to_pass=["test_parser.test_new_input"],
    pass_to_pass=["test_parser.test_existing_input"],
)
_fix_parser.slug = "fix-parser"

tasks = [_fix_parser]

test_command runs from the repository root and must write JUnit XML to {junit_path}. The default reward is the fraction of selected tests that pass; set binary=True to require every selected test.

Adapt it to another repository

Replace flask.bundle with a bundle containing the prepared baselines for your tasks.
Update REPO_SOURCE in env.py, or set REPO_URL for a local run.
Install repository dependencies in Dockerfile.hud so grading does not depend on runtime downloads.
Keep reference fixes outside the baseline history exposed to the agent; put hidden test patches in the task rows.
Add task rows in tasks.py; changing rows does not require changing the environment template.

The complete source lives in environments/coding.

Start here

Guides

Reference

Advanced

Experimental

Cookbooks

Internals

More

Coding environment

Create the environment

How the grading boundary works

Define a task

Adapt it to another repository

See also

Capabilities

Graders

Designing tasks for signal

​Create the environment

​How the grading boundary works

​Define a task

​Adapt it to another repository

​See also

Capabilities

Graders

Designing tasks for signal

Create the environment

How the grading boundary works

Define a task

Adapt it to another repository

See also