HUD Documentation - Evaluations and RL Environments.

A runtime chooses where each rollout’s environment runs. You pass it to task.run / taskset.run at execution time, and the same task and the same env.py run anywhere - only the runtime changes.

from hud import LocalRuntime

await taskset.run(agent, runtime=LocalRuntime("env.py"))   # serve env.py locally, run here

Built-in runtimes

Runtime	Where the env runs	When to reach for it
`LocalRuntime("env.py")`	This process, loading your source per rollout	Fastest iteration; local development
`SubprocessRuntime("env.py")`	A child process from your source	Local runs isolated from your process
`DockerRuntime("my-env")`	A fresh local container per rollout	Reproducibility and parity with production
`ModalRuntime("my-env")`	A fresh Modal sandbox per rollout	Cloud scale, no infra to manage
`DaytonaRuntime("my-env")`	A fresh Daytona sandbox per rollout	Cloud scale on Daytona
`Runtime("tcp://host:8765")`	A substrate you already started	Attaching to a long-lived container or sandbox you own
`Shared(provider, width=N)`	One substrate from `provider`, shared by up to N concurrent connections	Multi-agent evals - agents that share, compete, or coordinate over one live environment
`HUDRuntime()`	A HUD-hosted env, leased by name and tunneled	Local agent loop against a deployed env
`HostedRuntime()`	The whole rollout managed by HUD	Agent and env execute remotely together

Most runtimes are on the top-level package (from hud import LocalRuntime, DockerRuntime, HUDRuntime, HostedRuntime, Runtime); ModalRuntime and DaytonaRuntime import from hud.eval.

You can usually omit runtime=. A run without one uses its known placement:

a platform taskset (Taskset.from_api) runs on the platform
a row whose runtime_config names an image or Compose project starts it under DockerRuntime
a task created by @env.template runs against that environment, in this process

When none of these apply, run raises and lists the runtimes you can pass — it never silently picks one. For example, a JSON row with no image needs runtime=LocalRuntime(env) or another runtime.

To deploy an environment to the platform and run against it, see running an eval and deploying to the platform.

RuntimeConfig

RuntimeConfig carries the typed construction input a container-based runtime needs: an image or Compose project, hardware, and timeouts. Set it on the runtime (runtime_config=) or per row on Task.runtime_config; the runtime merges the two and applies what it supports.

from hud.eval import ComposeProject, RuntimeConfig, RuntimeGPU, RuntimeLimits, RuntimeResources, RuntimeTPU

RuntimeConfig(
    image="my-env",
    resources=RuntimeResources(
        cpu=4,
        memory_mb=8192,
        storage_mb=32768,
        gpu=RuntimeGPU(type=["H100", "A100"], count=1),
    ),
    limits=RuntimeLimits(startup_timeout_s=300, run_timeout_s=1800),
)

RuntimeConfig(compose=ComposeProject(document="./compose.yaml"))
RuntimeConfig(resources=RuntimeResources(tpu=RuntimeTPU(type="v5", topology="2x2")))

Field	Description
`image`	Image to run.
`compose`	`ComposeProject(document, root, service_access)`. Mutually exclusive with `image`.
`resources`	Placement requests: CPU, memory, disk, acceptable GPU types and count, OS, or TPU slice.
`limits`	`RuntimeLimits(startup_timeout_s, run_timeout_s)`.

Support differs per runtime. Providers reject unsupported requirements except storage_mb, which is best effort: DockerRuntime admits against available disk and DaytonaRuntime provisions enough whole GiB, while providers without disk sizing proceed with their default capacity. HUDRuntime proceeds on the deployed environment’s defaults and warns when it ignores CPU, memory, or storage requirements; it rejects GPU, OS, TPU, and limits. Daytona accepts a list of GPU alternatives; Modal accepts one GPU type. Docker ignores limits; Daytona rejects run_timeout_s and resource overrides when booting from an already-built snapshot. LocalRuntime rejects a per-task runtime_config. run_timeout_s bounds the lifetime of the provisioned environment, not the complete rollout. The optional rollout_timeout passed to Task.run or Taskset.run is the execution deadline through grading, including submission, queueing, and provisioning. It is not an upper bound on method-return latency: cancellation and trace reporting still run after expiry, and Taskset.run then flushes telemetry. When both are set, run_timeout_s and other known phase limits must be less than rollout_timeout; an agent timeout must also be less than the actor environment’s run_timeout_s. Contradictory values fail before execution. When rollout_timeout is omitted, the SDK adds no overall deadline; configured phase limits and limits imposed by the selected runtime still apply. When the SDK owns the environment process, its output is part of the local eval: SubprocessRuntime, Docker image and Compose runtimes, Modal image and Compose sandboxes, and Daytona sandboxes stream stdout and stderr directly to the terminal without reformatting them. LocalRuntime already shares the eval process’s terminal. HUD-hosted runtimes retain environment output on the hosted trace instead.

Runtime directory

The constructor for each built-in runtime:

`LocalRuntime`

LocalRuntime(source, *, env=None, ready_timeout=120.0)

Runs an environment in the current Python process. source can be:

A .py file or directory: load the environment from that source.
An Environment: reuse that instance.
A (task) -> Environment function: call it for each rollout.

env selects an environment by name when loading a source; it defaults to the task’s environment. ready_timeout limits how long initialization can take. Rollouts using the same environment instance run one at a time, including when a source or function returns a cached instance. Distinct instances can run concurrently. Imported modules share state between runs. Loading a second copy of a package from another directory raises an error. Use SubprocessRuntime for separate Python processes.

`SubprocessRuntime`

SubprocessRuntime(source, *, env=None, ready_timeout=120.0)

Starts a new process for each rollout and stops it afterward.

source - a .py file or directory exporting environments or tasks created by @env.template. The source must import all templates and hooks needed for the run.
env - the environment name to serve. Defaults to the task’s environment.
ready_timeout - seconds to wait for the process to start serving.

You can also pass an Environment whose templates all live in one file; the runtime serves that file. Each process runs from its source directory and shares host files and ports.

`DockerRuntime`

DockerRuntime(image=None, *, port=8765, run_args=(), runtime_config=None)

image - image name to run; shorthand for runtime_config.image.
port - port the image’s CMD serves inside the container (the scaffolded Dockerfile.hud serves 8765).
run_args - extra docker run flags, e.g. ["--gpus", "all"] or ["-e", "KEY=VAL"].
runtime_config - a RuntimeConfig (image or Compose file, resources) for finer control.

For Compose environments, the service named main must serve HUD’s control channel on port. DockerRuntime starts the project and publishes that service’s control port. Constructor-level image is shorthand for runtime_config.image. Project structure, build hooks, service access, and the security override are documented in Compose environments.

`ModalRuntime`

ModalRuntime(image_name=None, *, image=None, command=None, app=None, app_name=None, workdir=None, port=8765, runtime_config=None, env_vars=None, sandbox_secrets=None, registry_secret=None)

image_name - published Modal image name (the preferred durable handle), e.g. ModalRuntime("hud-libero-env").
image - an Image to build lazily on first use, as an escape hatch.
command - override the serving command (defaults to the scaffolded hud serve entrypoint).
workdir - working directory inside the sandbox. Left unset, Modal keeps the image’s WORKDIR.
app - an already-running, caller-owned Modal app. ModalRuntime does not manage its lifecycle.
app_name - a Modal app to look up. Defaults to hud-envs when app is not supplied; app and app_name are mutually exclusive.
port / env_vars - in-sandbox serving port and extra environment variables.
sandbox_secrets - Modal Secret objects to attach to the sandbox. Resolve named secrets with modal.Secret.from_name(...); their values never pass through the local process. Sandbox secrets require an image runtime and are rejected for Docker-in-Docker Compose because attaching them to the outer sandbox would not expose them to main.
registry_secret - optional Modal Secret for pulling a private registry image (REGISTRY_USERNAME / REGISTRY_PASSWORD). Unused for modal:// images and Compose.

Running a private Docker image

import modal
from hud.eval import ModalRuntime
from hud.eval.runtime import RuntimeConfig

ModalRuntime(
    runtime_config=RuntimeConfig(image="ghcr.io/org/private:tag"),
    registry_secret=modal.Secret.from_name("registry-creds"),
    sandbox_secrets=[modal.Secret.from_name("hf-token")],
)

For Compose input, ModalRuntime runs the project in a Docker-in-Docker sandbox in your Modal account and merges env_vars into main at acquisition time. GPU requests require a plain or platform-materialized image; Modal Compose rejects them because its nested Docker daemon cannot receive the sandbox GPU. See Compose environments. Requires the modal extra and a configured token.

`DaytonaRuntime`

DaytonaRuntime(snapshot_name=None, *, image=None, command=None, workdir="/app", port=8765, ssh_host="ssh.app.daytona.io", ssh_expires_minutes=1440, runtime_config=None)

snapshot_name - Daytona snapshot to boot from (the durable handle).
image - Dockerfile/registry ref to build the snapshot if it’s missing. Daytona records what a snapshot was built from, so when the image content changes (an edited Dockerfile or context file, a repointed registry ref) the snapshot is rebuilt in place under the same name instead of silently reusing the build from before the edit.
workdir / port - guest working directory and in-sandbox serving port.
ssh_host / ssh_expires_minutes - SSH tunnel settings (Daytona exposes services over an SSH local-forward).

Resources (cpu/memory/gpu) are fixed on the snapshot at build time. With image, a task’s runtime_config.resources builds a sized variant under a suffixed name (my-env-4cpu); without image, an already-built snapshot cannot be resized. For Compose input, Daytona runs main as the rollout sandbox and each sidecar as a linked sandbox. Sidecar images resolve to reusable Daytona snapshots. Compose authoring conventions are documented in Compose environments. Measured behavior at scale, up to 256 concurrent sandboxes with warm pools, is written up in the Daytona cookbook and the full guide on Daytona’s docs.

`HUDRuntime`

HUDRuntime(*, run_timeout=None, runtime_url=None)

run_timeout - deprecated alias for the rollout_timeout passed to Task.run or Taskset.run.
runtime_url - override the runtime endpoint the tunnel connects to.

The SDK leases your deployed env by name and tunnels to its control channel; the agent loop runs local.

`HostedRuntime`

HostedRuntime(*, poll_interval=5.0, run_timeout=None)

poll_interval - seconds between trace-status polls while the rollout runs remotely.
run_timeout - deprecated alias for the rollout_timeout passed to Task.run or Taskset.run.

For example, a task with RuntimeLimits(run_timeout_s=18_000) can omit rollout_timeout to avoid adding an SDK-wide cap. To add one, pass a larger value such as task.run(agent, runtime=HostedRuntime(), rollout_timeout=18_600), leaving time for queueing and provisioning. Where HUDRuntime runs the agent loop locally against a tunneled env, HostedRuntime runs the whole rollout remotely, with the agent running alongside the task environment. This process only submits the rollout and polls its trace to completion. It supports gateway agents from create_agent; agents with a custom model_client must use HUDRuntime or LocalRuntime.

`Runtime`

Runtime(url, params=..., config=...)

url - control-channel address of an already-running substrate (e.g. tcp://host:8765).
params - connection-time data a transport may need (auth token, sandbox id).

Provisioned runtimes expose snapshot_session(session_id) and restore_session(session_id, archive) for transferring session files to an independent verifier. The base runtime is a no-op; providers such as Docker and Modal implement the transfer directly on their provisioned endpoint.

`Shared`

Shared(provider, *, width)

provider - the Provider to provision once, e.g. DockerRuntime("my-env").
width - how many concurrent connections that one substrate accepts.

A lease pool over one substrate: it boots through provider on the first lease and lives for the enclosing async with scope, so every rollout in that scope is handed the same address - agents that need to share, compete, or coordinate in the same environment (a vectorized sim’s num_envs slots, see robots, are one case). Lease width + 1 waits for a slot to free instead of erroring, so group and max_concurrent keep their ordinary meanings. Taskset.run scopes a context-manager placement to the call, so a bare runtime=Shared(...) boots once per run(...); open the scope yourself to keep the substrate warm across several calls:

async with Shared(DockerRuntime("hud-isaac-env"), width=8) as rt:
    await taskset.run(agent, runtime=rt, group=8)
    await taskset.run(agent, runtime=rt, group=8)  # same live sim

Runtime(url) needs no wrapping since every caller already dials the same address.

Run on your own infra

A runtime is just a function: given a task, start a container somewhere and yield its control-channel URL. That one function is the whole integration surface for any provider - Modal, E2B, Runloop, your own Kubernetes:

run.py

from contextlib import asynccontextmanager
from hud import Runtime

@asynccontextmanager
async def my_runtime(task):
    sandbox = await start_my_sandbox(image="my-env")   # your infra brings it up
    try:
        yield Runtime(f"tcp://{sandbox.host}:{sandbox.port}")
    finally:
        await sandbox.terminate()                       # ...and tears it down

await taskset.run(agent, runtime=my_runtime)

DockerRuntime and the rest are just built-in versions of this. Anything that starts your image and hands back a URL plugs in with no change to the environment or the task - that’s what “run anywhere” means concretely. Constructed directly, Runtime(url) yields itself with a no-op lifecycle, since whoever provisioned the substrate owns teardown. Placement can also vary per task: a runtime is called once per rollout with the task row being placed, so one callable can route heavier rows to heavier substrates.

Start here

Guides

Reference

Advanced

Experimental

Cookbooks

Internals

More

Runtimes

Built-in runtimes

RuntimeConfig

Runtime directory

`LocalRuntime`

`SubprocessRuntime`

`DockerRuntime`

`ModalRuntime`

`DaytonaRuntime`

`HUDRuntime`

`HostedRuntime`

`Runtime`

`Shared`

Run on your own infra

​Built-in runtimes

​RuntimeConfig

​Runtime directory

​LocalRuntime

​SubprocessRuntime

​DockerRuntime

​ModalRuntime

​DaytonaRuntime

​HUDRuntime

​HostedRuntime

​Runtime

​Shared

​Run on your own infra

Built-in runtimes

RuntimeConfig

Runtime directory

`LocalRuntime`

`SubprocessRuntime`

`DockerRuntime`

`ModalRuntime`

`DaytonaRuntime`

`HUDRuntime`

`HostedRuntime`

`Runtime`

`Shared`

Run on your own infra