> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hud.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Robotics

> Evaluate robot policies and tool-calling agents in simulation, and replay episodes with camera video and state and action plots.

For teams evaluating robot policies, such as vision-language-action models, or LLM agents that
control a robot. A robot environment wraps a simulator and describes its cameras, state, and actions
in a contract. Each episode becomes a trace you can replay: every camera as video, with the robot's
state and actions plotted over time.

<Note>
  Robot support in the SDK is in beta. See the [robots guide](/v6/advanced/robots).
</Note>

## Two ways to control a robot

| | Policies | Tool-calling agents |
| - | - | - |
| **What drives the robot** | A policy, such as pi0.5 through [LeRobot](https://github.com/huggingface/lerobot), sending an action every control tick. | An LLM calling motion tools, such as moving the end effector to a target. |
| **Where it runs** | Your process and GPU, started from Python with the SDK. The simulator runs in its own container. | Any tool-calling model, from the web app or with `hud eval`. |
| **Start from** | The [robot benchmark cookbook](/v6/cookbooks/robot-benchmark), which runs pi0.5 on LIBERO. | [Direct control](/v6/advanced/robots#direct-control), which serves motion tools from the same contract, so one environment serves both. |

The featured **robot-gripper** environment is a quick way to try tool-calling control from the web
app: the agent drives a simulated gripper on a tabletop through tools, in the `open-drawer`,
`pick-object`, `move-object`, and `force-grasp` templates, and each is graded from the simulator's
state. It serves the simulator as a tool server rather than through a contract, so its traces show
the tool calls and the camera images the agent requests with `render`, not the
[episode replay](#replay-episodes).

## Build a robot environment

Start from the **Robotics** template, for MuJoCo and LIBERO, or **Worldsim Robotics**, for Newton
physics: on [Environments](https://hud.ai/environments), select **New Environment**, then
**See all templates**.
You implement a bridge around your simulator with `reset`, `step`, and `get_observation`, and
describe the embodiment in its contract; the SDK handles the connection and sends telemetry to HUD.
See the [robots guide](/v6/advanced/robots#environment-side).

Set the contract's frame rate and per-dimension names. The trace viewer uses them to play video at
real speed and to label each state and action plot.

## Replay episodes

Environments built on the SDK's robot support stream every control tick to the trace. Its
**Robotics** view plays the episode back:

* **Cameras** shows each camera as video, with play, pause, and playback speed.
* **Plots** charts each action and observation dimension over time, named from the contract.
* **Split** shows the cameras and the plots together.

Each template grades its episode, for example with the simulator's own success check, so a trace's
reward says whether the episode succeeded. Run each task with several seeds or repetitions to
measure a success rate. See [traces](/platform/traces#inspect-a-trace).

<Columns cols={3}>
  <Card title="Robots guide" icon="robot" href="/v6/advanced/robots">
    Bridges, contracts, policies, and batching.
  </Card>

  <Card title="Robot benchmark" icon="flask" href="/v6/cookbooks/robot-benchmark">
    Run pi0.5 on LIBERO end to end.
  </Card>

  <Card title="Evaluations" icon="https://mintcdn.com/hud-f5fd7c15/9ozVUWR-rlBrAGzN/platform/icons/job.svg?fit=max&auto=format&n=9ozVUWR-rlBrAGzN&q=85&s=84aaf059e6b0613cc306b27e8b63b7ca" href="/platform/evaluations" width="24" height="24" data-path="platform/icons/job.svg">
    Run tasks across models and read the results.
  </Card>
</Columns>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.