Hermes Embodied

What Hermes Embodied does

Hermes Embodied turns Hermes Agent into a trainer for robot policies, driven by natural language. It adds three skills: vast-gpu provisions, monitors and tears down GPU instances on Vast.ai; vla-trainer runs an end-to-end fine-tuning pipeline for Vision-Language-Action models, supporting SmolVLA (450M) and GR00T N1.5 (3B) on LeRobot data with WandB monitoring; robot-loop runs the autonomous improvement cycle.

The loop deploys a VLA checkpoint in simulation or on hardware, records each rollout as a LeRobot trajectory, keeps the trajectories whose reward clears a threshold, fine-tunes on a rented GPU, runs open-loop evaluation against the old checkpoint and promotes the new one if it is better. Hermes cron schedules the repeats. Simulation uses MuJoCo with LeRobot gym_hil Franka Panda tasks, and the optional physical setup is an SO-ARM101 leader and follower arm with USB cameras. It was built for the Nous Research Hermes Agent Hackathon.

Key features

• vast-gpu skill for provisioning, monitoring and tearing down Vast.ai GPU instances
• vla-trainer skill for SmolVLA and GR00T N1.5 fine-tuning with data prep and stats validation
• robot-loop skill that collects trajectories, retrains and A/B tests new checkpoints
• MuJoCo and LeRobot gym_hil simulation with Franka Panda tasks
• Optional SO-ARM101 hardware support with USB cameras
• Repeat runs scheduled through Hermes cron

When to use it

• Fine-tuning a robot policy by describing the goal to Hermes in plain language
• Running a scheduled collect, retrain and evaluate cycle in simulation
• Spinning up and tearing down a cloud GPU for a training job through the agent

Who it is for: Robotics hobbyists and researchers who want Hermes Agent to orchestrate VLA training and evaluation.

How it fits with Hermes Agent

Built on Hermes Agent as three skills, with Hermes acting as the reasoning layer that plans, monitors and schedules the loop through its cron scheduler.

Requirements: Optional for physical use: SO-ARM101 leader and follower arms, USB cameras and a Linux machine with USB ports

Note: It is a hackathon project that trains on rented Vast.ai GPUs, and the repository has no license file.

FAQ