Atropos

What Atropos does

Atropos is Nous Research's environment microservice framework for asynchronous reinforcement learning with language models. Environments run as services and send data to a trajectory API, from which a trainer pulls batches. The trainer and inference engine are not included in the package. The framework supports dataset environments such as GSM8K and MMLU, online game environments such as Blackjack and Taxi, RLAIF and RLHF, multi-turn RL, code execution and multimodal tasks.

The README reports results from models trained with its environments, such as a tool-calling specialist whose Berkeley Function Calling score on parallel tasks rose from 10% to 46%, and a financial-fundamentals model whose directional prediction accuracy rose from 20% to 50%. It links DeepHermes model artifacts on Hugging Face, including RLAIF personality experiments. The repository is archived and no longer maintained.

Key features

• Environment microservices with a trajectory API for trainers to pull batches
• Dataset environments such as GSM8K, MMLU and custom Hugging Face datasets
• Online game environments including Blackjack, Taxi and text-based games
• RLAIF, RLHF and multi-turn RL environments
• Code execution (MBPP, HumanEval) and multimodal (OCR VQA, Clevr) environments

When to use it

• Collect LLM trajectories from interactive environments for RL training
• Evaluate a model on dataset or game environments
• Study how the DeepHermes artifacts linked in the README were produced

Who it is for: Researchers studying reinforcement learning environments for language models.

How it fits with Hermes Agent

A Nous Research repository whose README links DeepHermes models trained with its environments; it concerns training and evaluating LLMs rather than running Hermes Agent.

Note: The repository is archived and no longer maintained, with no further updates, bug fixes or security patches.

FAQ