Agent Charm

What Agent Charm does

You hold the button and speak. A small Python server transcribes the audio with faster-whisper, sends it to your agent, then replies with a short card on screen and two spoken sentences using Edge TTS. Answers stream, so Dex starts speaking after the first sentence. The device talks to the server over a single WebSocket that carries JSON frames and 16 kHz PCM audio, with a JSON Schema for every card kind, and a turn-based HTTP API exists for clients that cannot hold a WebSocket.

The screen only shows states the backend has confirmed. Cards ask for approve, reject or later, with a stated default and deadline, and payments need a 2-second hold rather than a tap. Other modes include a reading mode with no spoilers past your chapter, verbatim 'Save this' notes stored as Markdown files, a six-card morning edition built from agent feeds and a second character, Coach Beard. The same LVGL UI builds into a macOS simulator, so it runs without hardware. Voice input supports English and Spanish.

Key features

• Hold-to-talk voice interface in English or Spanish with streamed answers
• Hermes Agent or any OpenAI-compatible backend, including Ollama and LM Studio
• Decision cards with approve, reject and later, and a 2-second hold for payments
• Reading mode that avoids spoilers past your current chapter
• macOS simulator built from the same LVGL UI code
• Documented WebSocket protocol and HTTP API

When to use it

• Asking a Hermes Agent a question by voice from a pocket device
• Approving small agent actions with a physical hold
• Prototyping the UI on a Mac with the simulator before buying hardware

Who it is for: Hardware tinkerers and agent users who want a physical voice remote for an agent they already trust.

How it fits with Hermes Agent

Hermes Agent is one of two agent backends, selectable as hermes next to a generic OpenAI-compatible backend, and the repository carries the hermes-agent topic.

Requirements: A Waveshare ESP32-S3-Touch-AMOLED-1.8 board (V2 only), a USB-C cable and a 5 V 2 A supply; the server and simulator run on a Mac or Linux box, and the macOS simulator needs uv, cmake, ffmpeg and SDL2

FAQ