J.A.R.V.I.S Voice + HUD

What J.A.R.V.I.S Voice + HUD does

A voice pipeline server runs on your own machine. Speech-to-text uses faster-whisper locally on CPU, and text-to-speech uses ElevenLabs Flash, streamed sentence by sentence so audio starts before the full reply is finished. The server talks to the Hermes Agent API on port 8642, and voice and typed chat share one persistent Hermes session, so memory survives restarts. The README gives a typical round trip of 3 to 5 seconds.

The HUD shows live agent activity with command previews, a stop button, and approval cards that pause dangerous commands for allow or deny. It embeds the Hermes kanban board and session browser, and a bundled Hermes plugin named hud_display lets the agent put video and media panels on screen. A privacy filter redacts secret-shaped strings before text reaches cloud TTS, and an optional sidecar on an NVIDIA machine on your LAN speeds up transcription, with fallback to local Whisper when that machine is off.

Key features

• Live transcription while you speak, with replies streamed back as speech
• Live view of agent tool calls and a stop button for runaway turns
• Approval cards for dangerous commands
• Embedded Hermes kanban board and session browser
• Bundled hud_display plugin for agent-summoned media panels
• Privacy filter that redacts secret-shaped strings before cloud TTS

When to use it

• Talk to a Hermes Agent hands-free from any browser on your home network
• Watch tool calls and approve or deny risky commands from a HUD
• Use a phone as a full-screen voice client through Add to Home Screen

Who it is for: Hermes Agent users who want a voice front end and dashboard running on their own hardware.

How it fits with Hermes Agent

Built specifically for Hermes Agent: it calls the Hermes API server and ships a bundled Hermes plugin, hud_display.

Requirements: A machine for the server (tested on a Mac mini with Apple Silicon), Hermes Agent installed with an LLM provider, Python 3.11+, an ElevenLabs API key (free tier works), and a modern browser on the LAN.

Note: Spoken replies depend on the ElevenLabs cloud API, and the README says the project was tested on macOS with Apple Silicon, with Linux needing minor changes.

FAQ