OpenVision

What OpenVision does

You speak to the glasses, for example 'Ok Vision, what am I looking at?', and the app routes the request to the backend you chose. The seven backends are local MLX models, Apple Intelligence, OpenClaw, Gemini Live, OpenAI, Grok and Hermes. A live video mode keeps the glasses camera on so you can ask questions about what you see until you say stop video, and it routes to the selected backend, for example Gemini Live or the OpenAI Realtime API.

With the Hermes backend, the app reaches your own Hermes Agent server over the internet, using its API key or by signing in to its web UI with a username and password. Hermes answers with its own tools, memory and skills, sees photos from the glasses, asks before running risky commands so you answer yes or no, and runs its slash commands by voice. The README notes that the username and password option relies on the Hermes Desktop app's protocol, which is not a public API and can change between Hermes versions, so the API key is the stable choice.

Key features

• Seven AI backends, from on-device MLX models to cloud services
• Hermes backend using your own Hermes Agent server
• Hermes tools, memory and skills available through the glasses
• Yes or no approval prompts for risky Hermes commands
• Hermes slash commands by voice
• Live video mode with continuous camera frames

When to use it

• Ask a Hermes agent what you are looking at while your hands are busy
• Translate a menu or sign by pointing the glasses at it
• Run a fully on-device assistant with no API cost using a local model

Who it is for: Meta Ray-Ban glasses owners and iOS developers who want a hands-free front end for AI assistants, including a self-hosted Hermes Agent.

How it fits with Hermes Agent

Hermes is one of seven backends; the app connects to a Hermes Agent server you run yourself.

Requirements: For the Hermes backend, your own Hermes Agent server reachable over the internet, with its API key or web UI username and password.

Note: The username and password sign-in for Hermes uses a protocol that is not a public API and can change between Hermes versions.

FAQ