Searcharvester: automation for Hermes Agent

Self-hosted search, page extraction and deep research API for AI agents, run with Docker Compose

What Searcharvester does

One docker compose up starts three endpoints. /search is Tavily-compatible and runs on SearXNG with more than 100 engines, including images and videos. /extract turns a URL into clean markdown using trafilatura, readability or Defuddle, with size presets and pagination. /research starts a team of research agents that answers a question with a markdown report and citations. Search needs no API keys or quotas, while /research needs credentials for any OpenAI-compatible endpoint.

A React web UI on port 9762 lets you start jobs, choose a model and reasoning mode per agent role, follow each agent live, read the report and browse sources and media. A settings page manages SearXNG engines, proxies and timeouts. Jobs run for up to 3,600 seconds by default, with at most 16 running at once, and finished jobs stay on disk across restarts. The research API includes a route for the raw Hermes log of a job.

Key features

Tavily-compatible /search endpoint backed by SearXNG with 100+ engines
/extract endpoint with trafilatura, readability and Defuddle extractors, size presets and pagination
/research endpoint that returns a markdown report with citations
Web UI on port 9762 to follow each agent live and browse sources and media
Search settings page for SearXNG engines, proxies and timeouts

When to use it

Giving an agent web search and page reading without paid search API keys
Running a deep research job and reading the cited report
Watching which pages and searches each research sub-agent used

Who it is for: Developers who want a self-hosted search, extraction and research backend for AI agents.

How it fits with Hermes Agent

The research API exposes the raw Hermes log of each job through GET /research/{id}/logs, so Hermes is part of how research jobs run. The search and extract endpoints are plain HTTP tools that any agent can call.

How to install Searcharvester

These commands are copied from the project's README. Check the repository for the latest steps before you run them.

git clone https://github.com/vakovalskii/searcharvester.git
cd searcharvester
cp config.example.yaml config.yaml
docker compose up

Requirements: Docker Compose, a server.secret_key of 32+ characters in config.yaml, and an OpenAI-compatible LLM endpoint for /research

FAQ

What is Searcharvester?

Searcharvester is a self-hosted search, extract and deep research service for AI agents. It combines SearXNG, FastAPI and trafilatura behind a Tavily-compatible API and a web UI.

Does Searcharvester work with Hermes Agent?

Hermes is involved in its research jobs, since the API has a route that returns the raw Hermes log for each job. Its search and extract endpoints are ordinary HTTP APIs that other agents can call too.

What do I need to run Searcharvester?

You need Docker Compose, a secret key of 32 or more characters in config.yaml, and credentials for an OpenAI-compatible LLM endpoint if you want to use /research. Search itself needs no API keys.