Building a Model Provider Plugin

Model provider plugins declare an inference backend — an OpenAI-compatible endpoint, an Anthropic Messages server, a Codex-style Responses API, or a Bedrock-native surface — that Hermes can route AIAgent calls through. Every built-in provider (OpenRouter, Anthropic, GMI, DeepSeek, Nvidia, …) ships as one of these plugins. Third parties can add their own by dropping a directory under $HERMES_HOME/plugins/model-providers/ with zero changes to the repo.

tip

Model provider plugins are the third kind of provider plugin. The others are Memory Provider Plugins (cross-session knowledge) and Context Engine Plugins (context compression strategies). All three follow the same "drop a directory, declare a profile, no repo edits" pattern.

How discovery works

providers/__init__.py._discover_providers() runs lazily the first time any code calls get_provider_profile() or list_providers(). Discovery order:

1. Bundled plugins — <repo>/plugins/model-providers/<name>/ — ship with Hermes
2. User plugins — $HERMES_HOME/plugins/model-providers/<name>/ — drop in any directory; no restart required for subsequent sessions
3. Legacy single-file — <repo>/providers/<name>.py — back-compat for out-of-tree editable installs

User plugins override bundled plugins of the same name because register_provider() is last-writer-wins. Drop a $HERMES_HOME/plugins/model-providers/gmi/ directory to replace the built-in GMI profile without touching the repo.

Directory structure

plugins/model-providers/my-provider/
├── __init__.py       # Calls register_provider(profile) at module-level
├── plugin.yaml       # kind: model-provider + metadata (optional but recommended)
└── README.md         # Setup instructions (optional)

The only required file is __init__.py. plugin.yaml is used by hermes plugins for introspection and by the general PluginManager to route the plugin to the right loader; without it, the general loader falls back to a source-text heuristic.

Minimal example — a simple API-key provider

# plugins/model-providers/acme-inference/__init__.py
from providers import register_provider
from providers.base import ProviderProfile

acme = ProviderProfile(
    name="acme-inference",
    aliases=("acme",),
    display_name="Acme Inference",
    description="Acme — OpenAI-compatible direct API",
    signup_url="https://acme.example.com/keys",
    env_vars=("ACME_API_KEY", "ACME_BASE_URL"),
    base_url="https://api.acme.example.com/v1",
    auth_type="api_key",
    default_aux_model="acme-small-fast",
    fallback_models=(
        "acme-large-v3",
        "acme-medium-v3",
        "acme-small-fast",
    ),
)

register_provider(acme)

# plugins/model-providers/acme-inference/plugin.yaml
name: acme-inference
kind: model-provider
version: 1.0.0
description: Acme Inference — OpenAI-compatible direct API
author: Your Name

That's it. After dropping these two files, the following auto-wire with no other edits:

Integration	Where	What it gets
Credential resolution	hermes_cli/auth.py	PROVIDER_REGISTRY["acme-inference"] populated from profile
--provider CLI flag	hermes_cli/main.py	Accepts acme-inference
hermes model picker	hermes_cli/models.py	Appears in CANONICAL_PROVIDERS, model list fetched from {base_url}/models
hermes doctor	hermes_cli/doctor.py	Health check for ACME_API_KEY + {base_url}/models probe
hermes setup	hermes_cli/config.py	ACME_API_KEY appears in OPTIONAL_ENV_VARS and the setup wizard
URL reverse-mapping	agent/model_metadata.py	Hostname → provider name for auto-detection
Auxiliary model	agent/auxiliary_client.py	Uses default_aux_model for compression / summarization
Runtime resolution	hermes_cli/runtime_provider.py	Returns correct base_url, api_key, api_mode
Transport	agent/transports/chat_completions.py	Profile path generates kwargs via prepare_messages / build_extra_body / build_api_kwargs_extras

ProviderProfile fields

Full definition in providers/base.py. The most useful ones:

Field	Type	Purpose
name	str	Canonical id — matches model.provider in config.yaml and the --provider flag
aliases	tuple[str, ...]	Alternative names resolved by get_provider_profile() (e.g. grok → xai)
api_mode	str	chat_completions | codex_responses | anthropic_messages | bedrock_converse
display_name	str	Human label shown in hermes model picker
description	str	Picker subtitle
signup_url	str	Shown during first-run setup ("get an API key here")
env_vars	tuple[str, ...]	API-key env vars in priority order; a final *_BASE_URL entry is used as the user base-URL override
base_url	str	Default inference endpoint
models_url	str	Explicit catalog URL (falls back to {base_url}/models)
auth_type	str	api_key | oauth_device_code | oauth_external | copilot | aws_sdk | external_process
fallback_models	tuple[str, ...]	Curated list shown when live catalog fetch fails
default_headers	dict[str, str]	Sent on every request (e.g. Copilot's Editor-Version)
fixed_temperature	Any	None = use caller's value; OMIT_TEMPERATURE sentinel = don't send temperature at all (Kimi)
default_max_tokens	int | None	Provider-level max_tokens cap (Nvidia: 16384)
default_aux_model	str	Cheap model for auxiliary tasks (compression, vision, summarization)

Overridable hooks

Subclass ProviderProfile for non-trivial quirks:

from typing import Any
from providers.base import ProviderProfile

class AcmeProfile(ProviderProfile):
    def prepare_messages(self, messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
        """Provider-specific message preprocessing. Runs after codex
        sanitization, before developer-role swap. Default: pass-through."""
        # Example: Qwen normalizes plain-text content to a list-of-parts
        # array and injects cache_control; Kimi rewrites tool-call JSON
        return messages

    def build_extra_body(self, *, session_id=None, **context) -> dict:
        """Provider-specific extra_body fields merged into the API call.
        Context includes: session_id, provider_preferences, model, base_url,
        reasoning_config. Default: empty dict."""
        # Example: OpenRouter's provider-preferences block,
        # Gemini's thinking_config translation.
        return {}

    def build_api_kwargs_extras(self, *, reasoning_config=None, **context):
        """Returns (extra_body_additions, top_level_kwargs). Needed when some
        fields go top-level (Kimi's reasoning_effort, OpenRouter's verbosity for
        adaptive Anthropic models) and some go in extra_body (OpenRouter's
        reasoning dict). Default: ({}, {})."""
        return {}, {}

    def fetch_models(self, *, api_key=None, base_url=None, timeout=8.0) -> list[str] | None:
        """Live catalog fetch. Default hits {models_url or base_url}/models with
        Bearer auth. Override for: custom auth (Anthropic), no REST endpoint
        (Bedrock → None), or public/unauthenticated catalogs (OpenRouter)."""
        return super().fetch_models(api_key=api_key, base_url=base_url, timeout=timeout)

Hook reference examples

Look at these bundled plugins for idioms:

Plugin	Why look
plugins/model-providers/openrouter/	Aggregator with provider preferences, public model catalog
plugins/model-providers/gemini/	thinking_config translation (native + OpenAI-compat nested forms)
plugins/model-providers/kimi-coding/	OMIT_TEMPERATURE, extra_body.thinking, top-level reasoning_effort
plugins/model-providers/qwen-oauth/	Message normalization, cache_control injection, VL high-res
plugins/model-providers/nous/	Attribution tags, "omit reasoning when disabled"
plugins/model-providers/custom/	Ollama num_ctx + think: false quirks
plugins/model-providers/bedrock/	api_mode="bedrock_converse", fetch_models returns None (no REST endpoint)

User overrides — replace a built-in without editing the repo

Say you want to point gmi at your private staging endpoint for testing. Create ~/.hermes/plugins/model-providers/gmi/__init__.py:

from providers import register_provider
from providers.base import ProviderProfile

register_provider(ProviderProfile(
    name="gmi",
    aliases=("gmi-cloud", "gmicloud"),
    env_vars=("GMI_API_KEY",