Configure model routing

This page describes how to configure, deploy, and test model routing in API Gateway using OpenAPI 3.x specifications.

Before you begin

Before configuring model routing, check that your environment meets the following prerequisites:

1. Check IAM permissions: Check that you have access to the API Gateway Management Plane and Gemini Enterprise Agent Platform Model Garden. You must have the API Gateway Admin (roles/apigateway.admin) role to create API configs and gateways. In addition, the service account used by your API gateway—either the default Compute Engine service account or a user-managed service account specified when creating the API config—must be granted the Agent Platform User (roles/aiplatform.user) role to access target models.
2. Check model availability and endpoint access: Check that your routable models are pre-deployed open models for Model as a Service (MaaS) in Agent Platform Model Garden. All models referenced by a single router must share the exact same hostname. Pick either the global endpoint (aiplatform.googleapis.com) or a single regional endpoint (for example, us-central1-aiplatform.googleapis.com) for every model referenced within that router.
3. Check gateway deployment eligibility: You cannot update an existing gateway deployed without model routing to enable model routing, nor can you update a gateway deployed with model routing to disable or remove model routing. To switch routing modes, you must create and deploy a new API config and gateway instance.
4. Check VPC Service Controls and endpoint compatibility: Model routing gateways do not support VPC Service Controls or Private Service Connect (PSC) endpoint configurations. Check that your target project and API Gateway instances are not restricted by VPC Service Controls perimeters, and that your models use public regional or global endpoints.

Configuration validation

When you deploy an API config, the API Gateway management plane validates your OpenAPI specification. The management plane rejects invalid configurations during deployment with an informational validation error. The validation process enforces the following rules:

Structural and location checks

• The x-google-api-management extension and its associated blocks (backends, ai.models.routing.routers, individual routers, and rules) must be well-formed. Keys must match their expected data types (map, list, or string). The management plane rejects type mismatches with an expected map/list/string error.
• The x-google-api-management extension must contain a valid backends block when model routing is enabled.
• The x-google-model-router extension is supported only in OpenAPI 3.x specifications (it isn't supported in OpenAPI 2.0 / Swagger).
• The x-google-model-router extension can only be specified at the operation level. The management plane explicitly rejects x-google-model-router definitions placed at the path level or root (top) level.
• The ai.models.routing.routers block must be defined inside x-google-api-management whenever any operation references x-google-model-router.
• You cannot specify both x-google-model-router and x-google-backend on the same API operation.
• An OpenAPI specification cannot contain a mix of model routing and non-model routing operations. You cannot specify standard routing extensions (such as x-google-backend) on some operations while using x-google-model-router on other operations within the same API specification.

HTTP method check

• The x-google-model-router extension can only be applied to operations using the POST HTTP method. The management plane rejects model routing on any other HTTP method (such as GET, PUT, or DELETE).

Backend validity

• Every backend defined under x-google-api-management.backends must include a non-empty address field.
• The backend address must be a valid URL using the http or https scheme. To protect prompt payloads and authentication credentials in transit across public or remote endpoints, always specify the https scheme when defining the address field.
• Every backend defined under x-google-api-management.backends and referenced by a model router must use pathTranslation: CONSTANT_ADDRESS. The management plane rejects configurations using pathTranslation: APPEND_PATH_TO_ADDRESS for model routing backends because path translation is ignored in the model router's runtime path.
• Model routing backends don't support VPC Service Controls or Private Service Connect (PSC) endpoint configurations. All backend address fields must point to public regional or global MaaS open model endpoints.

Router reference resolution

• The router name referenced by an operation's x-google-model-router must match a valid router key defined under ai.models.routing.routers.
• The backend referenced by a router's defaultModel must match a valid backend defined under x-google-api-management.backends.
• The backend referenced by each rule in a router must match a valid backend defined under x-google-api-management.backends.

Router contents

• Each router must define a defaultModel.
• The defaultModel must include a valid backend field.
• The defaultModel must include a non-empty targetModel field.
• Each entry under rules must include a non-empty model field. The string value default is reserved and cannot be used as a rule's model value.
• Each entry under rules must include a non-empty targetModel field.
• The model values defined across all rules within a single router must be unique. The management plane rejects duplicate model values within the same router.

Backend host and scheme consistency

• All backends referenced by a single router (including defaultModel.backend and every rule's backend) must share the identical hostname and URL scheme. The management plane rejects configurations with differing hostnames or inconsistent schemes (http versus https) within the same router, ensuring the router dispatches all requests to a consistent upstream service endpoint.

Target model validation

• The <provider> portion of the targetModel string (google, openai, or anthropic) and the <provider>/<model> identifier format are both validated at config-create (deploy) time. The management plane rejects a targetModel that isn't formatted as <provider>/<model> or whose provider isn't google, openai, or anthropic with an InvalidArgument: unsupported publisher error during deployment.

Step 1: Identify target models

Identify the target foundation models and their corresponding Agent Platform endpoint URLs. All routable models within a router must share a single hostname (for MaaS open models, this hostname is aiplatform.googleapis.com).

Endpoint URL paths vary based on the model provider:

• Google Gemini: Uses the :generateContent method.
• Anthropic Claude: Uses the :rawPredict method.
• OpenAI: Uses the /endpoints/openapi/chat/completions endpoint path.

The following table lists the MaaS endpoints used in the OpenAPI specification example later in this section:

Replace YOUR_PROJECT_ID with your Google Cloud project ID.

Step 2: Configure the OpenAPI 3.x specification

Create or update your OpenAPI 3.x specification to define your backend endpoints and model routing configurations.

The following example demonstrates an OpenAPI 3.0.3 specification defining two distinct model routers. To prevent horizontal scrolling, long backend address URLs use YAML double-quoted multi-line string continuation (``):

openapi: 3.0.3

info:
  title: OpenAPI 3.x spec using Model Routing
  description: Using Model Routing in an OAS 3.x spec
  version: 1.0.0

x-google-api-management:
  backends:
    gemini-35-flashlite:
      address: "https://aiplatform.googleapis.com/v1/projects/\
        YOUR_PROJECT_ID/locations/global/publishers/google/\
        models/gemini-3.5-flash-lite:generateContent"
      deadline: 60.0
      pathTranslation: CONSTANT_ADDRESS

    anthropic-claude-opus-47:
      address: "https://aiplatform.googleapis.com/v1/projects/\
        YOUR_PROJECT_ID/locations/global/publishers/anthropic/\
        models/claude-opus-4-7:rawPredict"
      deadline: 60.0
      pathTranslation: CONSTANT_ADDRESS

    openai-gpt-oss-120b:
      address: "https://aiplatform.googleapis.com/v1/projects/\
        YOUR_PROJECT_ID/locations/global/endpoints/openapi/\
        chat/completions"
      deadline: 60.0
      pathTranslation: CONSTANT_ADDRESS

  ai:
    models:
      routing:
        routers:
          # Router 1: route between Gemini (default) and Claude.
          gemini-claude-router:
            defaultModel:
              backend: gemini-35-flashlite
              targetModel: google/gemini-3.5-flash-lite
            rules:
              - model: "claude-opus-4-7"
                backend: anthropic-claude-opus-47
                targetModel: anthropic/claude-opus-4-7

          # Router 2: route between OpenAI GPT (default) and Gemini.
          openai-gemini-router:
            defaultModel:
              backend: openai-gpt-oss-120b
              targetModel: openai/gpt-oss-120b-maas
            rules:
              - model: "gemini-3.5-flash-lite"
                backend: gemini-35-flashlite
                targetModel: google/gemini-3.5-flash-lite

servers:
  - url: "https://my-gateway-url.com"

paths:
  /v1/chat/gemini-claude:
    post:
      summary: "Endpoint:defaults to Gemini & Claude as an option."
      operationId: "chatGeminiClaude"
      x-google-model-router: gemini-claude-router
      responses:
        '200':
          description: "OK"

  /v1/chat/openai-gemini:
    post:
      summary: "Endpoint:defaults to OpenAI & Gemini as an option."
      operationId: "chatOpenAIGemini"
      x-google-model-router: openai-gemini-router
      responses:
        '200':
          description: "OK"

Configuration properties

1. backends: The backends object under x-google-api-management defines all routable model endpoints. Each backend name represents a symbolic model name (for example, gemini-35-flashlite) containing the destination address. The backends field is an existing Google OpenAPI extension.
2. ai.models.routing: The model routing configuration resides under x-google-api-management as ai.models.routing, containing a map of named routers. Each map entry defines one model router, where the key represents the router's name (for example, gemini-claude-router) and the value contains:
   • defaultModel: The required fallback model destination used when an incoming request payload doesn't match any explicit rule. It shares the exact structure of a rule entry but omits the model matching field. For OpenAI-compatible routes, when a request falls back to defaultModel, the value of targetModel is forwarded as the outgoing model attribute in the request body sent to Agent Platform.
   • rules: An optional array where each element maps a client-payload model string to a destination backend and target model.
3. Rule properties: Each entry within rules (and the defaultModel) defines the following properties:
   • model (rules only): The string value matched against the model attribute within the client's incoming JSON prompt payload. The router compares the incoming payload's model value to this string. If no rule matches, the router selects the defaultModel. For OpenAI-compatible routes (where the destination backend is /openapi/chat/completions), this string is forwarded directly as the outgoing model attribute in the request body sent to Agent Platform. Therefore, for OpenAI-compatible routes, the