Backend services overview

A backend service defines how Cloud Load Balancing distributes traffic. The backend service configuration contains a set of values, such as the protocol used to connect to backends, various distribution and session settings, health checks, and timeouts. These settings provide fine-grained control over how your load balancer behaves. To get you started, most of the settings have default values that allow for fast configuration. A backend service is either global or regional in scope.

Load balancers, Envoy proxies, and proxyless gRPC clients use the configuration information in the backend service resource to do the following:

• Direct traffic to the correct backends, which are instance groups or network endpoint groups (NEGs).
• Distribute traffic according to a balancing mode, which is a setting for each backend.
• Determine which health check is monitoring the health of the backends.
• Specify session affinity.
• Determine whether other services are enabled, including the following services that are only available for certain load balancers:
  • Cloud CDN
  • Google Cloud Armor security policies
  • Identity-Aware Proxy
• Designate global and regional backend services as a service in App Hub applications.

You set these values when you create a backend service or add a backend to the backend service.

The following table summarizes which load balancers use backend services. The product that you are using also determines the maximum number of backend services, the scope of a backend service, the type of backends supported, and the backend service's load balancing scheme. The load balancing scheme is an identifier that Google uses to classify forwarding rules and backend services. Each load balancing product uses one load balancing scheme for its forwarding rules and backend services. Some schemes are shared among products.

Load balancer naming

For Proxy Network Load Balancers and Passthrough Network Load Balancers, the name of the load balancer is always the same as the name of the backend service. The behavior for each Google Cloud interface is as follows:

• Google Cloud console. If you create either a proxy Network Load Balancer or a passthrough Network Load Balancer by using the Google Cloud console, the backend service is automatically assigned the same name that you entered for the load balancer name.
• Google Cloud CLI or API. If you create either a proxy Network Load Balancer or a passthrough Network Load Balancer by using the gcloud CLI or the API, you enter a name of your choice while creating the backend service. This backend service name is then reflected in the Google Cloud console as the name of the load balancer.

To learn about how naming works for Application Load Balancers, see URL maps overview: Load balancer naming.

Backends

A backend is one or more endpoints that receive traffic from a Google Cloud load balancer, a Cloud Service Mesh-configured Envoy proxy, or a proxyless gRPC client. There are several types of backends:

• Instance group containing virtual machine (VM) instances. An instance group can be a managed instance group (MIG), with or without autoscaling, or it can be an unmanaged instance group. More than one backend service can reference an instance group, but all backend services that reference the instance group must use compatible balancing modes. For more information, in this document, see Restrictions and guidance for instance groups.
• Zonal NEG
• Serverless NEG
• Private Service Connect NEG
• Internet NEG
• Hybrid connectivity NEG
• Port mapping NEG
• Service Directory service bindings (Preview)
• Published service backends (Preview)

You cannot delete a backend instance group or NEG that is associated with a backend service. Before you delete an instance group or NEG, you must first remove it as a backend from all backend services that reference it.

Instance groups

This section discusses how instance groups work with the backend service.

Backend VMs and external IP addresses

Backend VMs in backend services don't need external IP addresses:

• For global external Application Load Balancers and external proxy Network Load Balancers: Clients communicate with a first-layer Google Front End (GFE) which hosts your load balancer's external IP address. The first-layer GFE communicates with a second-layer GFE, located in the same region as the backend VM or endpoint. Each second-layer GFE communicates with backend VMs or endpoints according to the following rules:

  • Load-balanced network interface: The network interface that the second-layer GFE sends request traffic to depends on the type of backend group:

    • For instance group backends, the load balancer always delivers packets to the nic0 interface of each backend VM. This rule is true even if the VM has multiple network interfaces in the same or in different VPC networks.

    • For GCE_VM_IP_PORT zonal NEG backends, the load balancer delivers packets to the network interface that the endpoint IP address is associated with. For backend VMs with multiple network interfaces, the network interface can be in any VPC network, subject to the following exception: If a backend VM has its nic0 interface and one or more non-nic0 interfaces in the same VPC network, then from the set of network interfaces in the same VPC network, the load balancer delivers packets to only the nic0 network interface.

  • Destination IP address on the load-balanced interface: The second-layer GFE sends request traffic whose packets have the following destination IP addresses:

    • For instance group backends, the packet destination is either the primary internal IPv4 address of the nic0 network interface or the first /128 IPv6 address from the /96 IPv6 range assigned to the nic0 interface, depending on the IP address selection policy of the backend service and network interface stack type.

    • For GCE_VM_IP_PORT zonal NEG backends, the packet destination matches an endpoint IP address specified in the NEG, depending on the IP address selection policy of the backend service and network interface stack type. For valid endpoint IP addresses, see NEGs with GCE_VM_IP_PORT endpoints. If a backend VM has its nic0 interface and one or more non-nic0 interfaces in the same VPC network, then from the set of network interfaces in the same VPC network, you can only specify an endpoint IP address associated with the nic0 network interface.

  Communication between second-layer GFEs and backends is facilitated through special routes.

• For regional external Application Load Balancers and regional external proxy Network Load Balancers: Clients communicate with a managed Envoy proxy that hosts your load balancer's external IP address. The Envoy proxy is located in a proxy-only subnet. Each Envoy proxy communicates with backend VMs or endpoints according to the following rules:

  • Load-balanced network interface: The network interface to which the Envoy proxy sends request traffic depends on the type of backend group:

    • For instance group backends, the load balancer always delivers packets to the nic0 interface of each backend VM. This rule is true even if the VM has multiple network interfaces in the same or in different VPC networks.

    • For GCE_VM_IP_PORT zonal NEG backends, the load balancer delivers packets to the network interface with which the endpoint IP address is associated. For backend VMs with multiple network interfaces, the network interface can be in any VPC network, subject to the following exception: If a backend VM has its nic0 interface and one or more non-nic0 interfaces in the same VPC network, then from the set of network interfaces in the same VPC network, the load balancer delivers packets to only the nic0 network interface.

  • Destination IP address on the load-balanced interface: The Envoy proxy sends request traffic whose packets have the following destination IP addresses:

    • For instance group backends, the packet destination is either the primary internal IPv4 address of the nic0 network interface or the first /128 IPv6 address from the /96 IPv6 range assigned to the nic0 interface, depending on the IP address selection policy of the backend service and network interface stack type.

    • For GCE_VM_IP_PORT zonal NEG backends, the packet destination matches an endpoint IP address specified in the NEG, depending on the IP address selection policy of the backend service and network interface stack type. For valid endpoint IP addresses, see NEGs with GCE_VM_IP_PORT endpoints. If a backend VM has its nic0 interface and one or more non-nic0 interfaces in the same VPC network, then from the set of network interfaces in the same VPC network, you can only specify an endpoint IP address associated with the nic0 network interface.

• For global external passthrough Network Load Balancers and regional external passthrough Network Load Balancers: Clients communicate directly with backends by using Google Maglev infrastructure. Packets are routed and delivered to backends with the original source and destination IP addresses preserved. The load balancer delivers packets to one of the following network interfaces:

  • For global external passthrough Network Load Balancers with instance group backends, the load balancer always delivers packets to the nic0 interface. For more information, see Instance group backends and network interfaces.

  • For global external passthrough Network Load Balancers with GCE_VM_IP NEG backends, the load balancer delivers packets to the network interface that is in the subnetwork associated with NEG. For more information, see Zonal NEG backends and network interfaces.

  • For backend service-based regional external passthrough Network Load Balancers with instance group backends, the load balancer always delivers packets to the nic0 interface. For more information, see Instance group backends and network interfaces.

  • For backend service-based regional external passthrough Network Load Balancers with GCE_VM_IP NEG backends, the load balancer delivers packets to the network interface that is in the subnetwork associated with NEG. For more information, see Zonal NEG backends and network interfaces.

  • For target pool-based regional external passthrough Network Load Balancers, the load balancer always delivers packets to the nic0 interface.

  • If a backend VM has its nic0 interface and one or more non-nic0 interfaces in the same VPC network, then from the set of network interfaces in the same VPC network, the load balancer delivers packets only to the nic0 network interface.

• For internal passthrough Network Load Balancers: Clients communicate directly with backends by using the Andromeda network virtualization stack. Packets are routed and delivered to backends with the original source and destination IP addresses preserved. For instance group backends and GCE_VM_IP NEG backends, the load balancer delivers packets to a network interface in the backend service's VPC network:

  • The backend service's VPC network can be specified explicitly, or it can be inherited from the first backend instance group or NEG added to the backend service or the first forwarding rule that references the backend service. For more information, see Backend service network specification.

  • If a backend VM has its nic0 interface and one or more non-nic0 interfaces in the same VPC network, then from the set of network interfaces in the same VPC network, the load balancer delivers packets to only the nic0 network interface.

Named ports

The backend service's named port attribute is only applicable to proxy-based load balancers (Application Load Balancers and Proxy Network Load Balancers) using instance group backends. The named port defines the destination port used for the TCP connection between the proxy (GFE or Envoy) and the backend instance.

Named ports are configured as follows:

• On each instance group backend, you must configure one or more named ports using key-value pairs. The key represents a meaningful port name that you choose, and the value represents the port number you assign to the name. The mapping of names to numbers is done individually for each instance group backend.

• On the backend service, you specify a single named port using just the port name (--port-name).

On a per-instance group backend basis, the backend service translates the port name to a port number. When an instance group's named port matches the backend service's --port-name, the backend service uses this port number for communication with the instance group's VMs.

For example, you might set the named port on an instance group with the name my-service-name and the port 8888:

gcloud compute instance-groups unmanaged set-named-ports my-unmanaged-ig \
    --named-ports=my-service-name:8888

Then you refer to the named port in the backend service configuration with the --port-name on the backend service set to my-service-name:

gcloud compute backend-services update my-backend-service \
    --port-name=my-service-name

A backend service can use a different port number when communicating with VMs in different instance groups if each instance group specifies a different port number for the same port name.

The resolved port number used by the proxy load balancer's backend service doesn't need to match the port number used by the load balancer's forwarding rules. A proxy load balancer listens for TCP connections sent to the IP address and destination port of its forwarding rules. Because the proxy opens a second TCP connection to its backends, the second TCP connection's destination port can be different.

Named ports are only applicable to instance group backends. Zonal NEGs with GCE_VM_IP_PORT endpoints, hybrid NEGs with NON_GCP_PRIVATE_IP_PORT endpoints, and internet NEGs define ports using a different mechanism, namely, on the endpoints themselves. Serverless NEGs reference Google services and PSC NEGs reference service attachments using abstractions that don't involve specifying a destination port.

Internal passthrough Network Load Balancers and regional external passthrough Network Load Balancers don't use named ports. This is because they are pass-through load balancers that route connections directly to backends instead of creating new connections. Packets are delivered to the backends preserving the destination IP address and port of the load balancer's forwarding rule.

To learn how to create named ports, see the following instructions:

• Unmanaged instance groups: Working with named ports
• Managed instance groups: Assigning named ports to managed instance groups

Restrictions and guidance for instance groups

Keep the following in mind when you use instance group backends:

• A VM instance can only belong to a single load-balanced instance group. For example, a VM can be a member of two unmanaged instance groups, or a VM can be a member of one managed instance group and one unmanaged instance group. When a VM is a member of two or more instance groups, only one of the instance groups can be referenced by one or more load balancer backend services.

• The same instance group can be used by two or more backend services. Each mapping between an instance group and a backend service can use a different balancing mode except for the incompatible balancing mode combinations.

  • The incompatible balancing mode combinations are as follows:

    • The UTILIZATION balancing mode is incompatible with all other balancing modes. If an instance group is a backend of multiple backend services, the instance group must use the UTILIZATION balancing mode on every backend service.

    • The CUSTOM_METRICS balancing mode is incompatible with all other balancing modes. If an instance group is a backend of multiple backend services, the instance group must use the CUSTOM_METRICS balancing mode on every backend service.

  • As a consequence of the incompatible balancing mode combinations, if an instance group uses either the UTILIZATION or CUSTOM_METRICS balancing mode as a backend for at least one backend service, the same instance group can't be used as a backend for internal passthrough Network Load Balancer and regional external passthrough Network Load Balancer. This is because instance group backends of regional passthrough Network Load Balancers (internal passthrough Network Load Balancer and regional external passthrough Network Load Balancer) use CONNECTION as their balancing mode.

• There's no single command that can change the balancing mode of the same instance group on multiple backend services. If an instance group is a backend for two or more backends services, and their balancing modes need to be updated to a new mode that is incompatible with their current mode, you can use the following technique:

  • Remove the instance group as a backend from all backend services except for one backend service.
  • Change the instance group's balancing mode for the one remaining backend service.
  • Re-add the instance group as a backend to the other backend services.

Consider the following best practices, which provide more flexible options:

• Avoid using the same instance group as a backend for two or more backend services. Instead, use multiple NEGs.

  • Unlike instance groups, a VM can have an endpoint in two or more load-balanced NEGs.

  • For example, if a VM needs to simultaneously be a backend of both a passthrough Network Load Balancer and either a proxy Network Load Balancer or an Application Load Balancer, use multiple load-balanced NEGs. Place a VM endpoint in a unique NEG compatible with each load balancer type. Then associate each NEG with the corresponding load balancer backend service.

• Don't add an autoscaled managed instance group to more than one backend service when using the HTTP Load Balancing Utilization autoscaling metric. Two or more backend services referencing the same autoscaled managed instance group can contradict with one another unless the autoscaling metric is unrelated to load balancer activity.

Zonal network endpoint groups

Network endpoints represent services by their IP address or an IP address and port combination, rather than referring to a VM in an instance group. A network endpoint group (NEG) is a logical grouping of network endpoints.

Zonal NEGs are zonal resources that represent collections of either IP addresses or IP address and port combinations for Google Cloud resources within a single subnet.

A backend service that uses zonal NEGs as its backends distributes traffic among applications or containers running within VMs.

There are two types of network endpoints available for zonal NEGs:

• GCE_VM_IP endpoints that are supported by all passthrough load balancers, except target pool-based regional external passthrough Network Load Balancers.

• GCE_VM_IP_PORT endpoints that are supported by all proxy-based load balancers (Application Load Balancers and Proxy Network Load Balancers) and Cloud Service Mesh.

To see which products support zonal NEG backends, see Table: Backend services and supported backend types.

For details, see Zonal NEGs overview.

Internet network endpoint groups

Internet NEGs are resources that define external backends. An external backend is a backend that is hosted within on-premises infrastructure or on infrastructure provided by third parties.

An internet NEG is a combination of a hostname or an IP address, plus an optional port. There are two types of network endpoints available for internet NEGs: INTERNET_FQDN_PORTand INTERNET_IP_PORT.

For details, see Internet network endpoint group overview.

Serverless network endpoint groups

A network endpoint group (NEG) specifies a group of backend endpoints for a load balancer. A serverless NEG is a backend that points to a Cloud Run, App Engine, Cloud Run functions, or API Gateway resource.

A serverless NEG can represent one of the following:

• A Cloud Run resource or a group of resources.
• A Cloud Run function or group of functions (formerly Cloud Run functions 2nd gen).
• A Cloud Run function (1st gen) or group of functions
• An App Engine standard environment or App Engine flexible environment app, a specific service within an app, a specific version of an app, or a group of services.
• An API Gateway that provides access to your services through a REST API consistent across all services, regardless of service implementation. This capability is in Preview.

To set up a serverless NEG for serverless applications that share a URL pattern, you use a URL mask. A URL mask is a template of your URL schema (for example, example.com/<service>). The serverless NEG will use this template to extract the <service> name from the incoming request's URL and route the request to the matching Cloud Run, Cloud Run functions, or App Engine service with the same name.

To see which load balancers support serverless NEG backends, see Table: Backend services and supported backend types.

For more information about serverless NEGs, see the Serverless network endpoint groups overview.

Service Directory service bindings

A Service Directory service binding is a backend resource that establishes a connection between a backend service in Cloud Service Mesh and a service registered in Service Directory. A backend service can reference several Service Directory service bindings. However, a backend service with a Service Directory service binding can't reference any other type of backend resource. For more information, see Cloud Service Mesh integration with Service Directory.

Published service backends

If both the producer and consumer VPC networks belong to the same organization, you can access a published service by using published service backends. Published service backends let you configure supported load balancers or Cloud Service Mesh to route traffic to published services through Private Service Connect endpoints. To route traffic to the published service, you associate a regional backend service with a service attachment. This configuration is known as a published service backend.

These backends provide unified connectivity by allowing supported load balancers and Cloud Service Mesh to share Private Service Connect endpoints. You establish this connection by providing a reference to the producer's service attachment. Published service backends also enable advanced traffic management and observability, such as enforcing centralized security policies and collecting centralized metrics.

For more information on how to configure published service backends for load balancers, see Access published services through published service backends and endpoints.

Mixed backends

The following usage considerations apply when you add different types of backends to a single backend service:

• A single backend service cannot simultaneously use both instance groups and zonal NEGs.
• You can use a combination of different types of instance groups on the same backend service. For example, a single backend service can reference a combination of both managed and unmanaged instance groups. For complete information about which backends are compatible with which backend services, see the table in the previous section.
• With certain proxy load balancers, you can use a combination of zonal NEGs (with GCE_VM_IP_PORT endpoints) and hybrid connectivity NEGs (with NON_GCP_PRIVATE_IP_PORT endpoints) to configure hybrid load balancing. To see which load balancers have this capability, refer Table: Backend services and supported backend types.

Protocol to the backends

When you create a backend service, you must specify the protocol used to communicate with the backends. You can specify only one protocol per backend service — you cannot specify a secondary protocol to use as a fallback.

Which protocols are valid depends on the type of load balancer or whether you are using Cloud Service Mesh.

Changing a backend service's protocol makes the backends inaccessible through load balancers for a few minutes.

IP address selection policy

This field is applicable to proxy load balancers. You must use the IP address selection policy to specify the traffic type that is sent from the backend service to your backends.

When you select the IP address selection policy, ensure that your backends support the selected traffic type. For more information, see Table: Backend services and supported backend types.

IP address selection policy is used when you want to convert your load balancer backend service to support a different traffic type. For more information, see Convert from single-stack to dual-stack.

You can specify the following values for the IP address selection policy:

Encryption between the load balancer and backends

For information about encryption between the load balancer and backends, see Encryption to the backends.

Balancing mode, target capacity, and capacity scaler

For Application Load Balancers, Cloud Service Mesh, proxy Network Load Balancers, and global external passthrough Network Load Balancers the balancing mode, target capacity, and capacity scaler are parameters you provide when you add a supported backend to a backend service. The load balancers use these parameters to manage the distribution of new requests (for HTTP load balancing), or new connections (for TCP proxy load balancing and global passthrough load balancing) to zones that contain supported backends:

• The balancing mode defines how the load balancer measures capacity. In other words, the balancing mode is the unit by which capacity is defined.

  Google Cloud has the following balancing modes:

  • CONNECTION: defines capacity based on the number of new TCP connections.
  • RATE: defines capacity based on the rate of new HTTP requests per second (for Application Load Balancers) or packets per second (for global external passthrough Network Load Balancers).
  • IN-FLIGHT: defines capacity based on the number of in-flight HTTP requests instead of the rate of HTTP requests. Use this balancing mode instead of RATE if requests take more than a second to complete.
  • UTILIZATION: defines capacity based on the approximated CPU utilization of VMs in a zone of an instance group.
  • CUSTOM_METRICS: defines capacity based on user-defined custom metrics.

• The target capacity defines the target capacity number.

  • The target capacity isn't a circuit breaker.
  • When capacity usage reaches the target capacity, the load balancer directs new requests or new connections to a different zone if backends are configured in two or more zones.
  • Global and cross-region load balancers also use capacity to direct requests or new connections to zones in different regions, if you've configured backends in more than one region.
  • When all zones have reached target capacity, new requests or new connections are distributed by overfilling proportionally.

• The capacity scaler provides a way to scale the target capacity manually. The values for the capacity scaler are as follows:

  • 0: indicates that the backend is completely drained. You can't use a value of 0 if a backend service has only one backend.
  • 0.1 (10%) - 1.0 (100%): indicates the percentage of backend capacity that is in use.

Supported backends

For proxy-based load balancers (Application Load Balancers and proxy Network Load Balancers) and Cloud Service Mesh, the following types of backends support the balancing mode, target capacity, and capacity scaler parameters:

• Instance group backends
• Zonal NEGs with GCE_VM_IP_PORT endpoints
• Zonal hybrid connectivity NEGs

For global external passthrough Network Load Balancers, the following types of backends support the balancing mode, target capacity, and capacity scaler parameters:

• Instance group backends
• Zonal NEGs with GCE_VM_IP endpoints

Internet NEGs, serverless NEGs, and Private Service Connect NEGs don't support the balancing mode, target capacity, and capacity scaler parameters.

Balancing modes for Application Load Balancers and Cloud Service Mesh

Available balancing modes for Application Load Balancer and Cloud Service Mesh backends depend on the type of supported backend and a traffic duration setting.

Traffic duration setting

For Application Load Balancer and Cloud Service Mesh backends, you can optionally specify a traffic duration setting. This setting is unique to the mapping between a supported backend and a backend service. The traffic duration setting has two valid values:

• SHORT: recommended for HTTP requests answered with responses from backends in less than one second. If you don't explicitly specify a traffic duration, the load balancer operates as if you'd specified SHORT.
• LONG: recommended for HTTP requests for which the backend needs more than one second to generate responses.

To explicitly set the traffic duration when you add a backend to a backend service, do one of the following:

• Run the gcloud compute backend-services add-backend command with the --traffic-duration flag.
• Create a backend service or update a backend service with the trafficDuration attribute.

Balancing modes for short traffic duration

When the traffic duration setting isn't specified or is set to SHORT, the available balancing modes for Application Load Balancer and Cloud Service Mesh backends depend on the type of supported backend.

Balancing modes for long traffic duration

When the traffic duration setting is LONG, the available balancing modes for Application Load Balancer and Cloud Service Mesh backends depend on the type of supported backend.

Balancing modes for Proxy Network Load Balancers

Available balancing modes for proxy Network Load Balancer backends depend on the type of supported backend.

Balancing modes for passthrough Network Load Balancers

The following tables outline the balancing modes for global and regional passthrough Network Load Balancers.

Balancing modes for global external passthrough Network Load Balancers

Available balancing modes for global external passthrough Network Load Balancer backends depend on the type of supported backend.

Balancing modes for regional passthrough Network Load Balancers

The following table shows the balancing modes for regional passthrough Network Load Balancers—regional external passthrough Network Load Balancer and internal passthrough Network Load Balancer.

Target capacity specifications for proxy-based load balancers

This section covers target capacity specifications for Application Load Balancer, Cloud Service Mesh, and proxy Network Load Balancer backends that support balancing mode, target capacity, and capacity scaler settings.

To learn more about target capacity specifications for global external passthrough Network Load Balancers, see Target capacity specifications for global external passthrough Network Load Balancers.

Connection balancing mode

Proxy Network Load Balancer backends can use the CONNECTION balancing mode with one of the following required target capacity parameters:

Using the max-connections parameter

When you specify the max-connections parameter, the value you provide defines the capacity for an entire zone.

• For a zonal instance group with N total instances and h healthy instances (where h ≤ N), the calculations are as follows:

  • If you set max-connections to X, the zonal target capacity is X.
  • The average connections per instance is X / h.

• Regional managed instance groups don't support the max-connections parameter because they consist of multiple zones. Instead, use the max-connections-per-instance parameter.

• For a zonal NEG with N total endpoints and h healthy endpoints (where h ≤ N), the calculations are as follows:

  • If you set max-connections to X, the zonal target capacity is X.
  • The average connections per endpoint is X / h.

Using the max-connections-per-instance or max-connections-per-endpoint parameter

When you specify either the max-connections-per-instance or max-connections-per-endpoint parameter, the load balancer uses the value you provide to calculate a per-zone capacity:

• For a zonal instance group with N total instances and h healthy instances (where h ≤ N), the calculations are as follows:

  • If you set max-connections-per-instance to X, the zonal target capacity is N * X. This is equivalent to setting max-connections to N * X.
  • The average connections per instance is (N * X) / h.

• For a regional managed instance group, if you set max-connections-per-instance to X, Google Cloud calculates a per-zone target capacity for each zone of the instance group. In each zone, if there are K total instances and h healthy instances (where h≤ K), the calculations are as follows:

  • The zone's target capacity is K * X.
  • The average connections per instance in the zone is (K * X) / h.

• For a zonal NEG with N total endpoints and h healthy endpoints (where h ≤ N), the calculations are as follows:

  • If you set max-connections-per-endpoint to X, the zonal target capacity is N * X. This is equivalent to setting max-connections to N * X.
  • The average connections per endpoint is (N * X) / h.

Rate balancing mode

Application Load Balancer and Cloud Service Mesh backends with an unspecified or short traffic duration setting can use the RATE balancing mode with one of the following required target capacity parameters:

Using the max-rate parameter

When you specify the max-rate parameter, the value you provide defines the capacity for an entire zone.

• For a zonal instance group with N total instances and h healthy instances (where h ≤ N), the calculations are as follows:

  • If you set max-rate to X, the zonal target capacity is X requests per second.
  • The average requests per second per instance is X / h.

• Regional managed instance groups don't support the max-rate parameter because they consist of multiple zones. Instead, use the max-rate-per-instance parameter.

• For a zonal NEG with N total endpoints and h healthy endpoints (where h ≤ N), the calculations are as follows:

  • If you set max-rate to X, the zonal target capacity is X requests per second.
  • The average requests per second per endpoint is X / h.

Using the max-rate-per-instance or max-rate-per-endpoint parameter

When you specify either the max-rate-per-instance or max-rate-per-endpoint parameter, the load balancer uses the value you provide to calculate a per-zone capacity:

• For a zonal instance group with N total instances and h healthy instances (where h ≤ N), the calculations are as follows:

  • If you set max-rate-per-instance to X, the zonal target capacity is N * X requests per second. This is equivalent to setting max-rate to N * X.
  • The average requests per second per instance is (N * X) / h.

• For a regional managed instance group, if you set max-rate-per-instance to X, Google Cloud calculates a per-zone target capacity for each zone of the instance group. In each zone, if there are K total instances and h healthy instances (where h≤ K), the calculations are as follows:

  • The zone's target capacity is K * X requests per second.
  • The average requests per second per instance in the zone is (K * X) / h.

• For a zonal NEG with N total endpoints and h healthy endpoints (where h ≤ N), the calculations are as follows:

  • If you set max-rate-per-endpoint to X, the zonal target capacity is N * X requests per second. This is equivalent to setting max-rate to N * X.
  • The average requests per second per endpoint is (N * X) / h.

In-flight balancing mode

Application Load Balancers (except classic Application Load Balancers) and Cloud Service Mesh backends with a long traffic duration setting can use the IN_FLIGHT balancing mode with one of the following required target capacity parameters: