Configure queue routing, limits, and retries

You can configure your Cloud Tasks queue when you create the queue or anytime afterwards. The configuration is applied to all tasks in that queue.

Note that configuring authentication at the queue level overrides authentication configured at the task level. For more information, see Use HTTP target tasks with authentication tokens.

There are three basic aspects to configuring your queues:

Configure queue-level routing

Configuring routing at the queue level overrides routing set at the task level. This is useful if you want to use Cloud Tasks as a buffer in front of your target service, or if you need to change the routing for all tasks in a queue.

Queue-level routing applies to:

  • Tasks that are in the queue
  • Tasks that are added to the queue after queue-level routing has been set

Limitations

Queue-level routing is not compatible with Cloud Key Management Service (Cloud KMS) customer-managed encryption keys (CMEK). If CMEK is enabled, you can't do the following:

  • Create tasks on a queue that has queue-level routing
  • Apply queue-level routing

Configure queue-level routing for HTTP tasks

You can configure a queue to override task-level routing either when creating the queue or when updating the queue. To configure queue-level routing, set the queue's uriOverride parameter to your preferred route.

If you are applying queue-level routing as an update to an existing queue, pause the queue before applying the changes and wait one minute after applying the changes to resume the queue.

  1. Pause the queue by running the following command:

      gcloud tasks queues pause QUEUE_ID
      

    Replace QUEUE_ID with the ID of your queue.

  2. Update or remove queue-level routing.

    • To update queue-level routing, set the uriOverride parameter to your updated route.

    • To remove queue-level routing using either the REST or RPC API:

      • REST API: Send a patch request for the queue with an empty payload and the updateMask parameter set to httpTarget.

      • RPC API: Send an updateQueueRequest for the queue with an empty payload and the update_mask parameter set to http_target.

    The following example uses the REST API to update the host that tasks are routed to:

    curl -X PATCH -d @- -i \
      -H "Authorization: Bearer ACCESS_TOKEN" \
      -H "Content-Type: application/json" \
      "https://cloudtasks.googleapis.com/v2/projects/PROJECT_ID/locations/LOCATION/queues/QUEUE_ID?updateMask=httpTarget.uriOverride" << EOF
    {
      "httpTarget": {"uriOverride":{"host":"NEW_HOST"}}
    }
    EOF
    

    Replace the following:

    • ACCESS_TOKEN: your access token. You can get this by running the following in your terminal:

      gcloud auth application-default login
      gcloud auth application-default print-access-token
    • PROJECT_ID: the ID of your Google Cloud project. You can get this by running the following in your terminal:

      gcloud config get-value project

    • LOCATION: the location of your queue.

    • NEW_HOST: the new host you want your queue to route to.

  3. Wait one minute.

    It can take up to one minute for the new configuration to take effect. Waiting to resume the queue helps to prevent tasks from dispatching with the old configuration.

  4. Resume the queue by running the following command:

    gcloud tasks queues resume QUEUE_ID

Configure queue-level routing for App Engine tasks

To configure queue-level routing for App Engine tasks, set the queue's appEngineRoutingOverride parameter to your preferred App Engine service and version.

  1. Set up queue-level routing and override any task-level routing:

    gcloud tasks queues update QUEUE_ID \
        --routing-override=service:SERVICE,version:VERSION

    Replace the following:

    • QUEUE_ID: the queue ID (its short name).
    • SERVICE: the App Engine worker service responsible for task handling.
    • VERSION: the app version.

    For example, if you set up a worker service to handle all tasks in a queue, you can route to that service and the default version:

    gcloud tasks queues update QUEUE_ID \
        --routing-override=service:SERVICE
  2. Verify your queue was configured successfully by running the following command:

    gcloud tasks queues describe QUEUE_ID --location=LOCATION

    Replace LOCATION with the location of the queue.

    The output should be similar to the following:

    appEngineRoutingOverride:
      host: SERVICE.PROJECT_ID.appspot.com
      service: SERVICE
    name: projects/PROJECT_ID/locations/LOCATION_ID/queues/QUEUE_ID
    rateLimits:
      maxBurstSize: 100
      maxConcurrentDispatches: 1000
      maxDispatchesPerSecond: 500.0
    retryConfig:
      maxAttempts: 100