Create a MIG with a multi-host Cloud TPU slice

This document describes how to create a managed instance group (MIG) with a multi-host TPU slice.

Prerequisites

Complete the following prerequisites:

Create a project for your TPUs as described in Set up a project for TPUs.
Determine your TPU requirements as described in Plan your resources.

Create a MIG with multi-host TPU slices

Create an instance template.
Create a workload policy.
Create the MIG.

Create an instance template

The command to create an instance template depends on the consumption option you use: on-demand, Spot, reservation-bound, or flex-start. For more information about consumption options, see Plan your TPU resources.

Create an instance template for an on-demand TPU VM

The following command creates an instance template using the on-demand consumption option:

gcloud

gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
  --machine-type=MACHINE_TYPE \
  --maintenance-policy=TERMINATE \
  --image-family=IMAGE_FAMILY \
  --image-project=IMAGE_PROJECT

REST

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "INSTANCE_TEMPLATE_NAME",
    "properties": {
      "machineType": "MACHINE_TYPE",
      "scheduling": {
        "onHostMaintenance": "TERMINATE"
      },
      "disks": [
        {
          "boot": true,
          "initializeParams": {
            "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
          }
        }
      ],
      "networkInterfaces": [
        {
          "network": "global/networks/default"
        }
      ]
    }
  }' \
  "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instanceTemplates"

Replace the following placeholders:

PROJECT_ID: The ID of your Google Cloud project.
INSTANCE_TEMPLATE_NAME: The name of your instance template.
MACHINE_TYPE: The machine type for the TPU VM, for example, ct6e-standard-8t.
IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.

Create an instance template for a TPU Spot VM

The following command creates an instance template using the Spot consumption option:

gcloud

gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
  --machine-type=MACHINE_TYPE \
  --maintenance-policy=TERMINATE \
  --instance-termination-action=STOP \
  --provisioning-model=SPOT \
  --image-family=IMAGE_FAMILY \
  --image-project=IMAGE_PROJECT

REST

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "INSTANCE_TEMPLATE_NAME",
    "properties": {
      "machineType": "MACHINE_TYPE",
      "disks": [
        {
          "boot": true,
          "initializeParams": {
            "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
          }
        }
      ],
      "scheduling": {
        "onHostMaintenance": "TERMINATE",
        "provisioningModel": "SPOT",
        "instanceTerminationAction": "STOP"
      },
      "networkInterfaces": [
        {
          "network": "global/networks/default"