Connecting to Cloud Storage Buckets

This page introduces Cloud Storage as an option for storing your machine learning data and training output, and describes how to give your Cloud TPU access to the data objects on Cloud Storage.

Before you begin

You need a Cloud TPU service account in order to access a Cloud Storage bucket.

  1. Create a Cloud TPU service account for your project.

    gcloud beta services identity create --service tpu.googleapis.com --project project-id

    The command returns a Cloud TPU service account with following format:

    service-project-number@cloud-tpu.iam.gserviceaccount.com
    
  2. Follow the instructions found in either the Cloud TPU quickstart guides or Manage TPUs to configure your Google Cloud project and create your Cloud TPU VM and TPU resources.

Write data to Cloud Storage

Console

  1. Go to the Cloud Storage page on the Google Cloud console.

    Go to the Cloud Storage page

  2. Create a new bucket, specifying the following options:

    • A unique name of your choosing.
    • Default storage class: Standard
    • Location: The region where you created the Cloud TPU. For more information about regions and TPU availability, see TPU regions and zones.

CLI

  1. Use the gcloud storage buckets create command to create a Cloud Storage bucket:

    gcloud storage buckets create gs://bucket-name --location region

    where:

    • region is the region where you created the Cloud TPU. For more information about regions and TPU availability, see TPU regions and zones.

    • bucket-name is the name of the bucket you want to create.

  2. Use the gcloud storage cp command to write files to the Cloud Storage bucket:

    gcloud storage cp -r local-data-dir gs://bucket-name