Connect from Compute Engine

This guide gives instructions on creating a single Compute Engine client and connecting it to your Google Cloud Managed Lustre instance. Managed Lustre supports connections from up to 20,000 clients.

For better performance, client Compute Engine VMs should be created in the same zone as the Managed Lustre instance.

Required permissions

You must have the following IAM roles:

  • Compute Instance Admin (v1) (roles/compute.instanceAdmin.v1) to create a Compute Engine VM.

  • Compute Admin (roles/compute.admin) or Compute Security Admin (roles/compute.securityAdmin) to create a firewall rule.

  • IAP-Secured Tunnel User (roles/iap.tunnelResourceAccessor) to SSH to a Compute Engine VM using Identity-Aware Proxy.

For a full list of the permissions granted by each role, see the IAM roles reference.

Limitations

Managed Lustre can't be mounted on Shielded VMs.

Create a Compute Engine VM

Follow the instructions to create a Compute Engine VM using one of the following Google Cloud image families:

  • Rocky Linux 8
  • Rocky Linux 9
  • Red Hat Enterprise Linux (RHEL) 9
  • Ubuntu 20.04 LTS, v20250213 or later. Note that Ubuntu 20.04 has reached end of support (EOS) and cannot be selected from the Google Cloud console.
  • Ubuntu 22.04 LTS, v20250128 or later
  • Ubuntu 24.04 LTS, for Arm VMs only.

If you're running a custom Ubuntu 22.04 or 24.04 kernel, support for DKMS is available.

Machine types, storage, and networking

You can choose any machine type and boot disk supported by your OS image. We recommend at least a c2-standard-4 machine type.

When selecting storage options:

  • Ensure you select a disk type supported by your machine series. 4th-generation machine families such as c4, c4a, or n4, require hyperdisk-balanced or another Hyperdisk type. Standard machine families such as c2d, c2, or n2, use pd-balanced or another Persistent Disk type.

To obtain the best network throughput, consider your machine type and vCPU count. In general:

  • Increase the number of vCPUs. Per-instance maximum egress bandwidth is generally 2 Gbps per vCPU, up to the machine type maximum.
  • Select a machine series that supports higher ingress and egress limits. For example, C2 instances with Tier_1 networking support up to 100Gbps egress bandwidth, while C3 instances support up to 200Gbps.
  • Enable per VM Tier_1 networking performance with larger machine types. Not all machine types support Tier_1 networking. If your selected machine type doesn't support it, omit the Tier_1 configuration option.
  • Select a machine series with multiple physical network interface cards (NICs) to aggregate network bandwidth. Physical NICs must be attached to regular VPCs, not VPCs with RDMA network profiles, in order to increase bandwidth. See Networking and GPU machines for additional details.

For detailed information, refer to Network bandwidth.

Create the VM

Google Cloud console

  1. In the Google Cloud console, go to the VM instances page.

    Go to VM instances

  2. Select your project and click Continue.

  3. Click Create instance.

  4. Enter a name for your VM in Name. For more information, see Resource naming convention.

  5. Select the Region and Zone from the drop-down menus for this VM. Your VM should be in the same zone as your Managed Lustre instance.

  6. Select a Machine configuration for your VM from the list.

  7. Click OS and storage in the left nav.

  8. Under Operating system and storage, click Change.

  9. From the Operating system drop-down, select one of:

    • HPC VM image for Rocky 8
    • Rocky Linux for Rocky 9
    • Red Hat Enterprise Linux for RHEL 9
    • Ubuntu for Ubuntu 22.04 LTS or 24.04 LTS

  10. From the Version drop-down, select one of:

    • HPC Rocky Linux 8
    • Rocky Linux 9
    • Red Hat Enterprise Linux 9
    • Ubuntu 22.04 LTS
    • Ubuntu 24.04 LTS

    Select either the x86/64 version or the Arm64 version to match your machine type. Ubuntu 24.04 LTS is only supported on Arm VMs.

  11. Select a Disk type compatible with your machine series, such as Hyperdisk Balanced for 4th-generation machine families like c4a or Balanced Persistent Disk for standard families like c2d, then click Select.

  12. From the left nav, click Networking.

  13. Select the VPC network you created in Configure a VPC network and the primary subnet (SUBNET_NAME) as the primary interface.

  14. (Optional) For Multi-NIC, click Add network interface and select the same VPC network and a different subnet (e.g., SUBNET_NAME_2).

  15. (Optional) To enable Tier_1 networking, under Network performance configurations, set Network bandwidth tier to Tier 1. If Tier_1 networking is not supported for your chosen machine type, leave this at the default setting.

  16. From the left nav, click Security.

  17. Under Access scopes, select Allow full access to all Cloud APIs.

  18. From the left nav, click Advanced.

  19. Under Automation, enter the following in the Startup script field:

    #!/bin/bash
sudo apt-get update
sudo apt-get install -y lustre-client-modules-$(uname -r)

    This script ensures that the Lustre client packages are updated if the Ubuntu kernel is upgraded.

  20. To create and start the VM, click Create.

gcloud

Use the gcloud command line tool to create a VM. When customizing the commands:

  • If your machine type doesn't support Tier_1 networking, omit --network-performance-configs=total-egress-bandwidth-tier=TIER_1 from the command.

  • Set type=hyperdisk-balanced for 4th-generation machine types such as c4a or n4, or type=pd-balanced for standard machine types such as c2d or c2.

HPC Rocky Linux 8

Create a VM using the gcloud compute instances create command. You can update the machine type and any disk specifications before running the command.

gcloud compute instances create VM_NAME \
  --project=PROJECT_ID \
  --zone=LOCATION \
  --machine-type=c2d-standard-112 \
  --scopes="https://www.googleapis.com/auth/cloud-platform" \
  --network-interface=stack-type=IPV4_ONLY,subnet=SUBNET_NAME,nic-type=GVNIC \
  --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \
  --create-disk=auto-delete=yes,boot=yes,device-name=VM_NAME,\
image-project=cloud-hpc-image-public,image-family=hpc-rocky-linux-8,\
mode=rw,size=100,type=pd-balanced

If your Compute Engine machine type has multiple physical network interface cards (NICs), assign each NIC to a subnet. This lets you aggregate bandwidth across multiple network interfaces.

To do so, add an additional --network-interface flag for each additional NIC, and specify a different subnet. See