You can use GPUs on Compute Engine to accelerate specific workloads onyour VMs such as machine learning (ML) and data processing. To use GPUs, youcan either deploy an accelerator-optimized VM that has attached GPUs, orattach GPUs to an N1 general-purpose VM.
Compute Engine provides GPUs for your VMs in passthrough mode so thatyour VMs have direct control over the GPUs and their associated memory.
For more information about GPUs on Compute Engine, seeAbout GPUs.
If you have graphics-intensive workloads, such as 3D visualization,3D rendering, or virtual applications, you can use NVIDIA RTX virtualworkstations (formerly known as NVIDIA GRID).
This document provides an overview of the different GPU VMs that areavailable on Compute Engine.
To view available regions and zones for GPUs on Compute Engine, seeGPUs regions and zone availability.
GPUs for compute workloads
For compute workloads, GPUs are supported for the following machine types:
- A3 VMs: these VMs have NVIDIA H100 80GB GPUs automatically attached.
- A2 VMs: these VMs have either NVIDIA A100 80GB or NVIDIA A100 40GBGPUs automatically attached.
- G2 VMs: these VMs have NVIDIA L4 GPUs automatically attached.
- N1 VMs: for these VMs, you can attach the following GPU models:NVIDIA T4, NVIDIA V100, NVIDIA P100, or NVIDIA P4.
A3 machine series
To run NVIDIA H100 80GB GPUs, you must use anA3 accelerator-optimizedmachine. Each A3 machine type has a fixed GPU count, vCPU count, and memory size.
A3 machine series are available in two types:
- A3 High (
a3-highgpu-8g
): these machine types have H100 80GB GPUs(nvidia-h100-80gb
) and Local SSD disks attached. In addition to the200Gbps of VM to VM network bandwidth for all A3 VMs, A3 High VMsprovide 800Gbps of GPU to GPU bandwidth, leading to a total maximumnetwork bandwidth speed of 1,000Gbps. - A3 Mega (
a3-megagpu-8g
): these machine types have H100 80GB Mega GPUs(nvidia-h100-mega-80gb
) and Local SSD disks attached. In addition to the200Gbps of VM to VM network bandwidth for all A3 VMs, A3 Mega VMsprovide 1,600Gbps of GPU to GPU bandwidth, leading to a total maximumnetwork bandwidth speed of 1,800Gbps.
Machine type | GPU count | GPU memory* (GB HBM3) | vCPU count | VM memory (GB) | Attached Local SSD (GiB) | Maximum network bandwidth (Gbps) | |
---|---|---|---|---|---|---|---|
VM to VM | GPU cluster | ||||||
a3-highgpu-8g | 8 | 640 | 208 | 1,872 | 6,000 | 200 | 800 |
a3-megagpu-8g | 8 | 640 | 208 | 1,872 | 6,000 | 200 | 1,600 |
*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.
A2 machine series
To use NVIDIA A100 GPUs onGoogle Cloud, you must deploy anA2 accelerator-optimizedmachine. Each A2 machine type has a fixed GPU count, vCPU count, and memory size.
A2 machine series are available in two types:
- A2 Standard: these machine types have A100 40GB GPUs (
nvidia-tesla-a100
)attached. - A2 Ultra: these machine types have A100 80GB GPUs (
nvidia-a100-80gb
) andLocal SSD disks attached.
A2 Standard
Machine type | GPU count | GPU memory* (GB HBM2) | vCPU count | VM memory (GB) | Local SSD supported | Maximum network bandwidth (Gbps) |
---|---|---|---|---|---|---|
a2-highgpu-1g | 1 | 40 | 12 | 85 | Yes | 24 |
a2-highgpu-2g | 2 | 80 | 24 | 170 | Yes | 32 |
a2-highgpu-4g | 4 | 160 | 48 | 340 | Yes | 50 |
a2-highgpu-8g | 8 | 320 | 96 | 680 | Yes | 100 |
a2-megagpu-16g | 16 | 640 | 96 | 1,360 | Yes | 100 |
A2 Ultra
Machine type | GPU count | GPU memory* (GB HBM2e) | vCPU count | VM memory (GB) | Attached Local SSD (GiB) | Maximum network bandwidth (Gbps) |
---|---|---|---|---|---|---|
a2-ultragpu-1g | 1 | 80 | 12 | 170 | 375 | 24 |
a2-ultragpu-2g | 2 | 160 | 24 | 340 | 750 | 32 |
a2-ultragpu-4g | 4 | 320 | 48 | 680 | 1,500 | 50 |
a2-ultragpu-8g | 8 | 640 | 96 | 1,360 | 3,000 | 100 |
*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.
G2 machine series
To use NVIDIA L4 GPUs(nvidia-l4
or nvidia-l4-vws
), you must deploy aG2 accelerator-optimizedmachine.
Each G2 machine type has a fixed number of NVIDIA L4 GPUsand vCPUs attached. Each G2 machine type also has a default memory and a custommemory range. The custom memory range defines the amount of memory thatyou can allocate to your VM for each machine type. You can specify your custommemory during VM creation.
Machine type | GPU count | GPU memory* (GB GDDR6) | vCPU count | Default VM memory (GB) | Custom VM memory range (GB) | Max Local SSD supported (GiB) | Maximum network bandwidth (Gbps) |
---|---|---|---|---|---|---|---|
g2-standard-4 | 1 | 24 | 4 | 16 | 16 to 32 | 375 | 10 |
g2-standard-8 | 1 | 24 | 8 | 32 | 32 to 54 | 375 | 16 |
g2-standard-12 | 1 | 24 | 12 | 48 | 48 to 54 | 375 | 16 |
g2-standard-16 | 1 | 24 | 16 | 64 | 54 to 64 | 375 | 32 |
g2-standard-24 | 2 | 48 | 24 | 96 | 96 to 108 | 750 | 32 |
g2-standard-32 | 1 | 24 | 32 | 128 | 96 to 128 | 375 | 32 |
g2-standard-48 | 4 | 96 | 48 | 192 | 192 to 216 | 1,500 | 50 |
g2-standard-96 | 8 | 192 | 96 | 384 | 384 to 432 | 3,000 | 100 |
*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.
N1 machine series
You can attach the following GPU models to anN1 machine type with theexception of the N1 shared-core machine type.
N1 VMs with lower numbers of GPUs are limited to a maximum number of vCPUs.In general, a higher number of GPUs lets you create VM instances with a highernumber of vCPUs and memory.
N1+T4 GPUs
You can attach NVIDIA T4GPUs to N1 general-purpose VMs with the following VM configurations.
Accelerator type | GPU count | GPU memory* (GB GDDR6) | vCPU count | VM memory (GB) | Local SSD supported |
---|---|---|---|---|---|
nvidia-tesla-t4 or nvidia-tesla-t4-vws | 1 | 16 | 1 to 48 | 1 to 312 | Yes |
2 | 32 | 1 to 48 | 1 to 312 | Yes | |
4 | 64 | 1 to 96 | 1 to 624 | Yes |
*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.
N1+P4 GPUs
You can attachNVIDIA P4GPUs to N1 general-purpose VMs with the following VM configurations.
Accelerator type | GPU count | GPU memory* (GB GDDR5) | vCPU count | VM memory (GB) | Local SSD supported† |
---|---|---|---|---|---|
nvidia-tesla-p4 or nvidia-tesla-p4-vws | 1 | 8 | 1 to 24 | 1 to 156 | Yes |
2 | 16 | 1 to 48 | 1 to 312 | Yes | |
4 | 32 | 1 to 96 | 1 to 624 | Yes |
*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.
†For VMs with attached NVIDIA P4 GPUs, Local SSD disksare only supported in zones us-central1-c
andnorthamerica-northeast1-b
.
N1+V100 GPUs
You can attachNVIDIA V100GPUs to N1 general-purpose VMs with the following VM configurations.
Accelerator type | GPU count | GPU memory* (GB HBM2) | vCPU count | VM memory (GB) | Local SSD supported† |
---|---|---|---|---|---|
nvidia-tesla-v100 | 1 | 16 | 1 to 12 | 1 to 78 | Yes |
2 | 32 | 1 to 24 | 1 to 156 | Yes | |
4 | 64 | 1 to 48 | 1 to 312 | Yes | |
8 | 128 | 1 to 96 | 1 to 624 | Yes |
*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.
†For VMs with attached NVIDIA V100 GPUs, Local SSD disksaren't supported in us-east1-c
.
N1+P100 GPUs
You can attachNVIDIA P100GPUs to N1 general-purpose VMs with the following VM configurations.
For some NVIDIA P100 GPUs, the maximum CPU and memory that is available forsome configurations is dependent on the zone in which the GPU resource is running.
Accelerator type | GPU count | GPU memory* (GB HBM2) | vCPU count | VM memory (GB) | Local SSD supported |
---|---|---|---|---|---|
nvidia-tesla-p100 or nvidia-tesla-p100-vws | 1 | 16 | 1 to 16 | 1 to 104 | Yes |
2 | 32 | 1 to 32 | 1 to 208 | Yes | |
4 | 64 | 1 to 64 1 to 96 | 1 to 208 1 to 624 | Yes |
*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.
NVIDIA RTX Virtual Workstations (vWS) for graphics workloads
If you have graphics-intensive workloads, such as 3D visualization, you cancreate virtual workstations that useNVIDIA RTX Virtual Workstations (vWS) (formerly known as NVIDIA GRID). When you create a virtualworkstation, an NVIDIA RTX Virtual Workstation (vWS) license is automatically addedto your VM.
For information about pricing for virtual workstations, seeGPU pricing page.
For graphics workloads, NVIDIA RTX virtual workstation (vWS) models are available:
G2 machine series: for G2 machine types you can enableNVIDIA L4 Virtual Workstations (vWS):
nvidia-l4-vws
N1 machine series: for N1 machine types, you can enable the followingvirtual workstations:
- NVIDIA T4 Virtual Workstations:
nvidia-tesla-t4-vws
- NVIDIA P100 Virtual Workstations:
nvidia-tesla-p100-vws
- NVIDIA P4 Virtual Workstations:
nvidia-tesla-p4-vws
- NVIDIA T4 Virtual Workstations:
General comparison chart
The following table describes the GPU memory size, feature availability,and ideal workload types of different GPU models that are available onCompute Engine.
GPU model | GPU memory | Interconnect | NVIDIA RTX Virtual Workstation (vWS) support | Best used for |
---|---|---|---|---|
H100 80GB | 80 GB HBM3 @ 3.35 TBps | NVLink Full Mesh @ 900 GBps | Large models with massive data tables for ML Training, Inference, HPC, BERT, DLRM | |
A100 80GB | 80 GB HBM2e @ 1.9 TBps | NVLink Full Mesh @ 600 GBps | Large models with massive data tables for ML Training, Inference, HPC, BERT, DLRM | |
A100 40GB | 40 GB HBM2 @ 1.6 TBps | NVLink Full Mesh @ 600 GBps | ML Training, Inference, HPC | |
L4 | 24 GB GDDR6 @ 300 GBps | N/A | ML Inference, Training, Remote Visualization Workstations,Video Transcoding, HPC | |
T4 | 16 GB GDDR6 @ 320 GBps | N/A | ML Inference, Training, Remote Visualization Workstations, Video Transcoding | |
V100 | 16 GB HBM2 @ 900 GBps | NVLink Ring @ 300 GBps | ML Training, Inference, HPC | |
P4 | 8 GB GDDR5 @ 192 GBps | N/A | Remote Visualization Workstations, ML Inference, and Video Transcoding | |
P100 | 16 GB HBM2 @ 732 GBps | N/A | ML Training, Inference, HPC, Remote Visualization Workstations |
To compare GPU pricing for the different GPU models and regions that areavailable on Compute Engine, see GPU pricing.
Performance comparison chart
The following table describes the performance specifications of different GPUmodels that are available on Compute Engine.
Compute performance
GPU model | FP64 | FP32 | FP16 | INT8 |
---|---|---|---|---|
H100 80GB | 34 TFLOPS | 67 TFLOPS | ||
A100 80GB | 9.7 TFLOPS | 19.5 TFLOPS | ||
A100 40GB | 9.7 TFLOPS | 19.5 TFLOPS | ||
L4 | 0.5 TFLOPS* | 30.3 TFLOPS | ||
T4 | 0.25 TFLOPS* | 8.1 TFLOPS | ||
V100 | 7.8 TFLOPS | 15.7 TFLOPS | ||
P4 | 0.2 TFLOPS* | 5.5 TFLOPS | 22 TOPS† | |
P100 | 4.7 TFLOPS | 9.3 TFLOPS | 18.7 TFLOPS |
*To allow FP64 code to work correctly, a small number of FP64 hardware units are included in the T4, L4, and P4 GPU architecture.
†TeraOperations per Second.
Tensor core performance
GPU model | FP64 | TF32 | Mixed-precision FP16/FP32 | INT8 | INT4 | FP8 |
---|---|---|---|---|---|---|
H100 80GB | 67 TFLOPS | 989 TFLOPS† | 1,979 TFLOPS*, † | 3,958 TOPS† | 3,958 TFLOPS† | |
A100 80GB | 19.5 TFLOPS | 156 TFLOPS | 312 TFLOPS* | 624 TOPS | 1248 TOPS | |
A100 40GB | 19.5 TFLOPS | 156 TFLOPS | 312 TFLOPS* | 624 TOPS | 1248 TOPS | |
L4 | 120 TFLOPS† | 242 TFLOPS*, † | 485 TOPS† | 485 TFLOPS† | T4 | 65 TFLOPS | 130 TOPS | 260 TOPS |
V100 | 125 TFLOPS | |||||
P4 | ||||||
P100 |
*For mixed precision training, NVIDIA H100, A100, and L4 GPUsalso support the bfloat16
data type.
†For H100 and L4 GPUs, structural sparsity is supported which youcan use to double the performance value. The values shown are withsparsity. Specifications are one-half lower without sparsity.
What's next?
- For more information about GPUs on Compute Engine,see About GPUs.
- Review the GPU regions and zones availability.
- Learn about GPU pricing.