Which GPU Instances Are Best on Google Cloud in 2026? A Practical Guide by Workload

Key points:

• Frontier LLM training: pick A4 (NVIDIA B200) or A4X (GB200 NVL72), or A3 Ultra (H200). The A3 Mega 8x H100 shape is still the most widely available Hopper configuration, and its GPUDirect-TCPXO networking (up to 1,800 Gbps per node) is what makes multi-node training scale.
• Inference and serving: G2 with the NVIDIA L4 (24GB) is the cheapest sane default for models that fit in 24GB after quantization, at roughly $0.70 per GPU-hour on-demand in us-central1, about an order of magnitude below H100-class machines.
• Fine-tuning and mid-size training: A2 with the A100 40GB or 80GB is the pragmatic middle ground. Broader regional availability and far cheaper per GPU-hour than Hopper or Blackwell.
• Cheapest GPU on Google Cloud: an N1 instance with an attached NVIDIA T4 (16GB), around $0.35 per GPU-hour. Fine for notebooks, CI, small-model inference and video transcoding, useless for training a modern LLM.
• The instance choice is the easy half. Per-region, per-GPU quota, the gap between on-demand and Spot/reservations/Committed Use Discounts/Dynamic Workload Scheduler, and idle GPU time decide your real bill. Qovery deploys GPU workloads into your own GCP project or existing GKE cluster (BYOC), so every discount and commitment stays in your name.

---

What are the best GPU instances on Google Cloud Platform right now?

On Google Cloud, pick A4 (B200) or A4X (GB200 NVL72) and A3 Ultra (H200) for frontier LLM training, A3 Mega for practical 8x H100 training, A2 (A100) for fine-tuning, G2 (L4) for inference, and an N1 with an attached T4 for dev, notebooks, CI and cheap batch. The biggest GPU is almost always the wrong answer for inference, so match the machine family to the workload, not to the top of the spec sheet.

The first thing that trips people up is the naming. Google names the machine family, not the GPU, so you have to translate. Here is the whole map:

• A4X carries the NVIDIA GB200 (Grace Blackwell), sold as the a4x-highgpu-4g shape. It went generally available on 29 May 2025 (Google Cloud).
• A4 carries the B200, sold as the 8-GPU a4-highgpu-8g.
• A3 Ultra carries the H200, sold as the 8-GPU a3-ultragpu-8g.
• A3 Mega / A3 High / A3 Edge carry the H100 80GB (a3-megagpu-8g, a3-highgpu-1g/2g/4g/8g, and the Edge shapes).
• A2 Ultra / A2 Standard carry the A100 (80GB and 40GB respectively).
• G2 carries the L4.
• G4 carries the NVIDIA RTX PRO 6000 Blackwell and is generally available (Google Cloud).
• N1 is a general-purpose family you attach older GPUs to: T4, V100, P100 or P4.

The structural difference between N1-plus-a-GPU and the accelerator-optimized A and G families matters more than the badge. On A2, A3, A4 and A4X the GPUs are wired together with NVLink, local SSD comes attached, GPU counts are fixed per shape, and the accelerator families carry serious node-to-node networking: GPUDirect-TCPX on a3-highgpu-8g, GPUDirect-TCPXO on a3-megagpu-8g, and RDMA over Converged Ethernet on A3 Ultra and A4 (Google Cloud accelerator-optimized machines). An N1 with a bolted-on T4 has none of that. It is a general VM with a single accelerator, which is exactly what you want for a notebook and exactly what you do not want for distributed training.

Use VRAM as your first filter, because it decides what fits before performance decides how fast. Per NVIDIA's own datasheets: T4 is 16GB, L4 is 24GB, V100 is 16 or 32GB, A100 is 40 or 80GB, H100 is 80GB (NVIDIA H100), H200 is 141GB (NVIDIA H200), and the B200 is 180GB in the DGX B200 configuration (NVIDIA DGX B200). The Blackwell GPU inside GB200 sits around 186GB per GPU, derived from the 13.4TB across 72 GPUs that NVIDIA publishes for the NVL72 rack (NVIDIA GB200 NVL72). If your model plus its optimizer state and KV cache does not fit, no amount of raw throughput helps you.

One more option worth a single line: Google also sells TPUs (v5e, v5p, Trillium) as an alternative accelerator. A TPU tends to beat a GPU when you are doing large-scale training on JAX or XLA-friendly models and can commit to Google's software stack. For everything portable, containerized, and PyTorch-shaped, a GPU is the safer default, and that is what the rest of this guide is about.

How do Google Cloud GPU instances compare on price, VRAM and workload fit?

Here is the whole GPU lineup in one table, with the number nobody else publishes: dollars per hour per 100GB of VRAM, computed from the on-demand list prices so you can compare a cheap 24GB card to an expensive 141GB card on the same axis.

A few things to read off this table, and one honest caveat on where the numbers come from. Google renders its price tables in JavaScript that changes often, so treat every dollar figure here as the us-central1 on-demand list level as of early September 2026 and confirm the live number on the Compute Engine GPU pricing page before you budget. The A3 High and A3 Mega H100 shapes are consistently listed around $87.83 for the full 8-GPU node, which is the ~$10.98 per GPU-hour above. Blackwell (A4, A4X) is generally available but is largely allocated through reservations, Dynamic Workload Scheduler, or Spot rather than a stable on-demand SKU, which is why I have not printed a per-hour number I cannot stand behind. For a real Blackwell price you can quote today, jump to the cross-cloud table further down, where Lambda, CoreWeave and RunPod publish B200 rates on their own pages.

The normalized column is the one I would tape to the wall. The math is simply on-demand price divided by VRAM, times 100. On that measure the T4 and L4 are the cheapest way to rent a gigabyte of accelerator memory, the H100 is the most expensive at about $13.73 per 100GB-hour, and here is the counterintuitive part: the H200 comes in cheaper per gigabyte of VRAM than the H100 (~$7.52 versus ~$13.73), because you pay roughly the same per GPU-hour for 141GB instead of 80GB. If your bottleneck is memory rather than compute, the newer card is the cheaper one.

Two more warnings the SKU price hides. First, the GPU is only part of the bill: vCPU, RAM, local SSD, persistent disk and network egress are billed separately on the A and G families, and several A3 shapes ship with mandatory local SSD you cannot remove. Second, a3-megagpu-8g, a3-ultragpu-8g and a4-highgpu-8g are sold only as 8-GPU VMs. You cannot rent a single H100, H200 or B200 in those shapes, so the minimum spend to run one experiment is eight GPUs at once. If you need one Hopper GPU, that is what a3-highgpu-1g is for.

Which Google Cloud GPU should you pick for training vs fine-tuning vs inference?

Match the GPU to the job in three moves: pretraining and 70B-plus training go to A4/A4X or A3 Ultra multi-node, fine-tuning in the 7B to 34B range goes to A2 Ultra 80GB or a single 8x H100 node, and inference goes to G2/L4 for anything that fits in 24GB after quantization, escalating to A3 only when context length or model size forces it.

Training frontier models. Once you cross a single node, inter-node bandwidth decides your throughput more than raw FLOPS, because gradients have to synchronize every step. This is why Google publishes the networking so prominently: a3-megagpu-8g delivers up to 1,800 Gbps of GPU-to-GPU bandwidth over GPUDirect-TCPXO, and A3 Ultra and A4 push up to 3,600 Gbps over RDMA (Google Cloud accelerator-optimized machines). A3 High tops out around 1,000 Gbps on GPUDirect-TCPX, which is why A3 Mega, not A3 High, is the shape people reach for when a training run spans many nodes. On the silicon itself, the H100 and H200 share the same compute die at 989 BF16 dense TFLOPS and 1,979 FP8 dense TFLOPS per NVIDIA's datasheets (NVIDIA H100); the H200's advantage is memory, 141GB of HBM3e at 4.8 TB/s versus the H100's 80GB of HBM3 at 3.35 TB/s (NVIDIA H200).

Fine-tuning and LoRA. Size the memory yourself with a rule of thumb instead of guessing. For inference or frozen weights, memory is roughly parameters times bytes per parameter: 2 bytes for BF16, 1 byte for FP8/INT8, 0.5 for INT4. For a full fine-tune with the Adam optimizer in mixed precision, budget around 16 bytes per parameter: 2 for the weights, 2 for the gradients, plus roughly 12 for the fp32 optimizer states and master copy, and then add activations and KV cache on top. Worked out:

• 7B full fine-tune: ~112GB of optimizer and weight state (16 x 7). That does not fit one 80GB card, so you want a single 8x A100/H100 node with sharding (FSDP or ZeRO), or you drop to LoRA.
• 13B full fine-tune: ~208GB. A full 8-GPU node handles it comfortably with sharding.
• 70B full fine-tune: ~1,120GB of state. An 8x H100 node gives you 640GB of HBM, which does not fit it without offload; an 8x H200 node gives you 1,128GB, which does. That single fact is the best argument for A3 Ultra over A3 Mega on large fine-tunes.
• LoRA / QLoRA: freeze and quantize the base weights and you only train small adapters. A 7B QLoRA fits on a single L4 24GB, a 13B QLoRA is comfortable on an A100 40GB, and a 70B QLoRA with 4-bit weights (about 35GB) fits on a single A100 80GB or H100 80GB.

Inference and serving. For anything under about 13B after quantization, an L4 on G2 is the default, and it is the cheapest capable card per gigabyte of VRAM in the whole lineup. The L4 does 121 BF16 dense TFLOPS and supports FP8 at 242 TFLOPS (NVIDIA L4), which is plenty for small-to-mid models. What actually drives your cost per token is batching, concurrency and KV-cache size, not the GPU badge. Reach for H100 or H200 only when context length or model size forces it. The KV cache grows with batch size times context length times layers, so at long context it can eat tens of gigabytes, and that is precisely why the 141GB H200 wins on long-context serving where an 80GB H100 starts paging. When you compare throughput, anchor it to NVIDIA datasheets and to published MLPerf Inference and MLPerf Training results rather than marketing decks. In the MLPerf Inference v5.1 round, NVIDIA reported Blackwell delivering more than 5x Hopper performance on Llama 3.1 405B interactive (NVIDIA).

Dev, CI and batch. For notebooks, CI runners, preview environments and video transcoding, an N1 with a T4 is the cheapest entry point on Google Cloud. The T4 is a Turing card with no BF16 or FP8 support, so it is useless for training a modern LLM, but for smoke-testing a container or serving a tiny model it is hard to beat on price.

How much do Google Cloud GPUs actually cost, and how do you cut the bill?

On-demand list price is the worst price you will ever pay for a Google Cloud GPU, and Spot, Dynamic Workload Scheduler, reservations and Committed Use Discounts are where the real savings live. Get the pricing model right and you can cut a GPU bill by more than half without changing a line of code.

Here is how the models actually work, with the published mechanics:

• Spot VMs advertise up to 91% off on-demand across many machine types, GPUs and Local SSDs (Google Cloud Spot). They have no maximum runtime, but they can be preempted at any time with about 30 seconds of shutdown notice. Great for interruption-tolerant training with frequent checkpointing, wrong for a customer-facing endpoint.
• Committed Use Discounts come in two flavors. Resource-based CUDs apply to the accelerator-optimized families and go up to roughly 55% off for most GPUs and up to 65% for some, with the three-year term deeper than the one-year (Google Cloud CUDs). Flexible CUDs, among GPU families, apply only to G2 and G4: G2 (L4) is 21% for one year and 43% for three, G4 is 16% and 42%.
• Reservations (on-demand and future reservations) hold capacity for you so a launch does not fail on a stockout, and they stack with CUDs for the discount.
• Dynamic Workload Scheduler has two modes (Google Cloud). Flex Start queues your request and provisions the GPUs when capacity appears, for runs up to 7 days, and you pay only for what you consume. Calendar mode lets you reserve fixed 7- or 14-day capacity blocks up to 8 weeks ahead for a dated training run.

The single biggest waste is not the price per hour, it is idle GPU time. A GPU you reserved and left running overnight bills the same as one doing real work. Three fixes matter: autoscale GPU node pools to zero when nothing is queued, auto-stop non-production GPU environments outside working hours, and share a single GPU across small workloads with GKE time-slicing or MIG (GKE time-sharing). MIG carves a supported GPU into up to seven hardware-isolated slices; time-slicing multiplexes one GPU across pods with no memory isolation and some context-switch overhead, so use MIG when you need isolation and time-slicing when you just need to pack dev workloads.

The math is stark once you run it. Take an 8x H100 a3-highgpu-8g node at roughly $87.83 per hour on-demand. Running it 24/7 for a 730-hour month is about $64,000. The same node on Spot, at the roughly $3.69 per GPU-hour that shows up around Google's Spot tier, lands near $21,500 for a month, if you can tolerate preemption. And the same node run only 6 hours a day on-demand is about $16,000. Same silicon, a 4x swing in the invoice, decided entirely by scheduling.

Be fair about the alternative: specialized GPU clouds routinely undercut hyperscaler on-demand $/GPU-hour on raw compute, and I get to that in the cross-cloud table. If you are optimizing purely for the cheapest H100-hour and can live without the surrounding platform, that is a real option worth cross-checking. Just watch the line items the sticker hides: mandatory local SSD on some A3 shapes, cross-zone traffic during multi-node training, persistent disk and Cloud Storage for datasets, and the premium Vertex AI adds over raw Compute Engine.

Why can't you get the H100 or B200 you want? Quota, regions and availability

On Google Cloud the binding constraint is almost never price, it is per-region, per-GPU-type quota and physical capacity in the zone, and both have a defined path. If your gcloud command fails, the first thing to check is your quota, and the second is whether the zone has stock at all.

Quotas are scoped per project, per region and per GPU model, and there is a separate quota for Spot/preemptible GPUs on top of the standard one (Google Cloud resource usage). New projects typically start at zero for A100, H100, H200 and B200, so a fresh project cannot launch a Hopper node until you ask. To raise it, in the Google Cloud console go to IAM & Admin > Quotas & System Limits, filter by the GPU model and region, select the quota, and submit an increase with a written justification (Google Cloud quotas). Small bumps often approve automatically on an established billing account; large accelerator requests usually route through your account team or a capacity request, and Free Trial billing accounts cannot request increases at all.

Region and zone availability is the other half. Google publishes exactly which zones carry each GPU on its GPU regions and zones page, and it is worth reading rather than guessing, because the footprint is uneven. At the time of writing, H100 (A3 Mega and A3 High) spans several US regions plus europe-west1 and parts of Asia-Pacific, H200 (A3 Ultra) is in a narrower set including us-central1, us-east4, us-west1, europe-west1, europe-west4 and asia-south1/2, and B200/GB200 (A4/A4X) shows up across a spread of US zones plus europe-north1, europe-west4 and a couple of Asia-Pacific zones. A100 and L4 are far more broadly available, and T4 is nearly everywhere. Check the live page for your specific region before you architect around it.

When capacity is gone, you will see ZONE_RESOURCE_POOL_EXHAUSTED. That is a stockout signal, not a configuration bug (Google Cloud troubleshooting). The sanctioned ways to guarantee capacity are reservations, future reservations and DWS Calendar mode for a dated run, or DWS Flex Start for queued, interruption-tolerant jobs. The practical fallbacks when you just need GPUs now: submit the same job across several regions, run on Spot with frequent checkpointing to Cloud Storage, drop to A2 or L4 with quantization, or split a run into smaller shapes. GKE node auto-provisioning and Autopilot GPU workloads handle a lot of this plumbing for you, including requesting the right node shape and cleaning it up when the job ends.

Should you run GCP GPUs on Compute Engine, GKE, Vertex AI, or a platform on top?

Use raw Compute Engine for one-off experiments, Vertex AI when you have no platform engineers and accept a managed premium, GKE when you need control, multi-tenancy and GPU sharing, and an internal developer platform on top of GKE when several teams need self-service GPU environments without owning the cluster. The decision is really about who you want owning the operational surface.

Compute Engine gives you full control and no abstraction. You own the NVIDIA drivers, the CUDA images, the health checks and the autoscaling logic. That is the right trade for a quick experiment or a tightly controlled single service, and the wrong trade when five teams start filing tickets to launch GPUs.

GKE is where most serious GPU work lands. With the NVIDIA GPU Operator or the managed device plugin you get GPU node pools, taints and tolerations to keep the wrong pods off expensive nodes, the cluster autoscaler and node auto-provisioning to scale pools to zero, time-slicing and MIG for sharing, and DWS integration for queued jobs. GKE flex-start on DWS advertises up to a 53% discount on the GPUs, TPUs and Local SSD it provisions (GKE DWS). The honest cost is that you own all of it, including upgrades and the day it breaks.

Vertex AI Training and Prediction is the fastest way to start, with managed serving and tuning and no cluster to run. The trade-off is money: Vertex bills the underlying accelerator plus a management fee, so it runs above the equivalent Compute Engine GPU price. Google does not print a single clean premium number, so compare the accelerator line on the Vertex AI pricing page against the raw Compute Engine GPU price for the same card and decide whether the managed convenience is worth it for your team.

Where Qovery fits. Qovery is a BYOC internal developer platform that deploys into your own GCP project or your existing GKE cluster, so the GPU node pools, reservations, CUDs and Spot savings all stay under your own billing account. What that buys your team: git-push deployments, a preview environment per pull request, environment auto-stop for non-production, managed cluster upgrades, per-environment RBAC, and databases backed by managed cloud services. The idle-GPU angle is the concrete win here: auto-stop on a non-production GPU environment is the cheapest FinOps lever there is, because a stopped environment bills nothing for GPU time. To be clear about what Qovery is not, it does not sell, resell or broker GPU capacity, and it does not replace Vertex AI's model-training APIs. It is the deployment and governance layer on top of the Kubernetes you already pay for. And because it is cloud-agnostic, the same setup targets AWS, Azure, Scaleway or your own self-managed Kubernetes, which matters the week your GCP region runs dry.

How does Google Cloud compare to AWS, Azure and specialized GPU clouds on GPUs?

The NVIDIA silicon is the same everywhere, so the real differences are capacity model, networking and price: Google Cloud leads on capacity scheduling (DWS, reservations) and GKE maturity, the specialized clouds usually win on raw $/GPU-hour, and AWS and Azure win on surrounding services and enterprise agreements. If you already run on one hyperscaler, the surrounding services usually decide it before price does.

The equivalent shapes map cleanly. Google's A3 Mega / A3 Ultra / A4 line up against AWS P5 / P5e and the Blackwell P6-B200 (AWS), and against Azure's ND H100 v5, ND H200 v5 and ND GB200 v6 series (Microsoft Learn). For inference, Google's G2 (L4) lines up with AWS G6 (L4) (AWS) and Azure's NC A100 v4 family sits alongside A2 for mid-range work.

On published on-demand H100 pricing, the specialized clouds are simply cheaper, and I will not pretend otherwise. As of early September 2026, RunPod lists the H100 SXM at $2.69 per GPU-hour on its Community tier and $3.29 Secure (RunPod), Lambda lists the H100 SXM at $3.99 in an 8-GPU node (Lambda), and CoreWeave lists HGX H100 at $6.16 per GPU-hour (CoreWeave). The hyperscalers sit higher on the on-demand sticker: Google around $10.98, AWS around $7 on the p5 line (AWS P5), and Azure roughly $12 to $16 depending on region (Azure). Those hyperscaler numbers are pre-discount, so compare them against your actual Savings Plan, reservation or CUD rate rather than the list, but even after discounts a RunPod or Lambda H100 often comes in lower on raw compute.

Where the hyperscalers earn their price is the capacity model, which matters more than the sticker for anyone running a dated training job. Google gives you reservations plus DWS Flex Start and Calendar. AWS gives you Capacity Blocks for ML, where you reserve GPU capacity for a fixed future window up to 8 weeks out, with a worked example on AWS's own page landing an effective p5.48xlarge around $31.46 per hour for a 48-hour block (AWS Capacity Blocks). Azure governs GPU capacity through per-family vCPU quota plus 1- and 3-year reservations. If you need eight H100s on a specific Tuesday, these mechanisms, not the hourly rate, are what determine whether you actually get them.

The portability point is short and factual: containers plus Kubernetes mean the workload is not welded to one provider's GPU queue. Standardize on the nvidia.com/gpu resource with the GPU Operator and the same manifests run on GKE, EKS, AKS or a specialized cloud's Kubernetes. That is exactly the seam Qovery sits in, so the same application can target GCP, AWS, Azure, Scaleway or an existing cluster depending on who has capacity at the right price this month.

The short version

If I had to compress this to a paragraph: G2 (L4) for inference and QLoRA, A2 (A100) for fine-tuning and mid-size training, A3 Mega for practical multi-node H100 training, A3 Ultra (H200) for long-context serving and large fine-tunes, and A4/A4X (B200/GB200) when you are training at the frontier and can get the capacity. Then spend your real effort on the parts that decide the bill: quota and reservations up front, Spot and CUDs for the discount, and auto-stop plus scale-to-zero so idle GPUs stop burning money. Run all of it in your own GCP project so the discounts stay in your name, and keep the deployment workflow cloud-agnostic so a stockout in one region is an inconvenience, not an outage.