8 Best Azure GPU VMs for AI Training in 2026: ND H100, H200, MI300X, GB200 and GB300 Compared

Key points:

• The best Azure GPU VM for most AI training in 2026 is the ND H200 v5. Eight H200 GPUs with 141 GB each, 400 Gb/s InfiniBand per GPU, and a list price of $84.80/hour in East US 2 (about $10.60 per GPU-hour), which is cheaper than the ND H100 v5 at $98.32/hour, per the Azure Retail Prices API (checked 14 November 2026).
• The ND MI300X v5 is the cheapest memory on Azure. Eight AMD MI300X GPUs with 192 GB each for $48.00/hour in East US 2 ($6.00 per GPU-hour). Pick it if your stack runs on ROCm. Skip it if your code depends on CUDA-only kernels.
• ND GB200 v6 and ND GB300 v6 are for frontier-scale training. GB200 lists at $108.16 per 4-GPU VM in East US. GB300 has no public price meter yet, so capacity goes through your Microsoft account team.
• For single-node fine-tuning, use the NC H100 v5 or NC A100 v4. One H100 NVL (94 GB) costs $6.98/hour in East US; one A100 80 GB costs $3.673/hour. No InfiniBand, so they do not scale across nodes for training.
• A price meter is not capacity. ND H100 v5 has Linux meters in 24 regions and ND H200 v5 in 31, but GPU quota per VM family starts small. Request quota before you plan a run.
• On AKS, the VM choice is half the work. Drivers, InfiniBand, node taints and scaling to zero decide what you really pay. Qovery runs your training jobs on AKS GPU node pools in your own Azure subscription, so reservations and MACC drawdown stay yours.

The best Azure GPU VM for AI training in 2026 depends on one question first: does your model fit on one node, or do you need InfiniBand across many? Get that wrong and you pay 8-GPU prices for a job that one H100 could run, or you buy PCIe cards that cannot talk to each other fast enough to train anything big.

Teams with an Azure commitment keep asking me the same thing: which ND or NC size, in which region, at what price? Microsoft's answer is spread over a dozen size pages, and the pricing page renders through JavaScript. So I pulled every number here from the Azure Retail Prices API and the Microsoft Learn size pages on 14 November 2026, with links so you can re-check them.

How did we pick the best Azure GPU VMs for AI training?

We ranked every current Azure ND and NC series against six criteria that decide whether a training run finishes on time and on budget:

• GPU memory per GPU. It decides how much of the model, optimizer state and activations fit before you shard. More HBM means fewer GPUs for the same model.
• Scale-up interconnect. NVLink or Infinity Fabric bandwidth inside the VM. It matters for tensor parallelism.
• Scale-out interconnect. InfiniBand bandwidth per GPU between VMs. Without it, multi-node data parallel training crawls.
• Price per GPU-hour. Linux pay-as-you-go list price from the Azure Retail Prices API, in the cheapest US region that lists the SKU, divided by GPUs per VM.
• Region footprint. Number of Azure regions with a Linux consumption meter for the size. A meter means you can buy it there; capacity may still be short today.
• Kubernetes fit. Whether AKS installs the GPU stack for you, or you need --gpu-driver none plus an operator.

We did not rank on vendor benchmarks. Run your own model for an hour on two shortlisted sizes; it costs less than a wrong reservation.

Which Azure GPU VMs are best for AI training in 2026? The shortlist

Here is the whole shortlist in one table. Prices are Linux pay-as-you-go list prices per VM hour, checked on 14 November 2026. Your Enterprise Agreement or MACC price may differ.

In the East US pair, the H200 VM costs less per hour than the H100 VM, even though it carries 76% more HBM per GPU. And the MI300X costs less than half the H100 per GPU-hour with 2.4 times the memory. So compare price per GB of GPU memory as well as price per GPU.

It is a crude metric that ignores FLOPs, FP8/FP4 support and software maturity, but it tells you fast whether 1 TB of training state fits on one VM or three.

1. Is the ND H200 v5 the best Azure VM for AI training?

Yes, for most teams training or fully fine-tuning models in the 7B to 400B range in 2026, the ND H200 v5 is the best Azure pick. It runs the same CUDA stack as the H100, carries 141 GB of HBM per GPU, and lists below the H100 v5 in the regions where both exist. Do not pick it if you need East US specifically: the Retail Prices API shows no H200 v5 meter there.

Microsoft's ND H200 v5 size page gives the hardware: eight H200 GPUs with 141 GB and 4.8 TB/s of memory bandwidth each, NVLink at 900 GB/s inside the VM, 96 Sapphire Rapids vCPUs, 1,850 GiB of RAM and eight local NVMe disks totaling 28 TiB. Each GPU gets a dedicated 400 Gb/s Quantum-2 InfiniBand link, so a VM has 3.2 Tb/s of scale-out bandwidth and supports GPUDirect RDMA inside a scale set.

Pricing, from the Retail Prices API:

• $84.80/hour in East US 2 and West US 3
• $95.82 in Central US, $101.76 in North Europe, $106.00 in UK South and France Central, $110.27 in Sweden Central
• 31 regions with a Linux meter, the widest of any ND size

The H200 v5 "Spot" meters list at the same price as on-demand, so treat H200 as on-demand or reserved capacity, and do not build your plan around spot discounts on it.

2. When should you still pick the ND H100 v5?

Pick the ND H100 v5 when you want the most battle-tested multi-node CUDA platform on Azure, or when you can run on spot. It is the default size in most Azure HPC and AKS InfiniBand guides, and it has the deepest spot discount of the 8-GPU NVIDIA sizes. Do not pick it on-demand if the H200 v5 is available in your region, because it costs more for less memory.

Hardware per the ND H100 v5 size page: eight H100 GPUs with 80 GB each (640 GB per VM), NVLink 4.0 inside the VM, 400 Gb/s InfiniBand per GPU (3.2 Tb/s per VM), 96 vCPUs and 1,900 GiB of RAM.

Pricing from the Retail Prices API:

• $98.32/hour on-demand in East US, East US 2 and West US 3 ($12.29 per GPU-hour)
• $117.97 in South Central US, $122.90 in UK South, $127.82 in West Europe and Sweden Central
• Spot at $18.17/hour in East US, about 82% below on-demand
• 24 regions with a Linux meter

That spot price is the reason the H100 v5 stays on this list. A training job with frequent checkpoints can run on spot for a fraction of the cost. Azure gives spot VMs 30 seconds of notice before eviction, with no SLA. If you go that way, read our guide on surviving a spot GPU reclaim without losing your training run first.

3. Is the AMD ND MI300X v5 a good alternative to NVIDIA on Azure?

Yes, if your training stack runs on PyTorch with ROCm and your bottleneck is memory. The ND MI300X v5 gives you 192 GB per GPU and 1.5 TB per VM at $6.00 per GPU-hour in East US 2, the lowest price per GB of HBM on Azure. Do not pick it if you depend on CUDA-only libraries, custom CUDA kernels, or a framework version that AMD has not validated.

Per the ND MI300X v5 size page, each VM has eight AMD Instinct MI300X GPUs linked by 4th-gen Infinity Fabric (128 GB/s per GPU link, 896 GB/s aggregate), 400 Gb/s InfiniBand per GPU, 96 vCPUs and 1,850 GiB of RAM. Collective communication uses RCCL instead of NCCL.

Prices from the Retail Prices API:

• $48.00/hour in East US 2 and West US 3
• $57.60 in North Europe, $60.00 in UK South and France Central, $67.20 in Sweden Central
• Spot from $8.87/hour in East US 2
• 17 regions with a Linux meter

On AKS, the MI300X is not on the list of sizes where AKS installs the GPU driver. AMD documents the path: create the node pool with --gpu-driver none and install the AMD GPU Operator, which registers GPUs as amd.com/gpu. Budget a day of platform work the first time.

4. What does the ND GB200 v6 add for large-scale training?

The ND GB200 v6 is for teams whose model does not fit in an 8-GPU NVLink domain. Each VM has four Blackwell GPUs and two Grace CPUs, and 18 VMs form one NVL72 rack where all 72 GPUs share NVLink. If your model and parallelism plan fit on eight H200s, you do not need it.

Microsoft's ND GB200 v6 size page lists 192 GB per GPU, fifth-generation NVLink at 1.8 TB/s per GPU, and four 400 Gb/s InfiniBand links per VM. At rack scale Microsoft quotes up to 1.4 exaFLOPS of FP4, 13.5 TB of shared HBM and 130 TB/s of NVLink bandwidth. The CPUs are Arm (Grace), so your container images need linux/arm64 builds.

The Retail Prices API lists $108.16 per VM-hour in East US, East US 2 and West US 3, which is $27.04 per GPU-hour, and 19 regions with a Linux meter. Like the H200, its spot meter shows no discount. In practice, nobody buys a single GB200 VM. You buy racks, and you buy them through capacity reservations negotiated with Microsoft.

5. Is the ND GB300 v6 available to buy yet?

The ND GB300 v6 is documented, but as of 14 November 2026 it has no public price in the Azure Retail Prices API. A query for "GB300" returns no meter. Treat it as account-team capacity: you buy racks from Microsoft under a signed commitment.

The ND GB300 v6 size page (updated October 2026) gives the specs. Four Blackwell Ultra GPUs with 288 GB each, two Grace CPUs, 864 GiB of LPDDR5X, and 800 Gb/s InfiniBand per GPU, double the GB200. Microsoft states GB300 offers about 1.5 times the FP4 compute of GB200, 50% more HBM, and up to 1.44 exaFLOPS of FP4 per NVL72 domain. Microsoft positions it mainly for high-throughput inference of reasoning and agentic models, so if your work is pure pre-training on a known budget, GB200 or H200 is a simpler buy.

6. Are A100 VMs still worth it on Azure in 2026?

Yes, the NDm A100 v4 is still the cheapest way to run multi-node InfiniBand training on Azure. At $32.77 per VM-hour in East US, it costs a third of an H100 v5. Do not pick it for FP8 training, since Ampere has no FP8 Tensor Cores, or for models that need more than 80 GB per GPU.

The NDm A100 v4 size page lists eight A100 80 GB GPUs with NVLink 3.0, 200 Gb/s HDR InfiniBand per GPU (1.6 Tb/s per VM), 96 AMD Rome vCPUs and 1,900 GiB of RAM. The Retail Prices API lists 29 regions, $32.77 in East US, East US 2 and West US 3, and spot from $8.44.

7. Which Azure GPU VM is best for single-node fine-tuning?

Use the NCads H100 v5 for single-node fine-tuning in 2026. One H100 NVL with 94 GB costs $6.98/hour in East US, and the two-GPU NC80adis size costs $13.96. That is enough for LoRA and QLoRA on most open-weight models up to 70B with quantization. Do not use it for multi-node training: it has no InfiniBand, only up to 80 Gb/s of Ethernet.

The NCads H100 v5 size page lists AMD Genoa CPUs, 320 to 640 GiB of RAM, and a large local temp disk (3.5 to 7 TiB) that is handy for dataset staging. The Retail Prices API shows 29 regions and spot from $1.42/hour in East US for one GPU, though spot prices for this size vary a lot by region (West US 3 spot is $4.35).

8. What is the cheapest Azure GPU VM with 80 GB of memory?

The NC24ads A100 v4 is the cheapest 80 GB GPU on Azure at $3.673/hour in East US. It is the right size for experiments, small fine-tunes and evaluation runs. Microsoft states on the NC A100 v4 size page that it is "only deploying net new capacity" for the NCads H100 v5 series, so expect A100 PCIe capacity to get scarcer.

The series runs one to four A100 PCIe GPUs per VM (NC24ads, NC48ads, NC96ads) on AMD Milan CPUs. The Retail Prices API lists 27 regions and spot from $0.68/hour for one GPU in East US. For a team that runs lots of short jobs, that spot price is hard to beat anywhere on a hyperscaler.

How do Azure GPU prices change with spot and reservations?

Spot cuts Azure GPU prices by roughly 80% on the sizes that have a real spot discount, and reservations cut them by up to 72%. Spot fits checkpointed training. Reservations fit a steady baseline you know you will use for one or three years.

Two practical rules. Reservations cover compute only, not storage or networking, per Microsoft's reservations page. And spot quota is a separate pool, so request both. The Retail Prices API needs no authentication: one curl per SKU returns every region, on-demand and spot.

For a cross-cloud view of the same H100 class, see our GPU instance pricing comparison across AWS, GCP, Azure and Scaleway.

Which Azure regions have the most GPU training capacity?

East US 2 and West US 3 are the best default regions for GPU training on Azure in 2026. They carry the lowest list price for ND H100 v5, ND H200 v5, ND MI300X v5 and ND GB200 v6 together. In Europe, North Europe, Sweden Central, UK South and France Central list H200 v5 and MI300X v5.

Source: Azure Retail Prices API, Linux consumption meters, per VM-hour, 14 November 2026.

Two warnings. First, a meter does not mean stock. Quota for ND families usually starts at zero, and big requests take days. If you hit a wall, our guide on bursting a training job onto a second cloud when quota is denied covers the fallback. Second, data residency beats price. If your training data must stay in the EU, the cheapest US region is not an option, whatever the spreadsheet says.

How do you run Azure GPU VMs as AKS node pools for training?

On AKS, you add a GPU node pool with the ND or NC size, taint it so only GPU pods land there, and let the cluster autoscaler scale it to zero between runs. AKS installs the NVIDIA driver, device plugin and DCGM exporter for supported sizes. For InfiniBand and for AMD, you install operators yourself.

The baseline from Microsoft's AKS GPU documentation looks like this:

az aks nodepool add \
    --resource-group myResourceGroup \
    --cluster-name myAKSCluster \
    --name h200 \
    --node-vm-size Standard_ND96isr_H200_v5 \
    --node-taints sku=gpu:NoSchedule \
    --enable-cluster-autoscaler \
    --min-count 0 \
    --max-count 4

What the docs say, and what bites teams:

• Taints and VM size are fixed at creation. You cannot change the VM size of an existing node pool. Plan one pool per GPU family.
• --skip-gpu-driver-install is retired. Since August 2025, you use --gpu-driver none instead when you want the NVIDIA GPU Operator or AMD's operator to own the driver.
• AKS-managed GPU node pools (driver, device plugin, metrics handled by AKS) are now Microsoft's recommended default for supported NVIDIA sizes.
• InfiniBand needs extra work. For multi-node NCCL over IB on ND sizes, Microsoft's AKS RDMA InfiniBand project uses the NVIDIA Network Operator, Ubuntu node pools, at least two GPU nodes and a skipped GPU driver install.
• Azure Linux 2.0 is gone. AKS stopped supporting it in November 2025, and node images are removed from 31 October 2026. Use Ubuntu or Azure Linux 3 for GPU pools.

Then you still need job definitions, dataset mounts, registry secrets, checkpoint volumes, and a way for engineers to launch runs without a production kubeconfig. Most teams underestimate that part.

Raw VMs, Azure ML, DIY AKS or AKS with Qovery: how should you run training?

The VM choice stays the same whichever way you run it. The approach decides who maintains drivers, scheduling and access. Azure Machine Learning is the fastest path if you live inside Microsoft's ML tooling. Raw VMs or CycleCloud suit HPC teams with Slurm. AKS suits teams who already ship everything else on Kubernetes.

Qovery is an Agentic DevOps platform. It does not sell GPUs and it does not host your workloads. It deploys and operates your applications, jobs and Helm charts on Kubernetes in your own cloud account. On Azure, that means AKS in your subscription, so your reservations, spot quota and MACC drawdown stay in your name. The same setup works on AWS, GCP, Scaleway or an existing Kubernetes cluster.

What it changes for GPU training:

• GPU node pools per service. On a Qovery-managed AKS cluster, you choose the GPU instance type for the workload, and Qovery places the job on it. The Qovery instance catalog for Azure includes the NCads H100 v5, NC A100 v4, ND H100 v5 and ND H200 v5 sizes, among others.
• Bring your own AKS for exotic hardware. For MI300X with the AMD GPU Operator, or GB200 racks with the Network Operator, you keep your hand-built node pools and install Qovery on the existing cluster (Kubernetes 1.31 or later).
• Self-service training runs. Engineers launch a run from git push, the Console, the CLI or a coding agent, with per-environment RBAC and audit logs. Nobody needs a production kubeconfig.

Where Qovery is the wrong pick: if your team is happy in Azure Machine Learning notebooks and pipelines, stay there. And if you need a Slurm scheduler with gang scheduling across 1,000 GPUs, use CycleCloud or a dedicated scheduler such as Kueue on AKS. Qovery deploys and operates the workloads; it is not a training scheduler. For the broader tool landscape, see our list of platforms that provision GPU clusters automatically and our guide to who manages AKS node pools, networking and scaling.

Which Azure GPU VM should you pick? A decision rule

Start from the model and the budget. Here is the rule I give teams:

1. Model plus optimizer state fits in one 80-94 GB GPU? NCads H100 v5 (or NC A100 v4 if budget-bound).
2. Needs 2-8 GPUs on one node with fast NVLink, CUDA stack? ND H200 v5 in East US 2 or West US 3.
3. Needs many nodes and you can checkpoint every 15-30 minutes? ND H100 v5 on spot, with on-demand fallback.
4. Memory-bound and ROCm-ready? ND MI300X v5.
5. Model needs more than 8 GPUs in one NVLink domain? ND GB200 v6, and talk to Microsoft about GB300 v6.
6. Tight budget, FP16/BF16 only? NDm A100 v4.

Before you sign anything, confirm quota for the VM family in your region, InfiniBand in your node pool setup, and a reservation term finance can live with.

How do I set this up with Qovery from Claude Code or Cursor?

If you work from a coding agent, you can hand it the setup. These are the only commands you need:

curl -fsSL https://skill.qovery.com/install.sh | bash

# 2. Or install them for the current project only
curl -fsSL https://skill.qovery.com/install.sh | bash -s -- --project

# 3. Add the Qovery MCP Server to Claude Code (OAuth, read-only by default)
claude mcp add --transport http qovery https://mcp.qovery.com/mcp --callback-port 4242

# 4. Only if you explicitly want write access (also enable it in the Qovery Console)
claude mcp add --transport http qovery "https://mcp.qovery.com/mcp?read_write=true" --callback-port 4242

Commands 1 or 2 install the skills (qovery-onboard, qovery-deploy, qovery-troubleshoot, qovery-optimize, qovery-speedup, qovery-preview, qovery-terraform); the agent asks you to approve each action. Command 3 gives read-only access through an OAuth login. Command 4 allows changes, so use it only when you mean to.

Then ask in plain language, for example: "set up Qovery on my Azure subscription with an AKS cluster in East US 2 and deploy my training job on an ND H200 v5 node pool." Start from the AI agent quickstart or the Claude Code integration.