7 best clouds for GPU-heavy AI inference in 2026 (and where OCI really fits)

Key Points:

• OCI is worth adding in two cases. Add it when egress dominates your bill, or when you need bare-metal 8x H100/H200 nodes with 3,200 Gb/s RDMA networking. Oracle's first 10 TB/month of outbound traffic is free, then $0.0085/GB (Oracle price list), against 100 GB free and $0.09/GB on AWS (AWS). Outside those two cases, OCI is a second control plane you pay for in engineering time.
• AWS now lists the cheapest H100 among the big four. Since the June 2025 price cut, a p5.48xlarge costs $55.04/hour, or $6.88 per GPU-hour. OCI lists $10.00, Google Cloud $11.06 and Azure $12.29.
• AWS and Google Cloud win everything around the GPU. That means managed endpoints (SageMaker, Vertex AI), model catalogs, IAM depth, region count and third-party tooling. If your data already lives there, a cheaper GPU elsewhere rarely survives cross-cloud egress plus a second deploy path.
• Specialists win on raw GPU price. Lambda lists H100 at $3.99 per GPU-hour (Lambda), Nebius at $4.50 (Nebius) and CoreWeave at $6.16 (CoreWeave). All three undercut every hyperscaler, by up to 3x against Azure. In exchange you get fewer managed services, fewer regions and a thinner compliance portfolio.
• Normalize every quote to one GPU-hour. Lambda's $3.99 looks 14x cheaper than AWS's $55.04 until you notice that one is per GPU and the other is per 8-GPU node. Per GPU, the real gap is 1.7x.
• Qovery is not a GPU cloud. It is the deployment and operations layer that runs inside your own AWS, GCP, Azure or Scaleway account, or on your own Kubernetes cluster, including OKE on OCI. One git-push workflow then covers every cloud without a second platform team.

The GPU hourly price is the least interesting number in this decision. I talk to teams every week who are about to sign a GPU capacity contract, and the ones who regret it later rarely got the hourly rate wrong. They got egress wrong, or capacity access, or they forgot that a second cloud means a second control plane, a second CI/CD path and a second on-call rotation.

So this guide ranks seven clouds on what decides inference economics in practice, with the hourly rate as one input out of six.

How did we rank the best clouds for GPU-heavy AI inference?

We scored seven clouds on six criteria: GPU supply, normalized price per GPU-hour, egress, managed Kubernetes, inference tooling and exit cost. Price per GPU-hour is only one of the six because on a high-QPS inference API, egress and operations often move the total more than the GPU line does.

1. GPU supply and shape variety. Can you get H100, H200, B200, GB200, L40S or MI300X in your region today, on-demand or reserved, bare metal or virtualized?
2. Effective cost per GPU-hour. On-demand, committed and spot, normalized to a single GPU. Not an 8-GPU node, and not a TPU chip.
3. Egress pricing and free allowance. Inference streams tokens, images and audio outbound. Free allowances range from about 1 GiB on Google Cloud Premium Tier to "Free" on CoreWeave.
4. Managed Kubernetes and control-plane fee. Most teams serve inference on Kubernetes with vLLM, KServe, Triton or Ray Serve. The CNCF reports that 82% of container users run Kubernetes in production (CNCF Annual Survey).
5. Inference tooling and ecosystem. Managed endpoints, model catalogs, GPU-metric autoscaling, observability and integrations.
6. Portability and exit cost. How much of your deploy path, IAM, networking and storage is cloud-specific.

Out of scope: serverless token APIs such as OpenAI, Anthropic or Amazon Bedrock. This guide is about running your own GPU inference.

Method: all figures are vendor-published on-demand list prices for US regions on November 4, 2026, linked to each vendor's own pricing page or price API. Scaleway and OVHcloud are EU-only and priced in their EU regions. Vendors publish per-node, per-chip and per-GPU prices interchangeably, so I divided every node price by its GPU count. Negotiated and committed rates will differ, and GPU pricing moves monthly.

Which cloud is best for GPU AI inference in 2026?

No single cloud wins. AWS and Google Cloud win on ecosystem and managed inference. OCI wins on egress economics and bare-metal RDMA clusters. CoreWeave, Lambda and Nebius win on raw published price per GPU-hour, with Lambda's H100 at $3.99 against $6.88 to $12.29 on the hyperscalers.

Two more clouds deserve a mention. Nebius lists H100 at $4.50 and H200 at $5.40 per GPU-hour, with free managed Kubernetes and free egress (Nebius). OVHcloud lists a single H100 at $2.99/hour and includes outbound traffic in instance prices outside Asia-Pacific (OVHcloud).

Here are the same seven, ranked for three buyer profiles:

*Lambda and Scaleway do not publish an instance egress rate on the pricing pages I checked. They rank last only because no rate is published. Ask before you sign.

A note on Qovery, since you are reading this on our blog. Qovery does not appear in these tables because it is not a GPU provider and does not resell compute. It is the deployment layer that runs on top of whichever of these clouds you pick.

• Pick a hyperscaler if your data, identity and compliance already live there.
• Pick OCI if egress is a big share of your bill and Google Cloud or Azure is your alternative.
• Pick a specialist if the GPU is most of your bill and you need little else from the cloud.

Is Oracle Cloud Infrastructure actually worth adding for GPU inference?

Yes, if your outbound traffic is large enough that OCI's 10 TB/month free egress and $0.0085/GB overage outweigh its GPU price, or if you need bare-metal 8x H100/H200 nodes with 3,200 Gb/s RDMA cluster networking. Otherwise, no. You would be buying a second control plane to save on one line item.

What OCI does well:

• Egress. The first 10 TB per month from North America, Europe and the UK is free, then $0.0085/GB (Oracle price list). That is roughly a tenth of the AWS and Azure list rates.
• Bare metal. The GPU shapes run without a hypervisor. BM.GPU.H100.8 and BM.GPU.H200.8 each list at $10.00 per GPU-hour, and BM.GPU.MI300X.8 at $6.00 (Oracle).
• RDMA. Oracle advertises "up to 3,200 Gb/sec of RDMA cluster network bandwidth" on its H100, H200, B200 and MI300X bare-metal shapes (Oracle GPU, shape reference). That matters when one model spans several nodes.
• Kubernetes. OKE runs standard upstream Kubernetes. Enhanced clusters cost $0.10 per cluster-hour, and the price list has no control-plane line for basic clusters. Your Helm charts, vLLM or KServe manifests and CI largely carry over, and that portability is what makes OCI addable at all.

Where OCI falls short:

• Managed inference tooling. Google Cloud and AWS beat OCI on managed inference tooling and ecosystem depth. Vertex AI and SageMaker are more mature than anything OCI offers for serving.
• Regions. Oracle lists "50+ public cloud regions across 28 countries" (Oracle regions), but not every region carries the newest accelerators.
• Skills. Far fewer engineers know OCI compartments and IAM policies than know AWS IAM.
• Data gravity. If your objects and feature store sit on S3 or BigQuery, every request that reads them pays cross-cloud egress and latency before you serve a token.

I ran the numbers for one 8x H100 node running all month (730 hours) and serving 50 TB of internet egress. Inputs are list prices: OCI at $80.00/node-hour, AWS p5.48xlarge at $55.04, and Google Cloud a3-highgpu-8g at $88.49. Egress follows each vendor's published tiers, with 50 TB taken as 50,000 GB.

Here is the arithmetic:

• OCI: $80 x 730 = $58,400. Egress is (50,000 - 10,240 free GB) x $0.0085 = $338.
• AWS: $55.04 x 730 = $40,179. Egress is 9,900 GB at $0.09 plus 40,000 GB at $0.085 = $4,292 (AWS).
• Google Cloud: $88.49 x 730 = $64,598. Egress is about 46,566 GiB on Premium Tier at $0.12, then $0.11, then $0.08 = $4,043 (Google Cloud).

The result surprised me when I first ran it. OCI beats Google Cloud by about $9,900 a month, and Azure too ($98.32 x 730 = $71,774 before egress). It loses to AWS at list price, because AWS's June 2025 cut put the H100 at $6.88 per GPU-hour (AWS announcement). Against AWS, OCI's egress saving only covers the $18,221 GPU gap at around 350 TB of egress per node per month, or at a committed OCI rate well below list.

My verdict: add OCI as a capacity pool behind a Kubernetes-level abstraction. Do not make it a replacement for your primary cloud.

When is AWS the right cloud for GPU inference?

AWS is the right pick when your data, identity and compliance already live on AWS. It now also lists the cheapest hyperscaler H100 at $6.88 per GPU-hour, so leaving rarely pays. It is the wrong pick for a lean team serving one open-weights model at high volume with heavy egress and no AWS data gravity.

Strengths:

• Range of shapes. All prices are US East on-demand, per GPU-hour (AWS):
  • p5.48xlarge (8x H100): $6.88
  • p5en.48xlarge (8x H200): $7.91
  • p6-b200.48xlarge (8x B200): $14.24
  • g6e.xlarge (1x L40S): $1.86
• Capacity Blocks for ML. You can reserve 1 to 64 instances for up to six months, booked up to eight weeks ahead, with dynamic pricing (AWS). The US East table lists p5.48xlarge at $5.970 per accelerator-hour (AWS).
• Inferentia2. This is AWS's underrated inference lever. inf2.48xlarge costs $12.98/hour for 12 Inferentia2 chips, or $1.08 per chip (AWS). The catch is porting your model to the Neuron SDK, which costs real engineering weeks.
• Managed tooling. SageMaker endpoints and EKS Auto Mode cover more ground than any other cloud on this list.

Weaknesses:

• Egress. 100 GB/month is free across all services, then $0.09/GB for the first 10 TB (AWS).
• EKS fees. Clusters cost $0.10 per hour on standard support and $0.60 on extended support. Auto Mode adds a per-instance management fee on top of the EC2 price (AWS).
• Spot is less useful for top-end GPUs. AWS advertises "up to 90%" off on Spot (AWS). Stateless inference replicas tolerate interruption far better than training jobs, but Spot capacity on top-end GPUs is often thin.

Skip AWS if you have no data there, serve one model to the public internet, and your egress bill already rivals your GPU bill.

When is Google Cloud the right cloud for GPU inference?

Google Cloud is the strongest pick when you want managed endpoints plus a credible non-NVIDIA supply curve. Vertex AI plus TPU v5e at $1.20 per chip-hour gives you a second source of capacity when NVIDIA allocation is tight (Google Cloud TPU pricing). It is the wrong pick if your serving stack has CUDA-specific kernels you cannot port to XLA.

Strengths:

• NVIDIA shapes. Per GPU-hour on-demand (Google Cloud):
  • A3 High (8x H100): $11.06
  • A3 Ultra (8x H200): $10.60
  • A4 (B200): no published on-demand price, only DWS and commitments
• Dynamic Workload Scheduler. Flex-start mode queues for capacity at a lower rate: $4.79 per H100-hour on A3 High.
• TPUs. Trillium (v6e) lists at $2.70 per chip-hour in us-east1. The constraint comes in the same breath: TPUs want JAX or PyTorch/XLA.
• GKE. It has the most mature GPU and TPU scheduling of the hyperscalers. The $0.10 per cluster-hour fee comes with a $74.40 monthly credit, enough for one zonal or Autopilot cluster (GKE pricing).

Weaknesses:

• Egress. Premium Tier starts at $0.12/GiB to North America after 1 GiB free. Standard Tier is cheaper ($0.085/GiB after 200 GiB free) but uses slower routing (Google Cloud).
• Quotas. New accounts requesting large GPU counts hit quota friction.

Autopilot or Standard? On Autopilot, GPU pods switch to node-based billing: Compute Engine price plus a management premium, and you pay for the whole node. Pick Standard if you pack several models per node. Pick Autopilot if you want Google to own node operations.

Skip Google Cloud if your estate is heavily Microsoft, or if deep Oracle Database dependencies pull you elsewhere.

When does Microsoft Azure make sense for GPU inference?

Azure is the right choice for enterprises whose identity, compliance and model access already run through Microsoft, especially if you want Azure OpenAI and self-hosted models in one tenancy under Entra ID. It is the wrong choice when the only goal is the lowest GPU price. ND H100 v5 lists at $98.32/hour in East US, or $12.29 per GPU-hour, against $6.88 on AWS and $11.06 on Google Cloud.

• Shapes. ND H200 v5 lists at $84.80/hour, or $10.60 per GPU, so the H200 is cheaper than the H100 here. ND MI300X v5 costs $6.00 per GPU (Azure retail prices API).
• AKS fees. The Free tier has no cluster fee, and Standard tier costs $0.10 per cluster-hour (Azure).
• Capacity. Microsoft's CFO said on the FY26 Q1 earnings call that "we now expect to be capacity constrained through at least the end of our fiscal year" (Microsoft), which ended in June 2026. Expect quota approvals measured in days.
• Pricing outside the list. Real prices often sit behind an Enterprise Agreement.

Skip Azure for cost-sensitive, steady-state inference where every GPU-hour counts.

Are GPU-specialist clouds like CoreWeave, Lambda and Nebius cheaper than the hyperscalers?

Yes. Lambda lists H100 at $3.99 per GPU-hour, Nebius at $4.50 and CoreWeave at $6.16. All three beat the cheapest hyperscaler list price, AWS at $6.88, and they come in 2x to 3x below Google Cloud and Azure. They are the right pick when the GPU is most of your bill and you need little else from the cloud.

Look past the hourly rate to tokens per dollar. The H200 carries 141 GB of HBM3e at 4.8 TB/s, against 80 GB at 3.35 TB/s on the H100 SXM (NVIDIA H200, NVIDIA H100). NVIDIA's MLPerf Inference v4.0 submission showed H200 running Llama 2 70B up to 28% faster than H100 at the same 700 W (NVIDIA). At CoreWeave's $6.31 against $6.16, a 2.4% premium for up to 28% more throughput makes the H200 the cheaper chip per token.

What specialists do well:

• Dense GPU fleets with InfiniBand multi-node clusters.
• Transparent, self-serve pricing with no Enterprise Agreement.
• Free or included egress on CoreWeave and Nebius.

What they do not offer: a managed Postgres you would trust with production, fine-grained IAM as mature as AWS's, dozens of regions, a broad compliance portfolio, or a long track record.

The pattern that works in production keeps the control plane, data and auth on a hyperscaler and puts the GPU inference pool on a specialist. The two meet at the Kubernetes layer, with one deployment workflow across both.

Skip specialists for regulated workloads that need broad certifications, or if procurement insists on a single vendor contract.

Which cloud is best for GPU inference with EU data residency?

Scaleway is the strongest fit when EU data residency is a hard requirement. It lists H100 PCIe at €2.87/hour and L40S at €1.47/hour in its Paris and Warsaw zones (Scaleway), with a free Kubernetes Kapsule control plane. Nebius and OVHcloud are the credible alternatives. The EU regions of AWS, Google Cloud and Azure remain the fallback when you need breadth over sovereignty.

• Sovereignty. Your data stays with an EU-headquartered operator, which is a shorter conversation with your DPO than a US hyperscaler's EU region.
• Shapes and Kubernetes. H100 SXM runs to 8-GPU nodes at €3.17 per GPU, in Paris only. Kapsule Mutualized is free, and dedicated control planes start at €0.11/hour with a 99.5% SLA (Scaleway).
• Limits. The catalog is smaller, AI-specific managed services are fewer, and there is less multi-node InfiniBand capacity for very large clusters.
• Alternatives. Nebius runs H100 and H200 in Finland and H200 in France (Nebius regions). OVHcloud lists H100 at $2.99/hour and L40S at $1.80/hour with egress included outside Asia-Pacific (OVHcloud).

Qovery supports Scaleway as a first-class cloud, exactly like AWS, GCP and Azure, and runs on any existing EU-hosted Kubernetes cluster.

What does it really cost to run inference on a second cloud, and how does Qovery cut that cost?

A second cloud costs you a second control plane, a second CI/CD path, a second RBAC model and a second on-call runbook long before it costs you a GPU-hour. Stack Overflow's 2025 survey puts the US median salary for a cloud infrastructure engineer at $189,000 (Stack Overflow), before benefits and overhead. That one hire wipes out the GPU savings of a mid-size fleet.

Qovery is not a GPU provider. It is the deployment and operations layer that installs inside your own AWS, GCP, Azure or Scaleway account, or on your own Kubernetes cluster, including OKE on OCI. A second cloud then does not need a second platform team.

The tax is real elsewhere too. Flexera's 2026 State of the Cloud Report finds wasted cloud spend rose to 29%, and 85% of respondents name cost as a top challenge (Flexera). On GPU fleets, a lot of that waste is an H100 sitting in a staging namespace overnight at $6 to $12 an hour.

What Qovery does:

• Git-push deployments to every cluster from one workflow.
• Preview environments for every pull request.
• Auto-stop for non-production environments, so that idle staging GPU gets switched off.
• Managed cluster upgrades.
• RBAC per environment.
• Databases backed by your cloud's managed services.

Why BYOC matters for GPUs: your Savings Plans, committed-use discounts and negotiated OCI or CoreWeave contracts stay in your own name. Qovery never resells compute.

Honest boundaries:

• If you run one cloud, one cluster and already have a platform team, you probably do not need Qovery. Adding a second cloud without adding headcount is the exact gap it fills.
• Qovery does not schedule GPUs, does not supply GPU capacity, and does not replace vLLM, Triton, KServe or Ray Serve. It runs the applications, environments and access control around them.

How do I deploy an inference service with Qovery from Claude Code or Cursor?

Install the Qovery Agent Skills with one curl command, then ask your coding agent in plain language to deploy into your own cloud account. The skills cover onboarding, deploys, troubleshooting and cost optimization. The MCP Server operates what already exists and is read-only by default.

curl -fsSL https://skill.qovery.com/install.sh | bash

# Or scope the install to the current project only
curl -fsSL https://skill.qovery.com/install.sh | bash -s -- --project

# Add the Qovery MCP Server to Claude Code over OAuth (read-only by default)
claude mcp add --transport http qovery https://mcp.qovery.com/mcp --callback-port 4242

# Write access, only when you explicitly want it (also enable it in the Qovery Console)
claude mcp add --transport http qovery "https://mcp.qovery.com/mcp?read_write=true" --callback-port 4242

The MCP install opens an OAuth approval in your browser. Each skill handles one job:

After that, the prompt is plain language: "deploy this vLLM inference API to my GCP account with Qovery and give me a preview environment per pull request."

Full setup is in the AI agent quickstart and the Claude Code integration page.