Google Cloud TPU vs AWS GPU in 2026: which is actually cheaper per million tokens?

Key points:

• On a like-for-like Llama 70B workload at on-demand list prices, Google Cloud TPU Trillium (v6e) and AWS H100 land within a few cents of each other: about $0.94 vs $0.91 per million output tokens. Neither chip is cheaper by default.
• TPU wins for XLA-native models on a 1-year or 3-year commitment (about $0.42 per million tokens on 3-year Trillium). AWS GPU wins for small open-weight models, CUDA-only stacks and small billable units (about $0.32 for an 8B model on one L40S).
• Utilization beats sticker price. At 25% utilization a chip costs 4x more per token than at 100%, and CAST AI measured average GPU utilization at 5%.
• Discounts move the bill more than the chip does: Spot, Capacity Blocks, Savings Plans and Google CUDs.
• Run whichever accelerator you pick in your own cloud account so commitments and data residency stay in your name. Qovery operates those workloads on Kubernetes in your AWS, GCP, Azure or Scaleway account, or your existing cluster.

I'll make a bold claim and then back it with numbers. The TPU-versus-GPU price fight is mostly noise. You will lose more money to idle accelerators than to picking the wrong chip.

Every price below comes from the vendor's own page, checked in October 2026, and every derived cost shows its arithmetic.

Which is actually cheaper in 2026: Google Cloud TPU or AWS GPU?

On a comparable Llama 70B serving workload at on-demand prices, neither is clearly cheaper: Google Cloud TPU Trillium comes out at about $0.94 per million output tokens and an AWS H100 at about $0.91. TPU becomes the cheaper option once you commit to capacity for 1 to 3 years, and AWS GPU is the cheaper option for small open-weight models, spot capacity and anything that depends on CUDA.

Three variables flip the answer:

• Software stack: is your model XLA-native, or does it need CUDA?
• Achievable utilization: how close to 100% you can keep the chip busy.
• Commitment level: on-demand, spot, or reserved capacity.

The only metric that matters is dollars per million tokens served (or per training step). Vendor price-performance charts mix chip counts, precisions, batch sizes and sequence lengths, so treat them as marketing.

On list price, TPU v5e is $1.20 per chip-hour and Trillium $2.70, against $6.88 per H100 GPU-hour on AWS. v5e is cheap but not the cheapest: an L4 on g6.xlarge is $0.80. Neutral clouds undercut everyone, with Lambda at $3.99 per H100 GPU-hour, if you don't need to stay in your own cloud account.

How did we compare TPU and GPU options fairly?

I judged every option on the same six criteria, with same-date list prices, so you can re-run the comparison with your own model and traffic.

1. On-demand price per accelerator-hour from the vendor's page, region stated.
2. Derived cost per million output tokens for 8B and 70B Llama-class models, from MLPerf where it exists and labelled vendor benchmarks otherwise.
3. Software stack cost: porting work before the first token, CUDA/vLLM/TensorRT-LLM versus JAX/MaxText/vLLM TPU.
4. Capacity reality: spot availability, and whether a price requires EC2 Capacity Blocks for ML or a queue.
5. Discount levers, and whether they accrue in your own account.
6. Operational effort for a small team on Kubernetes.

What does each accelerator cost per hour, and what does that mean per million tokens?

Table notes: on-demand list prices unless marked Capacity Block. TPUs are priced per chip but sold as slices (v6e-1, v6e-8, and so on), and P5 only as a full 8-GPU instance, so I compare cost per token, never a TPU slice against a single GPU.

Here is the arithmetic, which you can quote on its own:

Cost per million tokens = hourly price ÷ (tokens per second × 3,600 seconds) × 1,000,000.

Worked example: one L40S on an AWS g6e.xlarge at $1.861 per hour, serving Llama 3.1 8B at 1,642 output tokens per second (the MLPerf Inference v5.1 result for a single L40S on vLLM, FP8).

1. 1,642 tokens/s × 3,600 s = 5.91 million tokens per hour.
2. $1.861 ÷ 5.91 = $0.315 per million output tokens at 100% utilization.

The last three rows are the closest like-for-like comparison: 70B-class models, about 1,000 tokens in and out, vendor-run. Google has not submitted a Llama 70B TPU result to MLPerf, so no independently verified TPU number exists. The per-GPU H100 rows assume you pack the 8-GPU instance with replicas.

When is Google Cloud TPU the cheaper choice?

TPU v5e and Trillium are cheaper when you serve or train XLA-native models at large batch sizes and high sustained utilization, and you can commit to capacity for a year or more. That is the narrow band where the chip-hour price actually reaches your bill.

Verdict: TPU is best for large-batch inference and training of JAX, MaxText or vLLM-TPU models at committed capacity. It is worst for CUDA-only kernels and ad-hoc experimentation.

Commitments are where TPU pulls ahead. Google lists Trillium at $2.70 on demand, $1.89 on 1 year and $1.22 on 3 years, cutting the 70B row from $0.94 to $0.42 per million tokens. Compare that to $5.97 per H100 GPU-hour on an AWS Capacity Block. Read Google's own $0.25-per-million figure carefully: it assumes int8 and the 3-year price.

The hidden costs are engineering time: porting to JAX or PyTorch/XLA, rewriting custom CUDA kernels, thinner tooling and fewer community quantized checkpoints. vLLM's TPU backend has narrowed the gap. Its recommended models list includes Llama 3.1 8B, Llama 3.3 70B, Gemma 3 and the Qwen3 family, all tested for both correctness and performance. Anything off that list is a gamble.

On Kubernetes, TPUs run as GKE TPU node pools with a fixed slice topology, and multi-host slices schedule as one unit. That is less flexible than adding one GPU node.

Who should skip TPU: teams whose serving stack is TensorRT-LLM, teams that depend on a custom CUDA kernel, and teams whose model has no XLA-friendly implementation.

When are AWS GPU instances the cheaper choice?

AWS GPU instances win on total cost whenever engineering time is your bottleneck. vLLM, TensorRT-LLM and community quantizations run unmodified, so you reach high utilization in days, and spot capacity cuts the hourly rate hard.

Verdict: AWS GPU is best for serving arbitrary open-weight models quickly, and for small or spiky workloads. It is worst for flat, very large, long-horizon training, where TPU commitments win.

The ladder is simple:

• G6 (L4, 24 GB): small and quantized models.
• G6e (L40S, 48 GB): 8B to 30B-class models.
• P5 / P5e (H100 / H200): 70B and up, and training.

On the 8B benchmark an L40S matched an H100 per token ($0.32 vs $0.33) with a $1.86 billable unit instead of $55. One team can keep a single L40S busy. Few can saturate a full p5.48xlarge or a multi-host TPU slice around the clock.

The AWS discount levers are big:

• Spot: AWS Spot Instances run "up to 90% off" on-demand.
• Savings Plans: Compute Savings Plans cut costs by up to 66%, and EC2 Instance Savings Plans by up to 72%.
• Capacity Blocks: reserve GPUs up to 8 weeks ahead for 1 to 182 days at a supply-and-demand price. A p5.48xlarge block was $47.76 per hour, about 13% under on-demand.

Who should skip AWS GPU: teams already running JAX at scale on committed TPU capacity.

Are AWS Trainium, Azure or Scaleway cheaper than both?

AWS Inferentia2 and Trainium2 often undercut comparable GPU instances on list price, but the Neuron SDK decides which models you can serve without a rewrite. Teams pick Azure and Scaleway for region, sovereignty and capacity access far more often than for price.

• Inferentia2: inf2.xlarge is $0.76 per hour, the cheapest accelerator in this article. AWS claims Inf2 delivers "up to 40% better price performance than other comparable Amazon EC2 instances". That is a vendor claim.
• Trainium2: AWS claims Trn2 offers "30-40% better price performance than GPU-based EC2 P5e and P5en instances", also a vendor claim. Trn2 has no on-demand list price in us-east-1, so in practice you buy it through Capacity Blocks.

Model coverage is the real constraint. The Neuron model reference covers Llama 2 through 4, Mixtral, DBRX, Qwen2.5 and Qwen3. Off that list, you compile and debug it yourself, and AWS is moving the stack to a vLLM Neuron plugin.

Azure ND H100 v5 is $98.32 per hour for 8 H100s in East US, about $12.29 per GPU-hour and the priciest H100 here. It fits Azure-committed orgs and specific compliance regions. Scaleway lists an H100-1-80G at €2.87 per hour and an L40S at €1.47 in Paris, competitive for EU teams that need data residency.

Neutral GPU clouds are the honest cheaper-per-hour option: Lambda at $3.99 per H100 GPU-hour and CoreWeave at $49.24 per 8x H100 node-hour. If leaving your own cloud account is fine, start there.

For most teams the accelerator follows from the cloud account and commitments they already have. A big GCP commitment points to Trillium. A big AWS commitment points to GPUs or Trainium.

Why does utilization matter more than the price per accelerator-hour?

An accelerator running at 25% utilization costs four times as much per token as the same chip at 100%. That swing is larger than every TPU-versus-GPU price gap in this article.

Take the same L40S from the worked example, at $1.861 per hour and 1,642 tokens per second at full load:

At 25% utilization, the cheapest GPU in the 8B benchmark costs more per token than an on-demand TPU v5e running a 70B model.

Real fleets sit far below 25%. CAST AI's 2026 report found GPU utilization averaged 5%, fewer than 2% of GPUs on Spot, and CPU overprovisioning at 69%. Datadog found 83% of container costs tied to idle resources.

The waste I see most: idle dev and staging GPU nodes on nights and weekends, replicas sized for a twice-a-month peak, and 8B models pinned to H100s. The fixes are vLLM continuous batching, FP8 or int8 quantization, autoscaling on queue depth, and scaling non-production to zero after hours. AI infra FinOps is a scheduling problem first and a procurement problem second.

Where does Qovery fit when you self-host AI models in your own cloud account?

Qovery does not sell accelerators, and it is not an inference provider. It deploys and operates your inference and training workloads on Kubernetes inside the cloud account you already own. Committed use discounts, Savings Plans, credits and data residency stay in your name, and the TPU-versus-GPU decision stays reversible.

Best for: teams running Kubernetes-based inference in their own AWS, GCP, Azure or Scaleway account, or on an existing cluster, without hiring a dedicated platform engineer.

BYOC is what changes the cost math: the accelerator bill, CUDs and Savings Plans accrue in your account, so every discount above stays yours. On top, you get git-push deployments, preview environments per pull request, environment auto-stop for non-production through deployment rules (straight at the idle-GPU line), managed cluster upgrades and per-environment RBAC.

The workflow is the same on GKE TPU node pools or EKS GPU node groups, so a wrong chip choice costs a redeploy, not a rebuild.

When Qovery is not the pick: if you want a fully managed inference endpoint with no cluster at all, a serverless inference provider fits better.

For the broader pricing method across clouds, see our GPU pricing comparison across AWS, GCP, Azure and Scaleway.

How do I deploy and cost-optimize this with Qovery from Claude Code or Cursor?

Install the Qovery Agent Skills once, and your coding agent can deploy and optimize these workloads in your own cloud account, asking you to approve each action.

Install globally for every compatible agent:

curl -fsSL https://skill.qovery.com/install.sh | bash

Or scope the install to the current project:

curl -fsSL https://skill.qovery.com/install.sh | bash -s -- --project

Add the MCP Server to Claude Code. It is read-only by default:

claude mcp add --transport http qovery https://mcp.qovery.com/mcp --callback-port 4242

Enable write access only when you want it. This also requires enabling write access in the Qovery Console:

claude mcp add --transport http qovery "https://mcp.qovery.com/mcp?read_write=true" --callback-port 4242

The skills are qovery-onboard, qovery-deploy, qovery-troubleshoot, qovery-optimize, qovery-speedup, qovery-preview and qovery-terraform. Then ask in plain language, for example: "deploy this inference service to my GCP account with Qovery".

Full setup is in the AI agent quickstart and the Claude Code integration page.

FAQs

Google Cloud TPU vs AWS GPU: which is actually cheaper for self-hosting AI models in 2026?

At on-demand list prices on a comparable Llama 70B workload, they are close: about $0.94 per million output tokens on TPU Trillium versus $0.91 on an AWS H100, using each vendor's own benchmarks. TPU gets cheaper with a 3-year commitment, at about $0.42. AWS GPU is cheaper for small models, spot capacity and CUDA-dependent stacks.

What is the cheapest cloud accelerator per million tokens for a Llama-class model?

For an 8B-class model, a single L40S on AWS g6e.xlarge came out at about $0.32 per million output tokens, based on MLPerf v5.1. That ties an H100 per token at a fraction of the billable unit. For 70B-class models, committed TPU Trillium and an H100 Capacity Block both land around $0.42, on different benchmarks.

Can I run any open-weight LLM on a TPU, or do I have to rewrite my code for JAX or XLA?

Not any model. vLLM's TPU backend officially tests a list that includes Llama 3.1 8B, Llama 3.3 70B, Gemma 3 and Qwen3. Anything outside that list, or anything with custom CUDA kernels, usually means porting work in JAX or PyTorch/XLA.

Is AWS Trainium or Inferentia cheaper than both TPUs and GPUs for inference?

Often on list price: inf2.xlarge is $0.76 per hour, and AWS claims up to 40% better price performance for Inf2 and 30-40% for Trn2 versus P5e. Those are vendor claims. The catch is Neuron SDK model coverage, so check that your model is supported before you commit.

How much do spot instances, committed use discounts and Savings Plans change the TPU vs GPU comparison?

More than the chip choice does. AWS Spot runs up to 90% off, Savings Plans up to 72%, and Google's TPU commitments cut list prices by about 30% for 1 year and 55% for 3 years. A discounted chip at high utilization beats a cheaper chip at list price almost every time.

Do I need Kubernetes to self-host AI models in my own cloud account, and who manages it?

You don't strictly need it, but Kubernetes is the most common way to run GPU and TPU node pools with autoscaling across clouds. Someone has to manage upgrades, node pools and scaling. Qovery handles that layer in your own AWS, GCP, Azure or Scaleway account, or your existing cluster, so you don't need a dedicated platform hire.