7 best European GPU clouds for self-hosted AI inference in 2026 (Scaleway's AMD MI355X vs the NVIDIA options)

An AMD Instinct MI355X carries 288 GB of HBM3E, against 141 GB on an NVIDIA H200 and 80 GB on an H100 (AMD, NVIDIA H200, NVIDIA H100). A 70B model at FP16 fits on one MI355X, needs two H200s, and needs four H100s. The catch for European buyers: as of 8 November 2026, Scaleway's public catalog lists no MI355X instance at a self-serve price, so today the cheapest computable tokens per euro in an EU-headquartered cloud still come from NVIDIA hardware at Scaleway and OVHcloud.

I priced this the way a CTO would before signing anything. Here is the ranking, the arithmetic, and where the AMD story holds up today.

Key Points:

• Scaleway, OVHcloud and Nebius publish per-GPU-hour prices you can compute against without talking to sales. Scaleway and OVHcloud are French companies pricing in euros; Nebius is headquartered in Amsterdam and prices in US dollars.
• The MI355X has 288 GB of HBM3E per GPU against 141 GB on the H200 and 80 GB on the H100. A 70B model at FP16 needs about 140 GB of weights plus KV cache, so it fits on one MI355X, two H200s or four H100s. Scaleway's NVIDIA B300 instance also carries 288 GB per GPU, so memory alone no longer makes AMD the only single-GPU option.
• Scaleway does not list a self-serve MI355X instance in its public catalog as of 8 November 2026. Its catalog API and docs list H100, L40S, L4, RTX PRO 6000 Blackwell and B300 GPU instances only, so any MI355X price you see needs to come from Scaleway sales in writing.
• ROCm is production-viable for serving open-weight LLMs with vLLM and SGLang in 2026. For TensorRT-LLM, NVIDIA NIM or hand-written CUDA kernels, NVIDIA stays the right answer, and moving off them is a bad trade.
• Tokens per euro beats tokens per second as the deciding metric. Take published MLPerf throughput, divide by the published EU hourly rate, then multiply the cost by 1 / utilisation: a node billed 24/7 and used 30% of the time costs 3.3x more per token than its sticker suggests.
• An EU region is not EU jurisdiction. An AWS, Google Cloud or Azure region in Frankfurt or Stockholm is still operated by a US-headquartered company within reach of the US CLOUD Act (18 U.S.C. 2713).
• The GPU is half the job. Self-hosted inference also needs Kubernetes, a GPU operator, node pools and scale-to-zero. Qovery runs that deployment and cluster layer inside your own Scaleway, AWS, GCP or Azure account, or your existing cluster. Qovery does not sell, schedule or serve GPUs.

How did we rank these European GPU clouds for self-hosted inference?

We ranked on six criteria: a published per-GPU-hour price, memory per GPU, EU jurisdiction, managed Kubernetes with GPU node pools, inference-stack maturity on CUDA or ROCm, and how much platform work stays with your team. Providers that hide prices behind "contact sales" rank lower, because you cannot compute tokens per euro without a number.

Memory per GPU matters because it sets the largest model one node can serve, and so how much tensor parallelism you inherit. EU region and EU jurisdiction are two separate tests. Kubernetes support and software maturity were checked on each provider's and project's own docs, not vendor marketing.

I did not optimise for large-scale training, free credits or serverless token APIs. Every price below was checked on 8 November 2026, on-demand, excluding VAT, and normalised per GPU-hour. Scaleway's site blocks automated fetches, so I read its prices from Scaleway's public product catalog API, which bills per minute, and multiplied by 60.

Which European GPU cloud should you pick for self-hosted inference in 2026?

Scaleway is the best overall pick for EU-jurisdiction inference with public euro pricing and Kubernetes GPU pools that work out of the box. OVHcloud has the cheapest published H100 and H200 rates in euros with outbound traffic included, and Nebius has the broadest public NVIDIA price list. AWS, Google Cloud and Azure only win if you already have a spend commitment with them and accept US jurisdiction.

Verdict. Pick Scaleway for EU jurisdiction plus zero-setup GPU pools, OVHcloud for the lowest euro rates with egress included, and Nebius if you need H200 or B200 on demand. Two cases where a competitor beats Scaleway outright: a team with an existing AWS Enterprise Discount Program commitment should burn it on P5 in Stockholm, and a team that needs on-demand H200 or B200 should go to OVHcloud or Nebius, because Scaleway lists neither.

CUDO Compute moved from the middle of my shortlist to the bottom because it no longer publishes any on-demand price. I also left out IONOS, Exoscale, UpCloud and Hetzner, because none of them lists current top-tier datacenter GPUs at a published per-hour price.

Published prices mean nothing if you can't get the capacity. GPU instances in EU regions are often quota-gated, so ask for your quota and lead time in writing before you plan a migration around a rate card.

What exactly is Scaleway's AMD Instinct MI355X service, and what does it cost per hour?

As of 8 November 2026, I could not verify a self-serve Scaleway MI355X instance or an hourly price. Scaleway's public catalog API, its instance type lists for every zone, and its public docs repository contain no AMD Instinct GPU instance. If Scaleway is offering MI355X capacity, it is happening through sales, so get the rate, the zone and the quota in writing.

I refuse to invent a euro figure for it, so here is the price that matters instead. Scaleway's own same-page rates are €2.87 per GPU-hour for H100 PCIe, €3.17 for H100 SXM in the 8-GPU node, €1.47 for L40S and €7.50 for B300. An MI355X offer has to beat those on tokens per euro, and I give the threshold in the next section.

The 288 GB is real and useful. At 8 TB/s of memory bandwidth per GPU (AMD), a 70B model at FP16 sits on one GPU with room for KV cache. An 8-GPU node (2.3 TB) holds a 405B-class model at FP8 (about 405 GB of weights) with plenty of headroom, so you have no multi-node fabric to debug. Notice, though, that Scaleway's B300-SXM-8-288G already offers the same 288 GB per GPU on NVIDIA.

The Kubernetes path is a question to put to sales. Kapsule GPU pools come with the NVIDIA GPU Operator installed by default, and Scaleway's Kubernetes docs say nothing about AMD GPUs. Ask whether MI355X nodes can join a Kapsule pool, or whether you get bare instances and run the AMD GPU Operator yourself.

Teams whose serving stack depends on TensorRT-LLM, NVIDIA NIM or custom CUDA kernels should not pick MI355X at all, whatever the price.

How does the AMD MI355X compare to NVIDIA H100, H200 and B200 for LLM inference?

On published specs, the MI355X leads the H200 on memory (288 GB vs 141 GB) and bandwidth (8 TB/s vs 4.8 TB/s), and matches an 8x B200 system on MLPerf Inference Llama-2-70B. In the MLPerf Inference v6.0 closed division, AMD's 8x MI355X system scored 103,480 tokens/s offline against 103,954 for HPE's 8x B200 system. NVIDIA's lead in 2026 sits in the ecosystem far more than in raw silicon.

Sources: AMD MI355X, NVIDIA H100, H200, HGX, DGX B200, and MLCommons results for v5.0 and v6.0.

The tokens-per-euro arithmetic, with every input visible:

• Scaleway H100-SXM-8-80G: 31,306.8 tokens/s × 3,600 = 112.7 million tokens per hour. At €25.33 per node-hour, that is €0.225 per million tokens, or 4.45 million tokens per euro.
• Scaleway B300-SXM-8-288G: 115,730 tokens/s × 3,600 = 416.6 million tokens per hour. At €60.00 per node-hour, that is €0.144 per million tokens, or 6.94 million tokens per euro.
• MI355X break-even: to match the B300 on cost per token, an 8x MI355X node would need to cost under €60.00 × 103,480 / 115,730 = €53.65 per hour, which is €6.71 per GPU-hour. That is the number to hold a sales quote against.

Some honest caveats. The Blackwell and MI355X results run FP4 while the Hopper results run FP8, so part of that gap is precision rather than silicon. Vendor MLPerf entries use tuned stacks and hand-picked batch sizes, so stock vLLM will land lower on both AMD and NVIDIA. The closest thing to a stock reference is Supermicro and Red Hat's 8x MI350X entry running vLLM, at 80,478.1 tokens/s (ID 6.0-0100).

Where AMD genuinely wins: fewer GPUs per model means less tensor parallelism, fewer failure domains, simpler Kubernetes scheduling and a smaller minimum viable node. AMD also showed up in force: its v6.0 write-up lists nine partners submitting on Instinct.

Where NVIDIA genuinely wins: NVLink and NVSwitch topology, TensorRT-LLM, mature FP4 tooling on Blackwell, and a far deeper pool of answers when something breaks at 2am.

Check your own model with this rule of thumb: memory = parameters × bytes per parameter + KV cache, where KV cache = batch × sequence length × layers × 2 × KV heads × head dimension × bytes. A 70B model at FP16 is 70 × 2 = 140 GB before any KV cache.

Is ROCm ready for production inference in 2026, or is CUDA still the safe bet?

For serving open-weight LLMs, ROCm is production-viable in 2026. vLLM officially supports AMD MI200, MI300 and MI350-series GPUs, and SGLang ships an AMD platform guide, so pulling a container and serving a model works. Anything off that path, like custom kernels, TensorRT-LLM, NVIDIA NIM or some quantisation formats, is still materially easier on CUDA.

What still hurts on AMD: hand-written CUDA kernels, Triton kernels tuned for NVIDIA SMs, the AWQ and Marlin gap above, and a much thinner corpus of GitHub issues to search. The AMD GPU Operator works, but its first release came more than five years after NVIDIA started its operator repository, and most Helm charts still assume the NVIDIA one. Budget engineer-days for that difference.

My decision rule: if a container image plus vllm serve gets you to an endpoint, AMD is fine. If anyone on your team writes or patches kernels, stay on NVIDIA. Before you commit budget, check your exact GPU, OS and framework versions against the ROCm compatibility matrix, which lists the MI355X (gfx950) as of ROCm 10.1.0.

Know which migration you are signing up for. Switching a serving stack takes days. Switching a training pipeline takes months. If you run an existing TensorRT-LLM deployment or anything built on NVIDIA NIM microservices, NVIDIA is simply the better trade.

What does self-hosting inference in Europe actually cost beyond the GPU hour?

Beyond the GPU hour, the costs that decide self-hosting are idle capacity and the engineer who runs the cluster. Storage and text egress are rounding errors. Idle time is the single biggest lever, because a node billed 24/7 and used 30% of the time costs 3.3x more per token than its sticker price.

The break-even, worked through. The node costs €25.33 × 730 = €18,491 per month. At €0.90 per million tokens, the API costs the same once you push €18,491 ÷ €0.90 = 20.5 billion tokens a month. At MLPerf peak, the node can produce 112.7 million × 730 = 82.3 billion tokens a month, so break-even sits at about 25% of tuned peak. If stock vLLM gets you only half of that throughput, the same 20.5 billion tokens now take half of your node's real capacity.

Real utilisation is usually worse than teams expect. In ClearML's 2024 survey, only 7% of companies believed their GPU infrastructure exceeded 85% utilisation even at peak (ClearML). That is self-reported perception, but it matches what I see.

Cold starts are the other hidden cost. Pulling 140 GB of weights over an assumed 10 Gbit/s link takes about two minutes before vLLM even starts loading. So scale-to-zero suits batch and internal tools, and a warm baseline suits user-facing traffic.

The levers that move the maths most are scale-to-zero on GPU pools, continuous batching, FP8 quantisation and committed pricing. AWS publishes up to 66% off with Compute Savings Plans and up to 72% with EC2 Instance Savings Plans (AWS). Google Cloud publishes up to 55% for committed use (Google Cloud). Nebius offers reserved pricing through sales without publishing a percentage.

The uncomfortable conclusion: below roughly 20 billion tokens a month for a 70B-class model, self-hosting is a sovereignty, latency or control decision, and a hosted API beats it on cost. A smaller hosted model changes the comparison again, since Mistral Small 4 lists at $0.15 input and $0.60 output per million tokens (Mistral).

Who should run inference in their own cloud account instead of calling a token API?

Self-host when EU data residency, sustained volume above your break-even, latency control or ownership of fine-tuned weights forces your hand. Below the break-even volume it costs more. Running in your own cloud account keeps the bill, any committed-use discount and the data in your name.

The three honest triggers:

• Regulation. Health, finance and public-sector data with residency rules.
• Volume. Sustained traffic above the roughly 20 billion tokens a month worked out in the cost section.
• Weights. Fine-tuned or non-public models you cannot hand to a third party.

If none of those apply, use a hosted API and move on. Mistral AI, Scaleway Generative APIs and Hugging Face Inference Endpoints ($4.50 per hour for one H100 on AWS, Hugging Face) are the right answer for most teams below break-even. Frontier APIs sit far higher: OpenAI lists gpt-5.5 at $5 input and $30 output per million tokens (OpenAI), and Anthropic lists Claude Sonnet 5.5 at $2 and $10 (Anthropic).

Buyers get the jurisdiction question wrong all the time. The CJEU's Schrems II judgment (C-311/18, 16 July 2020) struck down the EU-US Privacy Shield. The EDPB's Recommendations 01/2020 then set out how to assess third-country access to transferred data. Separately, the CLOUD Act lets US authorities compel US providers to disclose data "regardless of whether such communication, record, or other information is located within or outside of the United States" (DOJ, statute). A Frankfurt region changes where the bytes live, while the operator remains a US company. That is the framing; ask your DPO for the legal conclusion.

Most teams land on a hybrid: a hosted API for bursts and experiments, a self-hosted GPU node pool for the steady baseline, and one gateway in front of both.

This is where Qovery fits. Qovery runs the Kubernetes and deployment layer inside your own Scaleway, AWS, GCP or Azure account, or your existing Kubernetes cluster. You get git-push deployments, a preview environment per pull request, auto-stop for non-production environments, managed cluster upgrades, per-environment RBAC and databases backed by managed cloud services.

Be clear on the boundary, though. Qovery does not sell GPUs, does not schedule GPUs and does not serve models. It operates what you rent, so the cloud bill, your discounts and your data residency all stay in your account. What teams buy is GPU inference on Kubernetes in their own account without a dedicated DevOps hire.

How do I deploy an inference service to my own cloud account from Claude Code or Cursor?

Install the Qovery Agent Skills once, then ask your coding agent in plain language to deploy the service into your own Scaleway, AWS, GCP or Azure account. The agent proposes each change and waits for your approval before touching anything.

Install for every compatible agent (Claude Code, Cursor, Codex, OpenCode, Gemini CLI), globally:

curl -fsSL https://skill.qovery.com/install.sh | bash

Or scope the skills to the current project:

curl -fsSL https://skill.qovery.com/install.sh | bash -s -- --project

Add the Qovery MCP Server to Claude Code (OAuth, read-only by default):

claude mcp add --transport http qovery https://mcp.qovery.com/mcp --callback-port 4242

Enable write access only when you want it. It also requires turning on write access in the Qovery Console:

claude mcp add --transport http qovery "https://mcp.qovery.com/mcp?read_write=true" --callback-port 4242

The skills are qovery-onboard, qovery-deploy, qovery-troubleshoot, qovery-optimize, qovery-speedup, qovery-preview and qovery-terraform. The skills deploy, and the MCP Server operates what already exists.

Then ask: "deploy this inference service to my own Scaleway account with Qovery". Setup details are in the AI agent quickstart and the Claude Code integration page.