How to Migrate GPU Inference to a European Sovereign Cloud: Providers, Costs, and the Platform Layer

Key Points:

• A GPU inference migration is two decisions, not one: who supplies the GPUs (Scaleway, OVHcloud, Nebius, T-Systems / Open Telekom Cloud, or IONOS in Europe, or your own on-prem cluster) and what platform layer runs your containers on top. Teams that collapse them into one decision stall on the second.
• An EU region of a US hyperscaler is not sovereignty. The US CLOUD Act lets US authorities compel a US-headquartered provider to hand over data it holds abroad, so jurisdiction follows the parent company, not the datacenter address.
• Performance parity is realistic because the silicon is identical. An NVIDIA H100 SXM in Paris or Frankfurt delivers the same tensor throughput as one in Virginia, and serving EU users from an EU region usually lowers p95 time-to-first-token. The real risks are GPU quota lead times, InfiniBand vs Ethernet interconnect, cold-start storage bandwidth, and thinner managed-service catalogs.
• Published EU H100 prices are competitive with AWS, but utilization dominates the bill. On-demand per-GPU-hour pricing from Scaleway, Nebius, and IONOS lands at or below AWS p5, and an idle GPU node costs exactly as much as a saturated one, so continuous batching and scaling non-production GPUs to zero beat price shopping.
• The portable path is containers on Kubernetes. Serve with vLLM, TGI, or Triton, pin your NVIDIA GPU Operator driver version, and keep deployment cloud-agnostic so a future provider change is a cluster swap, not a rewrite.
• Qovery is the platform layer, not the GPU supplier. Qovery deploys and operates your inference services inside your own Scaleway, AWS, GCP, or Azure account, or your own existing Kubernetes cluster (including a sovereign EU or on-prem one), so developers keep the same git-push workflow and the cloud contract stays in your name.

Choosing the Frankfurt region of a US hyperscaler does not make your workload sovereign. Three US companies - Amazon, Microsoft, and Google - hold 70% of the European cloud infrastructure market, with European-headquartered providers stuck at 15% (Synergy Research Group, full-year 2024). The datacenter sits in Germany; the legal entity that controls it answers to a US court.

I have spent years running Kubernetes in production and talking to engineering leaders across Europe, and the GPU-inference version of this question now comes up almost weekly: can we move H100 inference off a US cloud without losing performance, and who actually helps with the move? Here is the honest playbook - which EU providers have the GPUs, what a GPU-hour costs, whether you will be slower, and which platforms specialize in getting you there.

What does "European sovereign cloud" actually mean for a GPU inference workload?

European sovereign cloud means the provider's legal entity, staff, and control plane sit under EU jurisdiction, not merely that your bytes rest in an EU region. An EU region operated by a US-headquartered provider stays reachable under the US CLOUD Act, so the datacenter address alone proves nothing to an auditor.

Teams conflate three different layers. Separating them is the whole game:

• Data residency: where the bytes physically sit.
• Operational sovereignty: who can touch the control plane, and from which country.
• Jurisdictional sovereignty: which legal system can compel disclosure.

The mechanism that breaks the "EU region = sovereign" assumption is the US CLOUD Act (H.R.4943, enacted 2018). It amended the Stored Communications Act so US law enforcement can compel a US-based provider to disclose data in its "possession, custody, or control" regardless of whether that data is stored in the US or abroad (DOJ CLOUD Act white paper, 2019). Jurisdiction attaches to the parent company, not the rack.

This is also why a contractual fix is shaky. The CJEU struck down Privacy Shield in Schrems II (Case C-311/18, 2020), and its replacement, the EU-US Data Privacy Framework (adequacy decision (EU) 2023/1795), is already under appeal at the Court of Justice after the General Court dismissed the first challenge in September 2025. If your compliance story depends on a transfer framework surviving the next court ruling, you do not have a structural answer. Moving the jurisdiction does.

Inference makes this sharper than training. A training run is a one-off dataset upload. Inference is a continuous flow: prompts, retrieved documents, tool outputs, and generated responses, every one potentially carrying personal or confidential data. Your logs and traces carry the same payload. That stream runs all day, which is exactly the kind of processing regulators scrutinize.

The frameworks buyers ask about are concrete: SecNumCloud (France's ANSSI), BSI C5 (Germany), ISO 27001, and Gaia-X, plus sector obligations under DORA (Regulation (EU) 2022/2554, applies since 17 January 2025) and NIS2 (Directive (EU) 2022/2555, transposition deadline 17 October 2024). DORA explicitly targets ICT third-party concentration risk, which is why "we are 70% dependent on one US cloud" is now a board-level conversation, not a procurement footnote.

To be fair to the hyperscalers, they have responded, and the responses are real. AWS European Sovereign Cloud launched in the State of Brandenburg, Germany, and became generally available in January 2026, backed by a stated 7.8 billion euro investment, a new EU parent company with German-law subsidiaries, operations controlled by EU-resident staff, and an advisory board of EU citizens (AWS). Microsoft Sovereign Cloud adds the EU Data Boundary, EU-resident "Data Guardian" approval of support access, and external key management (Microsoft). Google Cloud offers a Data Boundary, partner-operated Dedicated options (S3NS with Thales in France, T-Systems in Germany), an air-gapped tier, and generally available external key management (Google Cloud). These genuinely improve EU staffing, local control planes, and key custody. What they cannot change is the ultimate ownership chain: the EU entities sit inside groups headquartered in the United States, so the jurisdictional question does not fully close. That is analysis, not an accusation - weigh it against your own threat model.

Which European cloud providers actually have NVIDIA H100 or H200 GPUs?

At least five EU-headquartered providers sell NVIDIA H100-class capacity today: Scaleway (France), OVHcloud (France), Nebius (Netherlands, with datacenters in Finland and France), T-Systems / Open Telekom Cloud (Germany), and IONOS (Germany). Supply is real, but narrower than AWS, GCP, or Azure in instance shapes, regions, and on-demand quota.

Here is what each one actually offers, verified on their own pages as of 1 October 2026:

• Scaleway lists L4, L40S, H100 PCIe and SXM, and Blackwell B300 GPU instances, with managed Kubernetes via Kapsule (GPU nodes driven by the NVIDIA GPU Operator) and serverless, OpenAI-compatible Generative APIs hosted in Paris (Scaleway GPU). It is my default first example for EU inference because the lineup, the managed cluster, and the inference API all sit in France.
• OVHcloud offers L4, L40S, H100, H200, and A100 80GB instances, is headquartered in Roubaix, France, and trades on Euronext Paris under the ticker OVH (OVHcloud GPU).
• Nebius publishes per-GPU-hour pricing for HGX H100, H200, B200, and B300, is headquartered in Amsterdam and listed on Nasdaq as NBIS, and runs EU datacenters in Finland and France (Nebius). Describe Nebius from its own filings: it reports capacity in megawatts, not a single company-wide GPU count, and its Finnish Mäntsälä site alone is scaling toward "up to 60,000 GPUs" (Nebius newsroom).
• T-Systems / Open Telekom Cloud is the German operator here, and Deutsche Telekom is adding serious GPU weight: the Industrial AI Cloud with NVIDIA in Munich is a one-billion-euro partnership targeting up to 10,000 NVIDIA Blackwell GPUs in its first phase, across more than a thousand DGX B200 and RTX PRO server systems, going live in Q1 2026 (Deutsche Telekom). Verify the current public GPU instance catalog before you commit to a specific SKU.
• IONOS sells H200 GPU VMs at a flat per-GPU rate and runs an OpenAI-compatible AI Model Hub in German datacenters (IONOS AI Model Hub). StackIT (Schwarz Group) adds H100, A100 80GB, and L40S instances with its own Kubernetes engine (StackIT GPU).

For batch inference and research, the EuroHPC supercomputers are a genuine option, though allocation-based rather than on-demand: JUPITER in Jülich pairs roughly 24,000 NVIDIA GH200 Grace Hopper superchips and is Europe's first exascale machine, Leonardo in Bologna runs 13,824 A100-class GPUs, and LUMI in Finland is AMD Instinct-based. Access comes through competitive EuroHPC calls, free for open science and pay-per-use for industry (EuroHPC).

Certifications are where marketing and reality diverge, so check the attestation, not the homepage. 3DS Outscale holds full SecNumCloud 3.2 qualification; OVHcloud holds SecNumCloud 3.2 on its Bare Metal Pod offering (OVHcloud); and Scaleway's SecNumCloud offer is listed by ANSSI as in progress, not yet qualified (ANSSI SecNumCloud providers). On the German side, Open Telekom Cloud has held BSI C5 Type 2 every year since 2018, StackIT holds C5 Type 2, and IONOS holds C5 Type 1 (Schwarz Digits on StackIT C5 Type 2).

Plan for the real constraints: GPU quota approval and reservation lead times, fewer instance shapes, no InfiniBand on some tiers, thinner managed-database and observability catalogs, and a smaller regional footprint than a hyperscaler.

Will inference performance drop if I move from a US hyperscaler to a European sovereign cloud?

For most inference workloads, no. The GPUs are the same NVIDIA parts, so tokens per second per GPU is comparable, and serving EU users from Paris or Frankfurt usually lowers p95 time-to-first-token versus us-east-1. The gaps that do appear are in interconnect, storage bandwidth, and autoscaling maturity, not raw FLOPS.

The same-silicon point is not a hand-wave. MLPerf Inference is a standardized, MLCommons-verified benchmark, and in round v4.1 an 8x H200 system served Llama 2 70B at 32,790 tokens/second in the Server scenario (NVIDIA on MLPerf Inference v4.1). An H200 in Finland and an H200 in Virginia are the same board running the same CUDA stack; they produce the same throughput. The provider does not change the physics.

Latency usually moves in your favor for EU users. Microsoft's published median backbone figures put East US to West Europe at roughly 83 to 85 ms round trip, versus 11 to 13 ms within Europe (Azure network round-trip latency). Routing EU traffic to a US-East region adds 80-odd milliseconds of transatlantic round trip before the model emits a single token. Measure your own p95 time-to-first-token from your real user geography rather than trusting any single number, but the direction is clear.

Where real gaps show up:

• Interconnect: InfiniBand versus Ethernet matters for multi-GPU, tensor-parallel serving. Confirm the tier before you assume parity on a 70B+ model split across GPUs.
• NVLink: SXM variants have more inter-GPU bandwidth than PCIe. For large tensor-parallel models, pick SXM.
• Storage read bandwidth: this dictates cold-start time for a 100 GB+ checkpoint. Thin object-storage throughput turns a scale-up event into a slow one.
• Autoscaling maturity: hyperscaler autoscalers are more battle-tested than some EU equivalents, which is exactly where a platform layer or KEDA/Karpenter earns its place.

Before you benchmark anything, define the five metrics every migration must compare: tokens/sec per GPU, time-to-first-token, p95 end-to-end latency, GPU utilization, and cost per million tokens. Then recover any difference in the serving stack, which usually dwarfs the provider gap. vLLM reports up to 24x higher throughput than HuggingFace Transformers and up to 3.5x over Text Generation Inference from continuous batching and PagedAttention (vLLM launch blog; the peer-reviewed PagedAttention paper reports 2-4x over FasterTransformer and Orca, arXiv 2309.06180). FP8 quantization on H100 adds roughly 1.4x to 2.3x depending on model, batch size, and latency target (NVIDIA TensorRT-LLM).

The method I trust: shadow-deploy on the candidate EU provider and replay a week of real production traffic before you commit. Pin the CUDA and driver versions so you are comparing the same stack on two clouds, not two different stacks.

What is the step-by-step migration path for GPU inference workloads?

Run it as a seven-step playbook: inventory, containerize on an open runtime, standardize on Kubernetes with the NVIDIA GPU Operator, move weights into EU storage, rebuild the in-region plumbing, abstract deployment, then dual-run and cut over. The model is almost never the hard part. The data, identity, and deployment plumbing around it is, and GPU quota approval usually sets the critical path.

1. Inventory. List the models and quantizations in production, the GPU SKUs and counts, your current tokens/sec and p95, and the data classes flowing through prompts, retrieved context, and logs. This is what prevents mid-migration surprises and wrong instance sizing.
2. Containerize on an open runtime. Serve with vLLM, Hugging Face TGI, NVIDIA Triton, or Ollama for small models, so nothing is bound to a proprietary inference API you cannot take with you.
3. Standardize on Kubernetes with the NVIDIA GPU Operator. Use the device plugin and pin your driver version against the operator's tested platform support matrix so one image schedules on any provider's GPU nodes (NVIDIA GPU Operator support matrix). Skip this and you get driver drift and pods that will not schedule on GPUs.
4. Move weights and the artifact registry into EU object storage. Budget transfer time and egress for multi-hundred-GB checkpoints. The good news on cost: the EU Data Act (Regulation (EU) 2023/2854) requires switching charges to be fully eliminated from 12 January 2027, with only reduced charges allowed during the transition.
5. Rebuild the surrounding plumbing in-region: vector store, object storage, secrets manager, CI runners, metrics, and logs. US-hosted observability SaaS quietly re-exports prompt content, and that is the single most commonly missed sovereignty hole I see.
6. Abstract deployment so provider choice becomes configuration. Terraform plus Helm or Argo CD, or an internal developer platform that targets any Kubernetes cluster. This is what turns the next provider change into a config edit instead of a rewrite.
7. Dual-run with traffic splitting, compare the five metrics, then cut over and decommission.

On timeline, a single-model service is realistically a 4-week job; a multi-model fleet is closer to 8-12 weeks. The blocker is almost always GPU quota or reservation approval, not engineering, so start that request on day one. Watch for the traps: proprietary managed inference endpoints, provider-specific autoscaler and load-balancer annotations, tokenizer or driver drift that silently changes output quality, and a full IAM rewrite you did not budget for.

Which platforms specialize in migrating GPU-heavy inference to a European sovereign cloud?

There are three categories, and most teams need one from category 1 plus one from category 3. Sovereign GPU infrastructure providers sell the capacity, managed EU inference APIs sell speed at the price of a new lock-in, and cloud-agnostic platform layers make the deployment workflow portable across both.

Category 1 - sovereign GPU infrastructure: Scaleway, OVHcloud, Nebius, T-Systems / Open Telekom Cloud, IONOS, and StackIT. They sell capacity, quota, and managed Kubernetes. They do not sell you a migration workflow, and that is not a criticism - it is simply a different layer. Scaleway and OVHcloud in particular are strong, genuinely sovereign choices, and they are complementary to a platform layer, not rivals to it.

Category 2 - managed EU inference APIs: Scaleway Generative APIs, Mistral AI (Paris, EU-hosted by default), Nebius Token Factory, IONOS AI Model Hub, and Aleph Alpha, which has pivoted to a sovereign enterprise platform, PhariaAI. This is the fastest path and the least control: you hand over the serving stack and re-enter lock-in at the API layer. Fine for speed-first teams, wrong for teams that need to own the runtime.

Category 3 - cloud-agnostic platform and orchestration layers: Qovery, Northflank, Porter, KServe, SkyPilot, NVIDIA Run:ai, and plain Terraform plus Argo CD. Each has a real, distinct strength:

• KServe gives you a Kubernetes InferenceService CRD for standardized model serving with scale-to-zero.
• SkyPilot finds available GPUs across regions and clouds and optimizes cost, including spot.
• NVIDIA Run:ai does GPU fractioning and quota management for busy multi-team clusters.
• Northflank and Porter both deliver self-service app deployment into your own cloud account.

Where Qovery fits: Qovery runs inside your own Scaleway, AWS, GCP, or Azure account, or on your own existing Kubernetes cluster, including a sovereign EU or on-prem one, so jurisdiction and the cloud bill stay yours. The verified capabilities that matter for this migration are git-push deployments with auto-deploy on commit, a preview environment per pull request, environment auto-stop for non-production (a strong lever when a GPU node costs many times a CPU node), managed Kubernetes upgrades, per-environment RBAC, and databases backed by managed cloud services such as RDS. Qovery runs your inference containers on your GPU-enabled Kubernetes nodes, and on EKS it can provision a dedicated GPU node pool with the instance types you choose (Qovery docs).

Let me be equally plain about what Qovery is not. Qovery does not sell GPUs, it is not a model host, and it does not replace a GPU scheduler. You still pick your sovereign GPU provider and negotiate the quota. The honest trade-off of bring-your-own-cloud is that owning the cloud relationship is more work than consuming an API key - and it is the only route to real jurisdictional sovereignty, because the contract and the control stay in your name.

Is a European sovereign cloud more expensive than AWS for GPU inference?

No, not on published list prices. On-demand per-GPU-hour pricing from Scaleway, Nebius, and IONOS is broadly competitive with, and in several cases below, AWS p5. The decisive variable is utilization, because idle GPU time wastes far more money than any price gap between providers.

Start with the list prices, all checked on 1 October 2026. Scaleway publishes H100 PCIe from EUR 2.73/GPU-hour, Nebius lists HGX H100 at USD 3.85 and H200 at USD 4.50, IONOS charges a flat EUR 3.00/GPU-hour for H200, and OVHcloud shows H100 from USD 2.99. AWS p5.48xlarge packs 8x H100 at a widely tracked USD 55.04/hour on-demand, which works out to roughly USD 6.88 per GPU-hour (AWS p5). The EU options are not a compromise on price.

Two caveats before you build a budget on those numbers. First, provider pricing pages are the source of truth and they change - confirm the live page, and for AWS p5e and p5en (H200) specifically, check AWS's current pricing page rather than any tracker. Second, at H100/H200 fleet scale EU providers frequently require reservations, so lead time is part of both the cost model and the schedule; the AWS equivalent is Capacity Blocks for ML, which you reserve up to 8 weeks ahead for 1 to 14 days (or multiples of 7 up to 182 days), paid upfront and non-cancellable (AWS Capacity Blocks).

Now the part that actually moves the bill. An idle GPU node costs exactly as much as a saturated one, and most organizations run their GPUs far below the line: in the ClearML and AI Infrastructure Alliance "State of AI Infrastructure at Scale 2024" survey, only about 7% of organizations reported exceeding 85% GPU utilization at peak (ClearML). The levers that beat price shopping are all utilization levers: vLLM continuous batching, request queueing, consolidating small models onto one GPU with MIG (up to 7 isolated instances per H100) (NVIDIA MIG) or time-slicing, and scaling non-production GPU environments to zero.

Budget the one-time migration costs honestly too: egress for weights and datasets (shrinking under the Data Act), dual-run duplication of observability, re-certification and audit work, and engineer time on driver and operator plumbing. This is where a platform layer pays for itself - environment auto-stop on non-production GPU environments, and far fewer DevOps hours maintaining bespoke deployment scripts across two providers.

As an illustration, not a benchmark: at the MLPerf-style figure of roughly 4,000 tokens/sec per H200 and IONOS's EUR 3.00/GPU-hour, a fully saturated GPU serves about 14.4 million tokens per hour, or roughly EUR 0.21 per million tokens at 100% utilization. Drop utilization to 30% and the same work costs about EUR 0.69 per million tokens. The provider price barely moved; your utilization tripled the unit cost.

Indicative European sovereign cloud GPU pricing versus AWS, as of 1 October 2026. Published on-demand list prices in the provider's quoted currency; confirm the live page before budgeting.

How do you prove sovereignty to auditors and customers after the migration?

You prove it component by component: document the legal entity, region, subprocessors, and admin-access path for every hop in an inference request, including logs, metrics, CI/CD, backups, and vendor support. Compute in Paris means nothing if your error tracker ships prompt text to Ohio.

Start with a data-flow map. Trace one inference request from client to gateway to serving pod to vector store to object storage to logs, metrics, and backups, and for each hop record the legal entity, the region, and the subprocessor list. The map is both your design review and your audit artifact.

Then audit subprocessors hard. A US-hosted observability, error-tracking, or feature-flag SaaS reintroduces transfer exposure for prompt content even when compute runs in France. This is the hole I see most often, and it hides in the tools nobody thinks of as "data processors."

Operational sovereignty is where claims usually fail: who can reach the control plane, from which country, under what break-glass procedure, and whether support is EU-staffed. Pair that with the right paperwork - SecNumCloud, BSI C5, or ISO 27001 attestations for the provider, plus your DORA and NIS2 obligations - and refresh the GDPR artifacts: your record of processing, a DPIA for the inference pipeline, and a transfer impact assessment kept on file even for an EU-only setup.

Make the evidence pack reproducible. Per-environment RBAC, deployment audit logs, and infrastructure-as-code history are what auditors actually accept, and a platform layer produces them consistently across every environment instead of as a one-off scramble before each audit.