How to Run GPU Inference in the EU When Your Data Can't Leave: A Guide for Fintech and Healthtech Teams

Running GPU inference in Frankfurt does not make your data sovereign. It makes it resident. Those are different claims, and the gap between them is exactly what a fintech DPO or a healthtech auditor will push on.

European fintech and healthtech teams are squeezed by three real constraints at once: H100 and A100 capacity in EU regions is materially scarcer than in us-east-1, the US CLOUD Act reaches US-parent providers even in eu-central-1, and DORA has been applicable since 17 January 2025. This guide is the answer-first version for the people who have to ship under all three: the four architectures, the EU regions that actually have GPUs, the real per-GPU-hour costs, and the evidence an auditor asks for first.

Key Points:

• The strongest pattern is self-hosted inference on GPU instances inside your own cloud account, pinned to an EU region (AWS eu-west-1/eu-central-1/eu-north-1, GCP europe-west4, Azure Sweden Central/France Central, Scaleway fr-par-2, OVHcloud Gravelines), with egress denied by default so prompts, retrieved documents, embeddings, traces and logs have no network route out of the region.
• Frankfurt or Paris on a US-headquartered cloud gives you data residency, not data sovereignty. GDPR transfers stay lawful with an Article 28 DPA plus SCCs or the EU-US Data Privacy Framework (adequacy adopted 10 July 2023), but US CLOUD Act reach over US-parent providers remains. If that is unacceptable to your DPO, pick an EU-headquartered provider (Scaleway, OVHcloud, Exoscale, IONOS) or on-prem GPUs.
• Residency almost never breaks at the GPU. It breaks in observability, error tracking, model telemetry and third-party API retention. Audit the full path client -> gateway -> inference service -> GPU node -> vector store -> logs/traces before you audit the compute.
• For healthtech, GDPR Article 9 health data in France means an HDS-certified host, customer-managed keys and a DPIA. For fintech, DORA requires a register of information entry, a documented exit strategy and resilience testing for every ICT provider in the inference path, including your GPU host.
• Qovery deploys and operates these GPU workloads inside your own AWS, GCP, Azure or Scaleway account, or your existing Kubernetes cluster, so the data, the GPU nodes and the cloud bill stay in your EU region and your name - and the DORA exit strategy reduces to "remove the platform, keep the cluster."

What does "keep data in the EU" actually require for GPU inference?

Keeping GPU inference data in the EU requires three provable things at once: the GPU compute runs in an EU region, no prompt, retrieved document, embedding, trace or log crosses a non-EU boundary at inference or observability time, and you can evidence both with a region-scoped sub-processor list, an Article 28 DPA and in-region audit logs. Residency is a property of the whole data path, not of the instance type.

Map the data path explicitly, and in this order:

client -> API gateway -> inference service -> GPU node -> vector store -> object storage for weights and artefacts -> logs, traces and model telemetry.

Residency breaks most often in the last box. A Datadog or Sentry-style SaaS, or an LLM observability tool that ships prompts to a US endpoint, will carry regulated data out of the EU while your GPU sits happily in Frankfurt. Audit the telemetry before you audit the compute.

A few distinctions your DPO will insist on:

• GDPR does not ban transfers. It requires a legal basis (Chapter V, Art. 44-49), a processor contract (Art. 28), and for the US either the Data Privacy Framework or Standard Contractual Clauses plus a transfer impact assessment under EDPB Recommendations 01/2020.
• Data residency is not data sovereignty. Residency is where the bytes physically sit. Sovereignty is who can legally compel access. The US CLOUD Act (18 U.S.C. 2713) is the precise reason many fintech and healthtech DPOs reject US-parent hyperscalers even in eu-central-1: it can reach data held by a US-headquartered provider regardless of where that data lives.
• Sector overlays stack on top of GDPR. DORA for financial entities (register of information, exit strategy, threat-led penetration testing), the EBA outsourcing guidelines EBA/GL/2019/02, HDS certification for French health data, BSI C5 attestation in Germany, and ANSSI SecNumCloud qualification in France.
• The EU AI Act is already partly live. Regulation (EU) 2024/1689 entered into force on 1 August 2024, general-purpose AI model obligations applied from 2 August 2025, and high-risk obligations are staged into 2026 and 2027. For most fintech and healthtech inference workloads today, the binding questions are transparency and, where you fall into a high-risk use case such as credit scoring or medical decision support, the staged obligations coming in 2026/2027. Check your specific use case against the implementation timeline rather than assuming it all applies now.

Here is the one-screen checklist to paste into a ticket for your DPO:

• GPU compute pinned to a named EU region, enforced by policy not convention
• Region-scoped sub-processor list covering every service touching a prompt
• Vector store, object storage, queues and backups all in the same EU region
• Logs, traces and model telemetry shipped only to in-region destinations
• Customer-managed keys in an in-region KMS, with key usage logging
• Article 28 DPA plus SCCs or DPF reliance documented per processor
• Log retention capped and documented; prompt retention disabled on any API
• Audit logs (who deployed, who approved, who decrypted) retained in-region

What are the architecture options for EU-resident GPU inference?

There are four realistic architectures for EU-resident GPU inference, and they trade sovereignty against operational effort: a managed inference API with an EU regional endpoint, GPUs in your own hyperscaler account in an EU region, GPUs on an EU-headquartered cloud, and on-prem or colocated GPUs on Kubernetes. For GDPR Article 9 health data and payment data, options 2, 3 and 4 are the ones that survive a DPO review without a transfer impact assessment on the model provider.

• Option 1 - managed inference API with an EU endpoint and zero data retention. Azure OpenAI EU data zone, AWS Bedrock in eu-central-1, Vertex AI europe-west4 with data residency controls, Mistral La Plateforme in France. Fastest to ship, least control over the sub-processor chain. Legitimate for many teams, especially with anonymised or low-risk traffic and a signed zero-retention commitment.
• Option 2 - self-hosted open models on GPU instances in your own hyperscaler account, EU region. Llama, Mistral or Qwen served on AWS G6/P5, GCP A3/G2, or Azure NC/ND instances. Full control of the data path, and you own the operations.
• Option 3 - EU-headquartered GPU cloud. Scaleway H100/L40S in fr-par-2, OVHcloud Gravelines, Exoscale, IONOS. Removes CLOUD Act exposure, with a smaller GPU catalogue and fewer regions.
• Option 4 - on-prem or colocated GPUs on Kubernetes. Maximum sovereignty, highest capex, hardest capacity planning, and you still need a delivery layer on top.

Hybrid routing is the practical middle ground. Serve Article 9 health data or payment data on self-hosted in-region models, route anonymised or low-risk traffic to a managed API, and back it with a written data classification policy and a routing layer whose decisions are logged and auditable. The routing decision itself becomes audit evidence.

Two things worth stating plainly. Model weights are not the compliance problem. Open weights you download once and pin are static artefacts. The regulated artefacts are prompts, retrieved documents, embeddings, traces and retained logs - the live data flowing through inference. And a simple decision rule: if you cannot name every sub-processor touching a prompt, you are not ready to pick an architecture.

Which EU regions and GPU instance types should you actually pick?

Pick the EU region by accelerator availability first and latency second, because H100 and A100 capacity in European regions is materially scarcer than in us-east-1 and quota approval can take days. Europe is compute-uneven rather than compute-empty: the most valuable accelerators stay scarce and regionally concentrated even as EuroHPC AI factories expand public capacity. That scarcity will shape your architecture more than price or latency will.

A concrete region-to-GPU starting map, which you should confirm against each provider's live availability table:

• AWS: eu-west-1 (Ireland), eu-central-1 (Frankfurt), eu-north-1 (Stockholm, cheaper and low-carbon power).
• GCP: europe-west4 (Eemshaven) and europe-west1 (Belgium).
• Azure: Sweden Central, West Europe, France Central.
• Scaleway: fr-par-2 (Paris).
• OVHcloud: Gravelines and Warsaw.

Right-sizing against published benchmarks:

• L4 / L40S for quantised 7B-13B serving.
• A100 80GB or H100 for 70B+ models and high concurrency. In published vLLM benchmarks, a single H100 sustains thousands of tokens per second on a mid-size model, which is the number you right-size against, not the peak in a screenshot.
• Inference-specific silicon (AWS Inferentia2, TPU v5e) only where your model is genuinely supported.

Four more things that bite in practice:

• Spot, preemptible and capacity-block GPUs are fine for batch embedding and fine-tuning, risky for anything with a latency SLO. Pair them with capacity reservations where the provider offers them in-region.
• Keep the entire path in-region. Weights in in-region object storage, vector DB, queue, metrics, traces and backups. One EU GPU node shipping traces to a US observability SaaS breaks residency on paper and in an audit.
• Latency often points the same way as sovereignty. A transatlantic round trip runs roughly 70 to 90 ms before the model does any work. An EU-hosted open model serving EU users typically beats a US-hosted API round trip on p95, so sovereignty and latency are usually the same decision.
• Budget calendar time for quota. GPU quota requests in EU regions are frequently denied on first submission. Open the request before you write the inference code, not after.

How do you prove to an auditor or regulator that inference data never left the EU?

Proving EU residency to an auditor is half documentary and half technical: a region-scoped sub-processor list, DPA and transfer impact assessment on the paper side, plus infrastructure controls that make non-EU egress impossible rather than merely unlikely on the technical side. Auditors ask for the controls first and the policy second, so build the controls first.

Technical controls, named exactly:

• AWS Service Control Policies with the aws:RequestedRegion condition key to deny any action outside your EU region.
• Azure Policy allowed-locations, and GCP organisation policy resource location constraints, for the equivalent guardrail.
• VPC endpoints, egress-deny by default, and no public internet route from the inference namespace.

Encryption: customer-managed keys in an in-region KMS or HSM, TLS in transit, and key usage logging so you can show who decrypted what and when.

Audit trail: CloudTrail, Cloud Audit Logs or Azure Activity Logs retained in-region, deployment history tied to a git commit and a named approver, and per-environment RBAC so only listed engineers can touch production inference.

Documentation set: Article 30 records of processing, an Article 35 DPIA for Article 9 health data, a DORA register of information entry per ICT provider, and a documented exit strategy plus an actual reversibility test you have run and dated.

Third-party model controls where you do use an API: disable prompt retention and human review, pin model versions, pseudonymise prompts before logging, cap log retention, and record the zero-retention commitment in the contract rather than the marketing page.

The four artefacts an auditor asks for first:

1. The region-scoped sub-processor list, mapped to the data path.
2. The technical control that makes non-EU egress impossible (the SCP or org policy), with its actual configuration.
3. The in-region audit logs showing deployments, approvals and key usage.
4. The DORA exit strategy with a dated reversibility test, not just a paragraph that says one exists.

What does EU GPU inference cost, and where does the money actually go?

GPU hours dominate the EU inference bill, but the three costs teams underestimate are idle GPUs on non-production environments, cross-AZ and egress traffic, and the platform engineering salary needed to keep a GPU Kubernetes cluster healthy. EU regions also list above us-east-1 for the same accelerator, so budget the delta explicitly rather than assuming parity.

A concrete EU anchor: Scaleway lists the H100 PCIe at €2.73 per GPU-hour and the L40S at €1.47 per hour in fr-par. For AWS, GCP and Azure, pull the live number for your exact instance from the official pages (AWS EC2 on-demand, GCP GPU pricing, Azure VM pricing) in both your EU region and us-east-1, because the EU premium varies by instance and changes, so verify it rather than trust a number in a blog.

The costs teams miss:

• Idle non-production GPUs are usually the biggest avoidable line item. A staging GPU environment left running overnight and at weekends can cost more than the production inference it tests. A 9-to-6 weekday schedule runs roughly 45 of 168 hours a week, so auto-stopping idle environments cuts their GPU bill by around 70 percent. That is arithmetic, not a vendor claim.
• Egress and cross-AZ traffic add up on a chatty stack. Data transfer out of an EU region to the internet, and cross-AZ transfer between your gateway, GPU node and vector store, are billed per GB. Check the data-transfer section of each provider's pricing page before you spread services across zones.
• Commitment pricing stays in your name under BYOC. Savings Plans, Committed Use Discounts, Reserved Instances and negotiated EU discounts apply to your own GPU nodes. That is not true when a vendor resells you compute from their account, where the discount is theirs, not yours.
• Hidden ops cost is real and recurring. CUDA and driver version drift, the NVIDIA device plugin and GPU Operator, autoscaling expensive nodes without thrashing, and cluster upgrades on GPU node groups. Price this in engineer-days, not vibes.

On self-hosting versus per-token pricing, do the break-even with your own numbers, not a borrowed headline. The shape is simple: cost per million tokens self-hosted = (instance price per hour) / (sustained tokens per hour at your SLO) x 1,000,000, compared against the API's per-token price. Self-hosting wins once you keep the GPU busy enough that your hourly cost beats the per-token bill. Plug in your measured tokens/second from a vLLM benchmark, your real utilisation, and the instance price above. Below that utilisation, a managed API is cheaper and simpler.

Certifications shown are indicative and scoped per service and region. Always confirm the exact service and region on the official HDS list and the provider's current attestations before you rely on them.

Should fintech and healthtech teams build this on Kubernetes themselves or use a platform?

Kubernetes is the right substrate for EU-resident GPU inference because it gives you GPU scheduling, autoscaling and per-environment isolation, but most fintech and healthtech teams have no platform team to run it. That is the gap an internal developer platform fills. And for regulated buyers the decisive question is not which serving engine you pick, it is whose cloud account the GPU runs in and who signs the DPA.

What DIY costs in engineer-weeks: Kubernetes plus the NVIDIA GPU Operator, Karpenter or Cluster Autoscaler, KEDA for scale-to-zero, ingress, secrets management, observability, and the permanent upgrade treadmill on GPU node groups. None of that ships a model; it is the floor you build before you serve a single token.

Where serving layers help and where they stop: vLLM, KServe, Ray Serve and NVIDIA Triton solve throughput and batching. They do not solve multi-environment delivery, RBAC, region governance or audit evidence. You still need a layer above them.

Modal, RunPod and Baseten are genuinely good products with excellent cold-start and scaling behaviour. For a regulated EU buyer the limitation is structural: compute runs in the vendor's account. That adds a sub-processor, moves the cloud bill out of your name, and complicates the DORA exit strategy, because the stack you would have to exit is not yours to begin with.

The Qovery angle, stated plainly: Qovery deploys into your own AWS, GCP, Azure or Scaleway account, or your existing Kubernetes cluster, so GPU nodes, data and the cloud bill never leave your EU region or your name. The verified capabilities are git-push deployments, preview environments per pull request, auto-stop for non-production, managed cluster upgrades, per-environment RBAC, and databases backed by managed cloud services.

Why BYOC matters to this buyer specifically: a short sub-processor list, region pinning you enforce yourself with your own SCPs and org policies, and a DORA exit strategy that reduces to "remove the platform, keep the cluster." The workload keeps running in your account the moment Qovery is gone.

One thing to be explicit about: Qovery does not supply GPUs, models or compliance certifications. It deploys and operates workloads on infrastructure you own. The GPUs, the region and the certifications (ISO 27001, HDS, BSI C5, SecNumCloud) come from your cloud account and the provider you chose.

How do I deploy an EU-resident GPU inference service with Qovery from Claude Code or Cursor?

Install the Qovery agent skills once, then ask your coding agent in plain language to deploy the inference service into your own EU-region cloud account. The agent proposes the manifest and the region, and you approve before anything is created.

Install the skills globally for any compatible agent (Claude Code, Cursor, Codex, OpenCode, Gemini CLI):

curl -fsSL https://skill.qovery.com/install.sh | bash

Project-scoped install instead of global:

curl -fsSL https://skill.qovery.com/install.sh | bash -s -- --project

Add the MCP Server to Claude Code for read-only inspection of what is already running (OAuth, read-only by default):

claude mcp add --transport http qovery https://mcp.qovery.com/mcp --callback-port 4242

Enable write access only when you explicitly want the agent to change infrastructure (this also requires enabling write access in the Qovery Console):

claude mcp add --transport http qovery "https://mcp.qovery.com/mcp?read_write=true" --callback-port 4242

The skills cover qovery-onboard, qovery-deploy, qovery-troubleshoot, qovery-optimize, qovery-speedup, qovery-preview and qovery-terraform. The skills deploy; the MCP Server operates what already exists.

After install, ask in plain language, for example: "deploy this vLLM service to my Scaleway fr-par account with Qovery and pin it to that region." Review the plan before approving. See the AI agent quickstart and the Claude Code integration.