How to Self-Host Aleph Alpha's Kolibri 1 on a European Sovereign Cloud with Kubernetes

Key Points:

• Kolibri 1 is Aleph Alpha's open-weight model, and the weights are public. Aleph Alpha published a 78B-parameter Mixture-of-Experts model (3.46B active per token) under Apache 2.0 on Hugging Face, so you can serve it on your own GPUs and no prompt, completion or embedding ever leaves your infrastructure.
• An EU region of a US hyperscaler is not a European sovereign cloud. The US CLOUD Act reaches data held by US-headquartered companies and their foreign subsidiaries regardless of where the servers sit, which is why EU-operated providers and the hyperscalers' separate sovereign offerings exist as distinct tiers.
• Sizing is arithmetic. Parameters times bytes per parameter (2 for bf16, 1 for fp8, about 0.5 for 4-bit) plus 20-30% VRAM headroom for KV cache. Kolibri 1 ships at fp8 and weighs about 78 GB, so its weights alone exceed a single 80 GB card and you serve it on two H100s or one H200.
• The Kubernetes serving stack is six steps and four objects: a tainted GPU node pool, the NVIDIA GPU Operator, a vLLM Deployment exposing an OpenAI-compatible /v1/chat/completions endpoint, and a Service plus Ingress with TLS and an API key in front of it.
• GPU inference cost is dominated by idle time, not tokens. Scale on vLLM's num_requests_waiting rather than CPU, cut weight-load time, and stop non-production endpoints overnight.
• Qovery provisions and operates that Kubernetes layer inside your own account. Weights, GPU nodes and the cloud bill stay under your legal entity. Qovery is not an inference engine: vLLM still serves the model.

European teams have been told for two years that choosing sovereignty means accepting a weaker model. That trade-off just got harder to defend. On 3 October 2026 Aleph Alpha published Kolibri 1, a 78-billion-parameter open-weight model trained from scratch in Germany and Finland, under an Apache 2.0 license. Open weights plus an EU GPU cloud plus Kubernetes means you can run a competitive model where your data already lives, under your own jurisdiction, with no per-token bill.

This guide is the how. I will cover what Kolibri 1 actually is, which clouds genuinely count as sovereign, the exact VRAM maths, a copy-pasteable Kubernetes deployment with vLLM, how to keep the GPU bill sane, and where Qovery removes the platform work. Every number here links to a primary source, and every GPU price was checked on 19 October 2026.

What exactly is Kolibri 1, and can you legally self-host it?

Yes, you can self-host Kolibri 1: Aleph Alpha published the weights for download under Apache 2.0, so running inference on your own hardware is the intended use, not a workaround. The one thing to confirm before you ship is the license text on the official model card, because open-weight licenses range from fully permissive (Apache 2.0, MIT) to community licenses with revenue or usage caps. Kolibri 1 is on the permissive end.

Here is what Aleph Alpha actually shipped, confirmed on the model card and the launch announcement: a Mixture-of-Experts (MoE) model with 78.1B total parameters and 3.46B active per token, 50 layers, 384 routed experts with 6 selected per token, German and English only, and a context window up to 1,048,576 tokens (Aleph Alpha recommends staying at or below 262,144 tokens for serving efficiency). It was trained on 768 NVIDIA B200 GPUs across data centres in Germany and Finland. The weights are published in fp8 (float8_e4m3fn).

A word on what MoE changes here. A Mixture-of-Experts model only runs a fraction of its weights per token (3.46B of 78.1B), so it is cheap to compute, but you still have to hold all 78 GB of expert weights in GPU memory. Low compute cost, full memory cost. Keep that in mind for the sizing section.

"Open-weight" is not the same as "open-source." Aleph Alpha published the weights and the inference code; the training data and recipe are not fully released. That distinction matters for reproducibility, but not for sovereignty. Sovereignty is about where inference runs, which is your decision, not where training ran, which was Aleph Alpha's. An auditor asks where the prompts and completions go, and when you self-host, the answer is "our cluster, in our region."

Why a German-built model specifically? For EU buyers it stacks vendor residency on top of data residency, it is tuned for German and European-language quality, and EU public-sector procurement increasingly rewards European suppliers (more on EUCS below). Self-hosting on top of that gives you full control of inference data, no per-token billing, no rate limits, and a frozen model version you can certify against. The cost is that uptime, GPU capacity planning and patching are now yours.

The honest alternatives, so you can decide: Aleph Alpha's own hosted API, Mistral AI's La Plateforme in the EU, the IONOS AI Model Hub, or another EU-hosted inference provider. Self-hosting wins when control, data policy or volume justify the operational work, and loses when they do not. A rough break-even: one always-on two-H100 node on Scaleway costs roughly EUR 5.46/hour, about EUR 3,990/month (see the sizing and cost sections). Against a hosted EU API at a blended EUR 1 per million tokens, that node pays for itself at around 4 billion tokens a month. Below that, call the API. Above it, the GPUs win. Redo the sum with your own throughput before you commit.

What counts as a "European sovereign cloud" for running an LLM?

A European sovereign cloud is one where the operating company, its parent and its staff fall under EU law only. That excludes ordinary EU regions of US hyperscalers, because the US CLOUD Act compels US-headquartered providers to produce data in their "possession, custody or control" wherever it is stored. "Our region is in Frankfurt" answers the latency question, not the jurisdiction question.

The reach is not theoretical. The EDPB-EDPS Joint Response to the LIBE Committee concluded that service providers subject to EU law cannot lawfully base a transfer of personal data to the US on a foreign order absent an MLAT or treaty, which is precisely the conflict the CLOUD Act creates. And since Schrems II (CJEU Case C-311/18), as a data controller you owe a transfer impact assessment and supplementary measures for any transfer exposed to US access. Encryption at rest with provider-held keys does not close that gap, because the provider can be compelled to produce both the data and the keys.

Think in three tiers:

• EU-native providers: Scaleway, OVHcloud, IONOS Cloud, StackIT, Open Telekom Cloud. The operating company and its parent sit under EU law.
• Hyperscaler sovereign offerings: the AWS European Sovereign Cloud (an EU-operated entity, first region in Brandenburg, Germany, GA on 15 January 2026), Microsoft Sovereign Cloud, and Google's sovereign offering operated in Germany by T-Systems. These are legitimate answers for many teams, with their own trade-offs around which controls are customer-operated.
• Ordinary EU regions of US hyperscalers: a latency and data-residency feature, not a sovereignty guarantee.

Sovereignty is a spectrum, not a binary. The right tier depends on your regulator and threat model. A French public-sector body under SecNumCloud and a startup that simply wants its data in the EU are not solving the same problem. Procurement will ask about BSI C5 (Germany), SecNumCloud (France, ANSSI), Gaia-X labels, ISO 27001, and the status of the EU Cloud Services Scheme (EUCS), whose "High" sovereignty requirements remain politically contested in 2026. And sovereignty is useless if you cannot get a GPU in-region, so check capacity before you check the logo.

Here is the comparison that matters for GPU hosting.

Prices and certification scopes checked on 19 October 2026; GPU pricing moves fast, so confirm on each provider's own page before you budget.

When you self-host open weights, only inference residency is in scope. The model was trained elsewhere, and that is almost never what an auditor asks about. For a deeper provider-by-provider breakdown, see our sovereign cloud providers comparison.

What hardware do you actually need to serve Kolibri 1?

Run one calculation before you buy anything: parameters times bytes per parameter, plus 20-30% of VRAM reserved for the KV cache at real concurrency. At bf16 that is about 2 GB of VRAM per billion parameters; at fp8 about 1 GB; at 4-bit about 0.5 GB. Kolibri 1 ships at fp8 and its weights total roughly 78 GB, so the weights alone exceed a single 80 GB card once you add any KV cache. You serve it on two H100 80GB GPUs with tensor parallelism, or on one H200 141GB.

The KV cache is the memory vLLM uses to store the attention keys and values for every token in flight. Its size scales with layers, KV heads, head dimension, context length and the number of concurrent sequences. This is where Kolibri's million-token context gets expensive: long context plus high concurrency can consume tens of gigabytes on top of the weights, which is exactly why Aleph Alpha recommends capping served context at 262,144 tokens. vLLM's PagedAttention helps a lot here, cutting KV cache waste from the 60-80% seen in naive serving down to under 4%, but it does not make the cache free.

A note on fp8: native fp8 execution needs Hopper-class hardware (H100) or newer, per the vLLM quantization docs. An A100 is Ampere and has no native fp8 path, so for Kolibri's published format you want H100, H200 or B200, not A100. The model card also lists additional quantizations; a 4-bit variant (AWQ or GPTQ, see the AWQ paper) roughly halves the weight footprint to around 40 GB, which brings a single 48 GB L40S or 80 GB H100 into play, at some cost to output quality. Verify the exact file size of any quant on the model card's Files tab before you size a node.

Node sizing beyond the GPU: provision host RAM at least equal to the model size for fast loading, size local NVMe to hold the weight cache (cold-start time is mostly weight-load time), and remember the container image and weight download are both tens of gigabytes. The only two throughput numbers worth benchmarking before you commit capacity are time-to-first-token and sustained output tokens per second, measured on your real prompts.

VRAM figures derive from the published fp8 weight size (about 78 GB) plus KV-cache headroom; the 4-bit row is an estimate from quantization arithmetic, not a measured file size. Prices checked on 19 October 2026.

How do you deploy Kolibri 1 on Kubernetes step by step?

Six steps and four Kubernetes objects: a tainted GPU node pool, the NVIDIA GPU Operator, a Secret holding your Hugging Face token, and a vLLM Deployment plus Service and Ingress that expose an OpenAI-compatible API over TLS. Everything else is ordinary Kubernetes. The only GPU-specific parts are the device plugin, the /dev/shm size and the weight cache.

Step 1: create a GPU node pool and taint it. On Scaleway you add a GPU pool to a Kapsule cluster; on OVHcloud you add a GPU flavour pool to Managed Kubernetes. Taint it nvidia.com/gpu=present:NoSchedule so ordinary CPU workloads never land on a node that costs several euros an hour. Only pods that explicitly tolerate the taint get scheduled there.

Step 2: install the NVIDIA GPU Operator. A GPU is not schedulable by default; Kubernetes needs a device plugin to advertise nvidia.com/gpu as an extended resource. The NVIDIA GPU Operator installs the driver, container toolkit and device plugin via Helm. Confirm it worked with kubectl describe node <gpu-node> and look for nvidia.com/gpu: 2 under Capacity (the Kubernetes GPU scheduling docs explain the model). Scaleway's Kapsule installs the operator for you on GPU pools.

Step 3: get the weights onto the node. Store your Hugging Face token as a Kubernetes Secret and mount a PersistentVolume or object-storage cache at /root/.cache/huggingface. The trade-off: downloading at boot keeps the image small but makes the first pod slow (tens of GB over the wire); baking the weights into the image starts fast but gives you a 30GB-plus image. A persistent cache gives you most of the speed without the image bloat.

Step 4: the vLLM Deployment. A minimal, correct fragment for the two-GPU tensor-parallel case:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kolibri-vllm
spec:
  replicas: 1
  selector:
    matchLabels: { app: kolibri-vllm }
  template:
    metadata:
      labels: { app: kolibri-vllm }
    spec:
      tolerations:
        - key: nvidia.com/gpu
          operator: Equal
          value: present
          effect: NoSchedule
      containers:
        - name: vllm
          image: vllm/vllm-openai:latest
          args:
            - "--model=Aleph-Alpha/Kolibri-1"
            - "--tensor-parallel-size=2"
            - "--max-model-len=262144"
            - "--gpu-memory-utilization=0.92"
            - "--api-key=$(VLLM_API_KEY)"
          env:
            - name: HUGGING_FACE_HUB_TOKEN
              valueFrom: { secretKeyRef: { name: hf-token, key: token } }
            - name: VLLM_API_KEY
              valueFrom: { secretKeyRef: { name: vllm-api-key, key: key } }
          ports:
            - containerPort: 8000
          resources:
            limits: { nvidia.com/gpu: "2" }
          volumeMounts:
            - { name: cache, mountPath: /root/.cache/huggingface }
            - { name: shm, mountPath: /dev/shm }
          readinessProbe:
            httpGet: { path: /health, port: 8000 }
            initialDelaySeconds: 600
            periodSeconds: 15
      volumes:
        - name: cache
          persistentVolumeClaim: { claimName: hf-cache }
        - name: shm
          emptyDir: { medium: Memory, sizeLimit: 8Gi }

Two details carry their weight. The emptyDir with medium: Memory mounted at /dev/shm raises shared memory above the container default of 64 MB, which tensor parallelism needs for inter-GPU communication; without it, NCCL hangs on startup (vLLM deployment docs). And initialDelaySeconds: 600 keeps the readiness probe from killing the pod while weights are still loading, which can take several minutes. Kolibri 1 also needs the aleph-alpha-inference vLLM plugin installed in the image; follow the model card for the exact base image or install step.

Step 5: Service, Ingress, TLS, and auth. Put a Service and an Ingress with TLS in front of the Deployment, and gate the endpoint with vLLM's --api-key or an auth proxy. An unauthenticated inference endpoint on the public internet is a budget incident waiting to happen: anyone who finds it runs your GPUs on your bill.

Step 6: verify. One curl proves the whole chain:

curl https://kolibri.your-domain.eu/v1/chat/completions \
  -H "Authorization: Bearer $VLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Aleph-Alpha/Kolibri-1",
    "messages": [{"role": "user", "content": "Erklaere Kubernetes in einem Satz."}]
  }'

Because the API is OpenAI-compatible, any existing OpenAI SDK client points at it by changing base_url only. No client rewrite.

The five failure modes that eat an afternoon: a pod stuck Pending because the toleration is missing; CUDA out-of-memory at startup from --gpu-memory-utilization set too high; NCCL hangs from the default 64 MB /dev/shm; the readiness probe killing the pod mid-weight-load; and the image pull timing out on a 30GB layer. Each maps directly to one line above.

On runtime choice: vLLM for production throughput and continuous batching, Hugging Face TGI for HF-native tooling, SGLang for structured-output workloads, and Ollama for laptop-scale evaluation only. Kolibri 1 ships a vLLM plugin, so vLLM is the path of least resistance.

For a deeper runtime comparison, see our write-up on vLLM, TGI and NVIDIA NIM.

How do you keep GPU costs and cold starts under control in production?

GPU inference cost is dominated by idle time, not tokens: one H100 left running all month costs the same whether it serves ten requests or ten million. The two levers that matter are autoscaling on queue depth instead of CPU, and cutting the time a new pod needs to become ready, which is almost entirely weight-load time.

Horizontal Pod Autoscaling on CPU utilisation is meaningless for LLM serving. The GPU is saturated while the CPU sits near idle, so CPU-based HPA never triggers. Scale on vLLM's own metrics instead: vllm:num_requests_waiting and vllm:gpu_cache_usage_perc from the /metrics endpoint (vLLM metrics docs), consumed through KEDA or the Prometheus Adapter. When the queue grows, add a replica; when it drains, remove one.

Cold start has four stages, each with a fix. Node provisioning: keep a warm node or hold capacity with a low-priority pause pod. Image pull: pre-pull the image onto the node or shrink the base. Weight download: use a persistent cache or a baked image. CUDA graph capture: --enforce-eager trades some steady-state throughput for a faster start. On a cold 78 GB model, weight load dominates everything else, so the persistent cache is the highest-leverage fix.

Scale to zero for non-production. Dev, staging and preview inference endpoints should not hold a GPU overnight, and this is where most of the realised savings actually come from. For always-on production traffic, a one-year committed or reserved GPU reservation on an EU provider usually beats every software optimisation you can ship in a sprint. And batch what can be batched: offline embedding, classification and summarisation jobs run happily on spot or preemptible GPU nodes at a fraction of on-demand price.

The observability minimum is five numbers plus one: time-to-first-token, output tokens per second, GPU utilisation, KV cache utilisation and queue depth, plus cost per million tokens, the one the business will ask for. Here is how that last number moves with utilisation.

Assumptions: 2x H100 at EUR 2.73/GPU-hour (Scaleway) = EUR 5.46/hour. Throughput is an illustrative 2,500 sustained output tokens/second (9 million tokens/hour); substitute your own benchmark, because Kolibri's low active-parameter count can push this higher. At full utilisation that is EUR 0.61/1M; dividing by realised utilisation gives the 30% and 80% rows. API rows are list output prices, not blended, checked 19 October 2026. The self-hosted rows exclude engineering time; the API rows exclude the sovereignty gap. Compare like for like.

The pattern is clear: a self-hosted GPU left half-idle is not obviously cheaper than a hosted EU API, and both are far below a US frontier API. Self-hosting wins on volume and control, not on a casually-loaded GPU. Our GPU pricing comparison method shows how to run this calculation for your own traffic.

Where does Qovery fit, and what does it remove from this setup?

Qovery provisions and operates the Kubernetes layer inside your own Scaleway, AWS, GCP or Azure account, or runs on the cluster you already have on OVHcloud or IONOS, so the weights, GPU nodes and cloud bill never leave your legal entity while developers deploy the inference service with a git push instead of maintaining the YAML above. Qovery is not an inference engine: vLLM still serves the model.

This is the sovereignty argument made operational. With Qovery's bring-your-own-cloud model, the cluster lives in your account, under your jurisdiction, with your committed GPU reservations and your discounts. Nothing moves into a vendor's tenancy, which is exactly what a transfer impact assessment under Schrems II has to demonstrate. Compare that with a managed GPU platform where the GPUs and your data sit in the provider's account.

What Qovery handles so a two-person platform team does not have to: cluster provisioning and managed upgrades, per-environment RBAC, secrets, preview environments per pull request, and environment auto-stop for non-production workloads, which is the scale-to-zero habit from the cost section turned into a default. In practice your vLLM container becomes a Qovery application with a Dockerfile, a port, resource limits and env vars, and Qovery wires the ingress, TLS, secrets, autoscaling and rollbacks around it.

For teams already standardised on an EU provider Qovery does not provision directly, the bring-your-own-Kubernetes path applies: Qovery runs on your existing cluster on OVHcloud, IONOS, StackIT or on-prem. Multi-cloud and BYOK are first-class, not an afterthought.

The boundary, stated honestly: Qovery does not quantize your model, pick your GPU SKU or tune your vLLM flags. It removes the platform work around the model, which is the part teams consistently underestimate.

See the full playbook in our guide to migrating GPU inference to a European sovereign cloud.

How do I do this with Qovery from Claude Code or Cursor?

Install the Qovery Agent Skills once, then ask your coding agent in plain language to deploy the inference service into your own cloud account. The agent generates the configuration, shows you the plan, and waits for your approval before anything is created.

Install globally for every compatible agent (Claude Code, Cursor, Codex, OpenCode, Gemini CLI):

curl -fsSL https://skill.qovery.com/install.sh | bash

Project-scoped install instead of global:

curl -fsSL https://skill.qovery.com/install.sh | bash -s -- --project

Add the MCP Server to Claude Code, OAuth and read-only by default:

claude mcp add --transport http qovery https://mcp.qovery.com/mcp --callback-port 4242

Enable write access only when you explicitly want it, which also requires enabling write access in the Qovery Console:

claude mcp add --transport http qovery "https://mcp.qovery.com/mcp?read_write=true" --callback-port 4242

The split is simple: the skills deploy (qovery-onboard, qovery-deploy, qovery-troubleshoot, qovery-optimize, qovery-speedup, qovery-preview, qovery-terraform), and the MCP Server operates what already exists. A prompt that gets you most of the way there:

Full setup is in the AI agent quickstart and the Claude Code integration.

Is Kolibri 1 free to use commercially if I self-host it?

Kolibri 1 is published under the Apache 2.0 license on Hugging Face, which permits commercial use, modification and redistribution, including for self-hosting. Apache 2.0 has no per-seat or revenue restriction. Always read the license text and any use-policy notes on the model card yourself before you ship, since licenses can carry attribution or notice obligations.

Which European cloud providers offer GPUs for self-hosting an LLM on Kubernetes?

Scaleway (Kubernetes Kapsule), OVHcloud (Managed Kubernetes), IONOS Cloud, StackIT and Open Telekom Cloud all sell GPU capacity in EU regions with managed Kubernetes. Scaleway and OVHcloud publish on-demand hourly GPU pricing and integrate the NVIDIA GPU Operator with their managed Kubernetes. StackIT and Open Telekom Cloud lean toward enterprise contracts and German public-sector compliance such as BSI C5.

How much GPU memory do I need to run Kolibri 1?

Kolibri 1 ships at fp8 with roughly 78 GB of weights, so you need more than a single 80 GB GPU once the KV cache is included: two H100 80GB GPUs with tensor parallelism, or one H200 141GB. A 4-bit quantized variant roughly halves the weight footprint to around 40 GB, which can fit a single H100 or a 48 GB L40S, at some cost to output quality. Reserve 20-30% of VRAM on top of the weights for the KV cache at your real concurrency.

Does hosting in an AWS or Azure EU region make my deployment sovereign under the CLOUD Act?

No. An ordinary EU region of a US-headquartered hyperscaler does not remove CLOUD Act exposure, because the Act reaches data in the provider's possession, custody or control regardless of where the servers sit, as the EDPB-EDPS Joint Response sets out. For a stronger position you need an EU-operated provider or a hyperscaler's dedicated sovereign offering such as the AWS European Sovereign Cloud, and a transfer impact assessment either way.

Should I use vLLM, TGI, SGLang or Ollama to serve Kolibri 1 on Kubernetes?

Use vLLM for production: it has continuous batching, tensor parallelism, an OpenAI-compatible API, and Kolibri 1 ships a vLLM plugin. Hugging Face TGI is a strong alternative for HF-native stacks, and SGLang suits structured-output workloads. Ollama is for laptop-scale evaluation, not production Kubernetes serving.

Can I run Kolibri 1 on my existing Kubernetes cluster instead of provisioning a new one?

Yes. If you already run managed Kubernetes on OVHcloud, IONOS, StackIT or on-prem, add a GPU node pool, install the NVIDIA GPU Operator, and deploy the vLLM workload described above. Qovery's bring-your-own-Kubernetes path operates that layer on your existing cluster, so you keep your provider, region and jurisdiction.

Is self-hosting Kolibri 1 cheaper than calling a hosted EU inference API?

It depends almost entirely on utilisation. A rough break-even: one always-on two-H100 node costs about EUR 3,990/month, so against a hosted EU API at a blended EUR 1 per million tokens, self-hosting wins above roughly 4 billion tokens per month and loses below it. Measure your real monthly token volume and your GPU's sustained throughput, then redo the sum before committing.

Sovereignty no longer means a weaker model. With Kolibri 1's open weights, an EU GPU cloud and Kubernetes, you can run a competitive model where your data already lives, under your own jurisdiction. Qovery operates the Kubernetes layer so your team ships it with a git push.