Air-Gapped AI Without Google's Appliance: Running Gemma, Mistral and NVIDIA GPUs on Your Own Kubernetes

Key points:

• You cannot self-host Gemini. Google does not distribute Gemini weights under any license, so the only way to run Gemini inside your own walls is Google Distributed Cloud (GDC) air-gapped, on Google-supplied hardware. Every other on-prem path means open-weight models.
• Open-weight models cover most air-gapped workloads. Gemma 3 27B runs on a single 80GB GPU, Mistral Small 3.x and Gemma 4 ship under Apache 2.0, and Llama, Qwen and DeepSeek all run on standard Kubernetes with NVIDIA GPUs and no call-home.
• Reproducing GDC air-gapped on your own cluster takes five layers: NVIDIA GPU Operator for drivers and device plugins, an offline OCI registry plus a model-weight store, an inference runtime (vLLM, NVIDIA NIM, SGLang or Ollama), an OpenAI-compatible gateway, and a developer platform for self-service.
• The supply chain is the hard part, and inference is the easy part. Every container image, Helm chart, GPU driver, CVE patch and 10-150GB weight file must be mirrored inside the boundary, pinned by digest, and re-mirrored on every upgrade.
• Buy the appliance if you need Gemini specifically, vendor accreditation, and you are starting from zero hardware. Build on your own cluster if you already run Kubernetes on-prem or in a sovereign cloud and open models are good enough.
• Qovery runs on your own Kubernetes cluster, including on-prem and sovereign clusters, as well as in AWS, GCP, Azure and Scaleway accounts. Product teams get git-push deploys, preview environments, per-environment RBAC and auto-stop for idle GPU-backed environments.

Google's GDC air-gapped appliance is a legitimate product for a narrow buyer, and most teams reading the news are not that buyer. On October 20, 2026, Google made Gemini 3.5 Flash and NVIDIA HGX B300 systems generally available on GDC air-gapped, and added open models like Gemma and Mistral (Google Cloud Blog). If you already operate Kubernetes in your own data center or a sovereign cloud, you do not need to buy a rack from Google to get private inference. You need five layers of software, a disciplined artifact pipeline, and an honest answer on whether you really need Gemini.

What exactly did Google add to Distributed Cloud air-gapped, and what does it not give you?

Google Distributed Cloud air-gapped is a Google-supplied hardware-plus-software appliance that runs fully disconnected in your data center on a GKE-based control plane, and it now bundles Gemini 3.5 Flash, open models including Gemma and Mistral, and NVIDIA Blackwell Ultra GPUs. What it does not give you is choice: Google's hardware, Google's release cadence, Google's procurement cycle, and no path to reshape the stack.

Google describes GDC air-gapped as infrastructure that "does not require connectivity to Google Cloud or the public internet" and is "built to remain disconnected in perpetuity" (Google Cloud). The hardware "comes fully integrated into racks", and the platform is built on Kubernetes and GKE Enterprise (GDC docs).

The October 2026 announcement added three things (Google Cloud Blog):

• Gemini 3.5 Flash, generally available on GDC air-gapped, "entirely within your air-gapped boundary".
• Open models "like Gemma and Mistral". Google does not name the specific versions or model sizes in the post.
• NVIDIA HGX B300 systems on the Blackwell Ultra architecture, generally available. Blackwell Ultra carries 288GB of HBM3e per GPU, 3.6x the on-package memory of an H100 (NVIDIA Technical Blog).

This builds on the general availability of Gemini on GDC air-gapped in August 2025, which then covered Gemini 2.5 Flash and Pro (Google Cloud Blog).

The compliance posture is the real product. GDC and the GDC air-gapped appliance hold DoD Impact Level 6 authorization, on top of existing IL5 and Top Secret accreditations (Google Cloud Blog). Google also says the product "meets technical requirements" of ISO 27001/27017, SOC 2, ISMAP and NATO D48, and is aligned with NIST SP 800-53 FedRAMP High controls (Google Cloud). Read that wording carefully: Google claims alignment with the controls, and lists no FedRAMP High authorization.

What you trade away:

• Hardware lock-in. You run Google-supplied racks and Google's GPU SKUs.
• Procurement lead time. An integrated rack goes through a hardware sales cycle, not a Helm install.
• Release cadence. Google decides when new models, patches and features land inside your boundary.
• Pricing opacity. Google publishes no list price for GDC air-gapped.
• Catalog limits. Google decides which models appear in the catalog, and you have little room to bring arbitrary fine-tuned checkpoints of your own.

Who should buy it: defense, intelligence and regulated buyers who need Gemini specifically, want a vendor-accredited stack, and have no on-prem Kubernetes practice.

Who should not: teams already running Kubernetes on-prem or in a sovereign cloud whose real requirement is private inference on open models.

The same category includes AWS Outposts, Azure Local with Azure Arc, Red Hat OpenShift AI on disconnected clusters, and SUSE Rancher with SUSE AI. Only GDC ships Gemini.

Can you self-host Gemini, or do you have to use open models like Gemma and Mistral?

You cannot self-host Gemini. Google publishes no Gemini weights under any license, so the only two ways to use Gemini are the Google Cloud API or the GDC appliance. If you want models running on hardware you control, the realistic set is open-weight models: Gemma, Mistral Small, Llama, Qwen and DeepSeek.

Here are the distribution terms for each model family:

• Gemini: API or GDC appliance only. No weights.
• Gemma 3: open weights under the Gemma Terms of Use. Gemma 4: Apache 2.0 (Gemma 4 model card).
• Mistral Small 3.x: Apache 2.0, 24B parameters (Mistral AI).
• Llama 3.3: Meta community license with conditions.
• Qwen3: Apache 2.0 (Hugging Face). DeepSeek-R1: MIT, weights included (Hugging Face).

The quality gap is real on hard reasoning. Gemini 3.1 Pro scores 94.3% on GPQA Diamond (Google DeepMind). Mistral Small 3.2 scores 46.13%, and Mistral measured Gemma 3 27B at 42.40% on the same benchmark (Mistral Small 3.2 card, Mistral Small 3.1 card).

For most air-gapped workloads, that gap matters far less than teams assume before they measure it. RAG over internal documents, classification, summarization, extraction and code assist all run well on 7B-30B models. Run your own evaluation set before you decide you need a frontier model.

Sizing math is simple: weights take roughly parameters x 2 bytes at FP16/BF16, x 1 byte at FP8 and x 0.5 bytes at INT4, plus KV cache that grows with context length and concurrent users.

• An H100 has 80GB (NVIDIA). An H200 has 141GB of HBM3e (NVIDIA). A Blackwell Ultra GPU has 288GB.
• Gemma 3 27B ships as about 54.9GB of weights (Hugging Face), so it fits on one H100 with room for KV cache. Google calls it "the most capable model you can run on a single GPU or TPU" (Google).
• Llama 3.3 70B has 70.5B BF16 parameters, about 141GB of weights (Hugging Face), so you need two H100s at FP16 or one H200 with FP8.
• Mistral says Small 3.1 "can run on a single RTX 4090 or a Mac with 32GB RAM" when quantized (Mistral AI).

Licensing needs a compliance read before deployment. The Gemma Terms of Use require you to flow the use restrictions down into downstream agreements, and Google "reserves the right to restrict (remotely or otherwise) usage" it believes violates the terms (Gemma Terms, Prohibited Use Policy). Above 700 million monthly active users, the Llama 3.3 license requires a separate license from Meta, and the license incorporates an acceptable use policy (Meta). Apache 2.0 and MIT models skip that review.

On your own cluster you can also fine-tune and serve your own checkpoints, which a curated appliance catalog does not offer.

INT4 and FP8 figures are my weight-only estimates from parameter counts, except where linked.

What is the full stack needed to run disconnected AI on your own Kubernetes cluster?

Five layers on top of a Kubernetes cluster you already operate reproduce what GDC air-gapped bundles: GPU enablement, an offline artifact supply chain, an inference runtime, an OpenAI-compatible access layer, and a developer platform. Each layer has a mature open-source option and a commercially supported one.

You are probably not starting from zero. 82% of container users ran Kubernetes in production in 2025 (CNCF Annual Survey), and in the 2024 survey on-prem data centers matched public clouds at 59% each (CNCF 2024). The 2025 survey found 23% fully running AI inference on Kubernetes and another 43% partially.

• Layer 1, GPU enablement. The NVIDIA GPU Operator installs the driver, container toolkit, device plugin, GPU feature discovery and DCGM exporter. Share GPUs with MIG, which partitions Ampere-and-later GPUs into "up to seven separate GPU Instances" (NVIDIA MIG guide), or with time-slicing. The air-gapped install requires "all images to be hosted in a local image registry" and a local mirror of your OS package repository, unless you use precompiled driver containers (NVIDIA).
• Layer 2, offline artifact supply chain. Run a local OCI registry (Harbor or Zot), mirror Helm charts and OS and driver packages, and store model weights in internal object storage (MinIO) or as OCI artifacts.
• Layer 3, inference runtime. vLLM is the default for good reason: continuous batching and PagedAttention delivered "up to 24x higher throughput" than Hugging Face Transformers in its launch benchmarks, cutting KV cache waste from 60-80% to under 4% (vLLM blog, PagedAttention paper). NVIDIA NIM gives vendor-supported containers, SGLang is a strong alternative, and Ollama or llama.cpp suit small models. KServe or Ray Serve act as the serving control plane.
• Layer 4, access layer. vLLM exposes an OpenAI-compatible endpoint natively. Put LiteLLM or an internal gateway in front for routing, API keys, per-team quotas and audit logs, so application code stays portable between a hosted API and your cluster.
• Layer 5, developer platform. Environments, secrets, RBAC, namespaces and deployments as self-service instead of tickets. Teams forget to budget for this layer.

Observability cuts across all five. Push DCGM metrics into Prometheus and Grafana from day one, because idle GPU time is the largest cost leak in self-hosted AI.

In my experience, a team that already runs Kubernetes reaches a first token on layers 1-4 in a few weeks. Layer 5 is the multi-month part.

What actually breaks first in an air-gapped Kubernetes cluster?

The artifact supply chain and the upgrade path break first, long before inference does. Every image, chart, driver, CVE patch and multi-gigabyte weight file has to cross the boundary deliberately, and cross again on every upgrade.

• Image mirroring discipline. No implicit Docker Hub or ghcr.io pulls, no latest tags, digests pinned everywhere, and a documented promotion process across the boundary with an approval trail.
• Version coupling. GPU driver, container toolkit, CUDA and runtime versions must align. The current GPU Operator supports Kubernetes 1.33 through 1.37 (NVIDIA platform support), so a cluster upgrade and an operator upgrade must move together.
• Model weight logistics. Llama 3.3 70B's full Hugging Face repo is about 282GB once you include the original checkpoint files (Hugging Face). Checksum every transfer, plan storage for several versions, and version weights with the same rigor as code.
• The unglamorous killers. Expired certificates with no ACME, NTP drift, internal DNS, and registries that quietly run out of disk.
• Offline security. Trivy supports fully air-gapped scanning with self-hosted vulnerability databases (Trivy docs). Add cosign verification with local keys and SBOM generation, and triage CVEs without internet access.

Recurring upgrades are the cost everyone underestimates. Kubernetes ships about three minor releases a year, and each gets roughly 14 months of patch support (Kubernetes). Kubernetes 1.34 reached end of life on October 27, 2026 (Kubernetes). That cadence is the main reason teams end up buying either an appliance or a managed platform layer.

Pre-flight checklist:

• Registry mirrored, every image pinned by digest
• Driver, CUDA, GPU Operator and Kubernetes versions pinned against NVIDIA's matrix
• Weights checksummed and versioned in internal storage
• Internal CA, cert rotation and NTP verified
• Upgrade rehearsed end-to-end in a connected staging cluster

What does self-hosting cost compared to buying Google Distributed Cloud air-gapped?

Self-hosting swaps an appliance contract for hardware capex plus platform engineering headcount, and the deciding variables are headcount and GPU utilization, not the GPU sticker price. A GPU node sitting at 15% utilization makes every other line item irrelevant.

DIY cost components:

• GPU servers, power and data center space
• 2-4 platform engineers plus 24/7 on-call
• Recurring upgrade cycles and a model evaluation effort

GDC cost components:

• Bundled hardware plus software subscription, with no public list price
• Months of procurement lead time
• Vendor support, with the accreditations listed above already in place

Reference prices anchor the math. An AWS p5.48xlarge with 8 H100s lists at $55.04 per hour on demand in us-east-1, about $6.88 per GPU-hour (AWS pricing), after AWS cut P5 prices by up to 45% in June 2025 (AWS). On headcount, the 2025 Stack Overflow survey puts the US median for cloud infrastructure engineers at about $189,000 (Stack Overflow). Three engineers cost more per year than many GPU nodes.

GPU utilization is the dominant variable. In ClearML's 2024 survey, only 7% of companies said their GPU infrastructure exceeds 85% utilization at peak, and 15% said less than half their purchased GPUs were in use (ClearML). MIG, time-slicing and auto-stopping idle non-production environments are the cheapest levers you have.

The break-even heuristic: if you already run Kubernetes on-prem with a platform team and open models meet your bar, build it. If you start from zero hardware and need Gemini specifically with accreditation, buy the appliance.

Sovereignty and regulation, not cost, usually drive the decision. The teams I talk to choose an air-gapped stack because a regulator, a defense contract or a data residency rule requires it, then optimize cost inside that constraint.

The hidden DIY line item is the self-service layer for application teams. After month three, that is where the platform team's time actually goes.

Where does an internal developer platform fit in a disconnected AI stack?

Once GPUs and model servers run, the bottleneck moves up the stack: application teams need environments, secrets, RBAC and deployments against those endpoints. An internal developer platform owns that layer, above the inference runtime, and hand-writing manifests per team is how platform teams get buried.

The split that works: the platform team owns the cluster, the GPU layer and the inference endpoints. Product teams own the RAG services, agents and internal apps that call the OpenAI-compatible endpoint.

This is where Qovery fits. On a Kubernetes cluster you already run, on-prem and sovereign clusters included, or on clusters Qovery provisions in your AWS, GCP, Azure or Scaleway account, Qovery adds:

• Git-push deployments for the services that call your models
• Preview environments per pull request
• Per-environment RBAC so a product team cannot touch the inference namespace
• Environment auto-stop for non-production
• Managed cluster upgrades on the clusters Qovery provisions in your cloud account, and managed databases where a cloud account is in play

Auto-stop matters most here. A non-production environment holding GPU-backed pods is the most expensive idle resource in the building, and nobody remembers to shut it down on Friday.

Honest scoping: Qovery does not sell a turnkey air-gapped SKU. Fully disconnected deployments should be scoped with our team before you plan around them.

Building the same layer from Backstage, Argo CD and Crossplane is achievable. It also becomes a product your team maintains forever.

Red Hat OpenShift AI and NVIDIA AI Enterprise are genuinely strong at model serving and MLOps, and vLLM remains the right default runtime. Qovery covers the application self-service layer and cluster lifecycle above them. It does not replace the serving runtime.

How do I set this up with Qovery from Claude Code or Cursor?

If you already operate a Kubernetes cluster, you can hand the wiring to a coding agent. Install the Qovery Agent Skills once, then ask your agent in plain language to deploy a service onto your own cluster, approving each action as it goes.

Install the skills globally for any compatible agent (Claude Code, Cursor, Codex, OpenCode, Gemini CLI):

curl -fsSL https://skill.qovery.com/install.sh | bash

This adds qovery-onboard, qovery-deploy, qovery-troubleshoot, qovery-optimize, qovery-speedup, qovery-preview and qovery-terraform. To scope the install to a single repository instead:

curl -fsSL https://skill.qovery.com/install.sh | bash -s -- --project

Add the Qovery MCP Server to Claude Code so the agent can inspect what already runs (OAuth, read-only by default):

claude mcp add --transport http qovery https://mcp.qovery.com/mcp --callback-port 4242

Enable write access only when you want the agent to change things. This also requires enabling write access in the Qovery Console:

claude mcp add --transport http qovery "https://mcp.qovery.com/mcp?read_write=true" --callback-port 4242

Then ask, for example, "deploy this inference gateway to my existing Kubernetes cluster with Qovery". The agent proposes a plan and asks for approval on each step. The skills deploy, and the MCP Server operates what already exists. See the AI agent quickstart and the Claude Code integration.