NVIDIA NVKE for Self-Hosting AI Models: What It Gives You on Bare-Metal GPUs and What You Still Build
NVIDIA open-sourced NVKE on November 10, 2026: a preview, Apache 2.0 management plane that creates isolated Kubernetes clusters on bare-metal NVIDIA GPU servers. Here is what it really provides, the inference layer you still build on top (vLLM, NIM or llm-d, autoscaling, model cache, observability, upgrades), and how it compares with managed EKS and GKE GPU node pools.
NVKE is real, open source and early. NVIDIA released the NVIDIA Kubernetes Engine (NVKE) and NVIDIA Kubernetes Recipes (NVKR) on November 10, 2026, under Apache 2.0, as part of its DSX AI Factory Platform. Both are in public preview, and NVIDIA says it does "not recommend NVKE for production workloads" yet.
NVKE gives you clusters, not an inference platform. It provisions isolated workload clusters (each with its own control plane, etcd and node pools) on bare-metal GPU servers through NVIDIA Infra Controller (NICo), attests nodes with TPM and SPIFFE/SPIRE, and handles node pool scaling, Kubernetes upgrades, node replacement and certificate renewal.
The model-serving layer is still yours. Serving engines (vLLM, NVIDIA NIM, llm-d), an inference-aware gateway, token or queue-based autoscaling, a model weight cache, a metrics and logs backend, and the upgrade path of the management cluster itself are not in the box.
Managed EKS and GKE GPU pools are the safer default for most teams that want to self-host models today: the cloud provider runs the control plane, GPU drivers ship in the node images, and you pay about $0.10 per cluster-hour. NVKE makes sense when you already own GPU racks and want to sell or share clusters on them.
Qovery sits above either choice. A cluster you already run, including a conformant bare-metal one, connects to Qovery with qovery cluster install, so your team ships inference services with git push, Helm, RBAC and audit logs instead of hand-written manifests.
The question I get from teams with GPU racks has changed this month. It used to be "how do we turn these servers into Kubernetes without writing our own control plane?" Since KubeCon North America, it is "NVIDIA just open-sourced NVKE, is that the answer?"
Partly. I spent the week reading the NVKE repository, the NVIDIA documentation and the launch posts, and the honest summary is that NVKE solves the cluster factory problem and leaves the model-serving problem exactly where it was. That is still a big step. The CNCF Annual Cloud Native Survey found 82% of container users run Kubernetes in production, and 66% of organizations hosting generative AI models use it for some or all of their inference. A credible open-source way to stamp out GPU clusters on your own hardware fills a gap. Here is what you get, what you still build, and when a managed GPU node pool on EKS or GKE is the better call.
What exactly did NVIDIA open-source with NVKE?
NVIDIA open-sourced NVKE, a management plane that an AI factory or platform team installs on its own NVIDIA GPU infrastructure to create and operate Kubernetes clusters on demand, plus NVKR, which builds the node images and components those clusters run. Both were announced on November 10, 2026, both are Apache 2.0, and both are in public preview.
The details come straight from NVIDIA's own pages:
Announcement. The NVIDIA open source blog post lists NVKE and NVKR as "now available in public preview" alongside the OpenShell donation to the CNCF and a new component called Snapshot.
Code. The repository is github.com/dsx-ai-factory/nvke, licensed Apache 2.0. Its README states: "APIs, configuration and behavior can change between releases, and we do not recommend NVKE for production workloads."
Kubernetes version. The NVIDIA developer blog says NVKE is conformance tested on Kubernetes 1.36.
Conformance. The NVKE overview says the CNCF certified that NVKE creates conformant Kubernetes clusters and lists it as a Certified Kubernetes AI Platform.
Ecosystem. NVIDIA says Mirantis, Netris, OpenNebula Systems, Rafay, Spectro Cloud and vCluster, together with Firmus and SK Telecom, "have aligned with NVIDIA on integration plans" (NVIDIA blog).
One correction to how the project is often described. NVKE does not install itself on any pile of bare-metal servers. At preview, the NVKE docs state that NVIDIA Infra Controller (NICo) is the only supported infrastructure provider. NICo is a separate Apache 2.0 project for bare-metal lifecycle management (hardware discovery, firmware validation, DPU provisioning, network isolation and tenant sanitization). So the real stack is NICo for the machines, NVKE for the clusters, NVKR for the artifacts.
NVKE also describes itself narrowly. The repository says it is "not a Kubernetes distribution and not a Cluster API flavour" but "a self-hosting factory for Kubernetes clusters" (aiweekly summary of the README).
How does NVKE work on bare-metal GPU servers?
NVKE uses three tiers of clusters: a temporary seed cluster bootstraps a management cluster, and the management cluster then creates and lifecycle-manages many isolated workload clusters on NICo-provisioned GPU machines. Your users only ever see the workload clusters, through plain kubectl.
Seed cluster. A throwaway bootstrap cluster on one Linux host or VM (6+ CPUs, 12 GiB free memory, 100 GiB disk, Docker and a virtual TPM).
Management cluster. Runs the NVKE REST API and controllers. NVIDIA asks for three CPU servers for it.
Workload clusters. One per tenant or team, "each with its own Kubernetes control plane, etcd, and node pools." You create, describe, update and delete them through the NVKE API or nvkectl.
Each workload cluster ships with OVN-Kubernetes networking (an NVIDIA-curated build), NodeLocal DNSCache, a SPIRE agent and an OpenTelemetry-based telemetry pipeline. A node "must present an attested SPIFFE identity before it receives join credentials," which is how NVKE keeps one tenant's machine from joining another tenant's cluster.
What the operator brings is spelled out too: "An HTTP server for the node image, a container registry for the controller image, and internet access to download the open source images and Helm charts used during installation," plus a NICo installation with enrolled capacity. NVKR builds those node images and components "from version-pinned sources and records every input," so you can say exactly which software runs on each node.
If you want to kick the tyres before touching hardware, NVIDIA offers a local quickstart that runs on a single machine without GPUs.
What does NVKE give you, and what do you still build to self-host AI models?
NVKE gives you isolated, attested, upgradeable Kubernetes clusters on your own GPU hardware. To self-host AI models on them, you still build the GPU runtime inside each cluster, the serving engines, request routing, autoscaling, a model weight cache, the observability backend, and the operating procedures for the management plane itself.
The NVKE overview is unusually clear about the split. NVKE and NVKR own artifact building, signing, node provisioning, cluster formation, security and upgrades. The operator owns infrastructure, the management cluster, artifact publishing and cluster availability. The customer owns "workloads, images, namespaces, secrets, RBAC." NVKE explicitly does not manage bare metal, does not ship OS images or Kubernetes binaries (you build and host them with NVKR), and does not run customer workloads.
NVKE workload clusters come up as conformant Kubernetes with networking and telemetry. The GPU-accelerated runtime comes from a separate project, NVIDIA AI Cluster Runtime, which publishes "validated, version-locked combinations of the GPU Operator, Network Operator, DRA and other runtime components" as recipes (NVKE docs). The developer blog says teams add AICR, node packages with NodeWright and break-fix automation with NVSentinel "inside each workload cluster."
I like this split, because the GPU stack is pinned and reproducible instead of whatever helm install gpu-operator resolved to on the day you built the cluster. The cost is that somebody on your team owns choosing the recipe, rolling it to every tenant cluster and handling the day a driver breaks a model. We wrote a runbook for exactly that kind of NIM and driver upgrade; none of it goes away with NVKE.
The preview gaps are operational, and they are real
Read the limitations section of the NVKE README before you plan anything:
"API authentication is off and the API uses plain HTTP after installation." The docs have a page on securing the API; until you follow it, anyone who can reach the endpoint can create clusters.
"The management cluster does not upgrade, scale or replace itself."
There is no backup and restore procedure yet.
Full-disk-encrypted node images provide "no data-at-rest confidentiality" in this release.
That is normal for a preview. In practice, the most important cluster in the whole design, the one that creates and upgrades every other cluster, is today a pet you maintain by hand.
Observability stops at the collector
NVKE ships an OpenTelemetry-based Helm chart that "collects logs, metrics, and traces and exports them over OTLP to a monitoring backend" (NVIDIA developer blog). The backend is yours. For inference you also need the serving-level signals: time to first token, tokens per second, KV cache usage and queue depth. vLLM exposes them on a Prometheus endpoint; somebody still has to scrape, store, chart and alert on them per tenant.
What does the model-serving stack on top of NVKE look like?
On an NVKE workload cluster, a working self-hosted model stack is the same one you would build on any Kubernetes cluster: GPU runtime, a serving engine, a router, an autoscaler, a weight cache and dashboards. NVKE hands you a clean, isolated cluster, and the list stays the same length.
Here is the minimum I would put in front of a team that wants to serve open-weight models to internal apps:
GPU runtime. Apply an AICR recipe so drivers, the GPU Operator and the NVIDIA DRA driver match your hardware.
Multi-node or disaggregated serving, only if the model needs it. llm-d or NVIDIA Dynamo split prefill and decode across GPUs. They pay off on large mixture-of-experts models and long contexts, and add real operational weight. Our llm-d 1.0 article covers which path a small team should start with.
Routing. The Gateway API Inference Extension (v1.0.0 is on GitHub) routes on model name and replica load instead of round-robin.
Autoscaling. CPU-based HPA is the wrong signal for GPUs. Scale on queue depth or pending requests with KEDA, and set a floor of one warm replica for anything user-facing.
Model cache. Pulling a 100+ GB checkpoint on every pod start makes autoscaling useless. Cache weights on local NVMe or a shared volume, and look at checkpoint restore.
Dashboards and alerts. GPU utilization from DCGM, plus vLLM or NIM metrics, per model and per tenant.
Startup time is where the stack decides whether autoscaling works at all. NVIDIA announced Snapshot in the same release, an open-source component that restores checkpointed replicas "without model reload or warmup." Its design partners reported large drops: Crusoe cut a GLM-5.3 workload's initialization from "roughly eight minutes to 31 seconds," and Saturn Cloud cut vLLM startup from "between two and 8.5 minutes to six and 34 seconds" (NVIDIA blog). Those are vendor-reported numbers from partners, so measure on your own models, but the direction matches what I see: weight loading, not scheduling, dominates cold starts.
Clusters are step one. Shipping models on them is the rest.
Connect the Kubernetes cluster you already run, on bare metal or in your own AWS, GCP, Azure or Scaleway account, and deploy inference services with git push, Helm, RBAC and audit logs. The Free plan includes one cluster of your own.
How does NVKE compare with managed EKS or GKE GPU node pools?
For most teams that want to self-host models in 2026, a managed GPU node pool on EKS or GKE is less work than NVKE, because the cloud provider runs the control plane, ships GPU-ready node images and handles hardware failures. NVKE wins when you already own the GPU servers, need many isolated tenant clusters on them, and have people to run NICo and the management plane.
Criterion
NVKE on your bare metal (preview)
Amazon EKS GPU node pools
Google GKE GPU node pools
Who runs the control plane
You (management cluster) and NVKE (workload clusters)
AWS
Google
Production readiness
Preview, "not recommended for production"
GA
GA
Control plane fee
None, you pay for 3 CPU servers and your time
$0.10 per cluster-hour on standard support (EKS pricing)
Auto Mode and Karpenter 1.15+, since November 9, 2026 (AWS)
Supported on recent GKE versions
Capacity
The servers you bought
On-demand, Spot, Capacity Blocks
On-demand, Spot, DWS
Tenant isolation
Separate control plane and etcd per cluster, TPM-attested nodes
One cluster per tenant, or namespaces
One cluster per tenant, or namespaces
Hardware failure handling
NICo plus NVSentinel, your runbooks
AWS replaces instances
Google replaces instances
Upgrade of the thing that upgrades clusters
No procedure yet
AWS
Google
Serving stack (vLLM, NIM, llm-d)
You build it
You build it
You build it
Best for
AI factories, neoclouds, enterprises with owned GPU racks
Teams on AWS who rent GPUs
Teams on GCP who rent GPUs
Look at the serving stack row. Whether the cluster comes from NVKE, EKS or GKE, you still deploy and operate the inference layer. The cluster choice changes who pays for hardware and the control plane, and the serving work stays with you either way.
The control plane fee is a weak reason to choose. $0.10 per hour is about $73 a month per cluster. On a single 8-GPU node the GPUs dominate the bill by two orders of magnitude, so pick NVKE for ownership, data location or tenancy reasons, not to save the EKS fee. If you are weighing rented versus owned capacity in the first place, our piece on the resale chain between you and the GPU covers the risk side.
When does NVKE make sense instead of building a GPU control plane from scratch?
NVKE makes sense when you own a fleet of NVIDIA GPU servers, need to hand out isolated Kubernetes clusters to several teams or customers, and would otherwise write your own cluster-provisioning layer on top of Cluster API, kubeadm or a vendor distribution. If you would only run one or two clusters, NVKE is more machinery than you need.
Pick NVKE (and start evaluating it now, in a lab) if:
You are an AI factory, neocloud or GPU reseller that wants to offer managed Kubernetes per customer. That is NVKE's stated audience.
You run a shared GPU estate inside a large company and want hard isolation per business unit, with a separate control plane and etcd each, instead of namespace-level multi-tenancy.
Your hardware already runs, or will run, NICo. Without NICo, NVKE has no infrastructure provider at preview.
You can staff the management plane. Someone owns upgrading, scaling and backing up the management cluster by hand until NVIDIA ships procedures for it.
Skip NVKE for now if:
You need production in the next quarter. NVIDIA tells you not to run production on it yet.
You only need one or two GPU clusters. A single conformant cluster built with a standard distribution, or a managed cloud cluster, is less to operate.
You rent GPUs by the hour. EKS, GKE, AKS or a neocloud's managed Kubernetes already gives you the cluster layer.
Your bottleneck is getting models served and shipped, not getting clusters created. NVKE does nothing for that bottleneck.
Compared with building your own control plane from scratch, NVKE is a clear win once it matures: signed, reproducible node images through NVKR, attested node identity, per-tenant control planes and an API for cluster lifecycle are each months of work. I would rather adopt that and contribute fixes than write it again.
What does a realistic plan look like for a team evaluating NVKE today?
Run NVKE in a lab now, keep production inference on a cluster type you can already operate, and design your serving layer so it does not care which one it lands on. That way the day NVKE leaves preview, moving a workload is a cluster change, not a rewrite.
A plan I would sign off on:
Weeks 1-2: local quickstart. Run the single-machine NVKE quickstart without GPUs. Learn the API, nvkectl, the seed-to-management handoff and the failure modes.
Weeks 3-6: one NICo rack. Bring up a management cluster on three CPU servers, enable API authentication immediately, and create two workload clusters on real GPU nodes. Apply an AICR recipe.
In parallel: a portable serving layer. Package your serving engine (vLLM or NIM) as a Helm chart with values per environment, expose vLLM metrics, put a weight cache in front, and deploy it to whatever cluster you run today.
Exit criteria for production. NVIDIA lifts the preview warning, a documented management-plane upgrade and backup procedure exists, and you have rehearsed losing the management cluster.
Teams that skip step 3 end up rebuilding their deployment pipeline when they change cluster type. Your developers should deploy a model the same way whether the cluster came from NVKE, EKS, GKE or AKS.
Where does Qovery fit with NVKE and managed GPU clusters?
Qovery is an Agentic DevOps platform that deploys and operates applications on Kubernetes in your own cloud account or on a cluster you already run; it is the delivery layer above whatever creates your clusters, and it never hosts your workloads. With NVKE, the cluster comes from NVKE and Qovery handles how teams ship and operate inference services on it.
Concretely:
Bring an NVKE workload cluster. A conformant cluster you already run connects with qovery cluster install (the BYOK path, Kubernetes 1.31+). The newer Helm-installed Qovery Operator, in Preview, targets EKS, GKE and Scaleway Kapsule; other clusters, including on-premises and bare metal, use qovery cluster install. Bare-metal clusters usually need you to provide load balancing and a default StorageClass, which managed clouds include.
Or let Qovery run the cluster in your cloud account. On Qovery-managed clusters in your AWS, GCP, Azure or Scaleway account, you attach GPU node pools per service, with Karpenter on AWS. Our EKS GPU walkthrough shows the flow.
Ship the serving layer the same way everywhere. Deploy vLLM or NIM as a container or as a Helm service, with environment variables and secrets, from git push. Clone an environment to test a new model version, or open a preview environment per pull request for the app that calls the model.
Keep control. RBAC per environment, audit logs on every action, and OPA-based API policy tokens when an AI agent deploys on your behalf.
Qovery does not provision bare metal or replace NICo and the NVKE management plane. On a cluster you bring, the GPU runtime (AICR or the GPU Operator) stays yours. The Qovery Free plan covers 3 users, unlimited AI agents and one cluster of your own, which is enough to connect a lab NVKE cluster and see whether the delivery layer fits.
How do I do this with Qovery from Claude Code or Cursor?
Install the Qovery Agent Skills and the Qovery MCP Server in your coding agent, then ask it in plain language to deploy your serving chart to the cluster you connected. These are the only commands you need:
BASH|1. Agent Skills for every compatible agent (Claude Code, Cursor, Codex, OpenCode, Gemini CLI)
curl -fsSL https://skill.qovery.com/install.sh | bash
# 2. Or scope the skills to the current project only
curl -fsSL https://skill.qovery.com/install.sh | bash -s -- --project
# 3. MCP Server in Claude Code over OAuth, read-only by default
claude mcp add --transport http qovery https://mcp.qovery.com/mcp --callback-port 4242
# 4. Write access, only when you explicitly want it (also enable write access in the Qovery Console)
claude mcp add --transport http qovery "https://mcp.qovery.com/mcp?read_write=true" --callback-port 4242
Step 1 asks you to approve a shell install. Step 3 opens an OAuth login in your browser, and the MCP Server then acts with your own Qovery permissions. Step 4 is optional and needs a second switch in the Console, so an agent cannot quietly grant itself write access.
Then ask, for example: "deploy this vLLM Helm chart to the GPU cluster connected to Qovery, staging environment" or "why is the inference service in production restarting?". The skills (qovery-onboard, qovery-deploy, qovery-troubleshoot and others) deploy; the MCP Server operates what already exists. Start from the AI agent quickstart or the Claude Code integration page. For more on GPUs, schedulers and serving engines, the Kubernetes for AI hub collects our guides.
Frequently asked questions
What is NVIDIA NVKE?
NVKE, the NVIDIA Kubernetes Engine, is an open-source management plane that a platform team installs on its own NVIDIA GPU infrastructure to create and operate isolated Kubernetes clusters on demand. Each workload cluster gets its own control plane, etcd and node pools. NVIDIA released it with its companion build system NVKR on November 10, 2026, under the Apache 2.0 license, in public preview.
Is NVKE ready for production?
No. NVIDIA's README says it does "not recommend NVKE for production workloads" during the preview. The API has authentication off and uses plain HTTP after installation until you secure it, the management cluster does not upgrade, scale or replace itself, and there is no backup and restore procedure yet.
What does it take to self-host AI models on NVKE instead of building a GPU control plane from scratch?
NVKE removes the cluster-provisioning work: isolated clusters, attested nodes, Kubernetes upgrades and node replacement through an API. You still need NICo for the bare metal, three CPU servers for the management cluster, an AICR recipe for GPU drivers and operators, a serving engine such as vLLM, NVIDIA NIM or llm-d, an inference-aware router, queue-based autoscaling, a model weight cache and an observability backend. Compared with writing your own control plane, NVKE saves months; compared with a managed EKS or GKE GPU pool, it adds the management plane to your on-call list.
Does NVKE work on any bare-metal GPU server?
Not at preview. NVKE's only supported infrastructure provider is NVIDIA Infra Controller (NICo), which handles hardware discovery, firmware, DPU provisioning and tenant isolation for the physical machines. Nodes also need TPM-based attestation through SPIFFE/SPIRE before they can join a cluster.
Should I use NVKE or a managed EKS or GKE GPU node pool?
Use a managed EKS or GKE GPU node pool if you rent GPUs and want production now: the provider runs the control plane for about $0.10 per cluster-hour and ships GPU-ready node images. Consider NVKE if you own GPU racks, run NICo, and need many isolated tenant clusters on them. In both cases you still build and operate the model-serving layer yourself.
Can I deploy to an NVKE cluster with Qovery?
Yes. An NVKE workload cluster is conformant Kubernetes, and a cluster you already run connects to Qovery with qovery cluster install. Your team then deploys inference services and the apps that call them with git push or Helm, with RBAC and audit logs, while the GPU runtime on that cluster stays under your control.
Guillaume is a Staff Solutions Architect at Qovery. He helps customers troubleshoot production infrastructure and writes about Kubernetes operations, AI agents, and reliable cloud platforms.
Next step
Clusters are step one. Shipping models on them is the rest.
Connect the Kubernetes cluster you already run, on bare metal or in your own AWS, GCP, Azure or Scaleway account, and deploy inference services with git push, Helm, RBAC and audit logs. The Free plan includes one cluster of your own.