11 Platforms for Provisioning GPU Clusters Automatically (and How to Pick One in 2026)

Most lists of "platforms for provisioning GPU clusters automatically" compare tools that do not compete with each other. They put a job scheduler next to a bare-metal provisioner next to a GPU marketplace and call it a ranking. That is why teams end up owning five tools that each solve a different problem and none of the ones they actually had.

Here is the framing that fixes it. Automatic GPU cluster provisioning is three separate jobs stacked on top of each other:

1. Infrastructure provisioning - creating GPU nodes, installing drivers, wiring networking.
2. GPU scheduling - fitting many jobs onto scarce GPUs with queues, quotas, and fair sharing.
3. Developer self-service - letting an engineer get a GPU environment on demand without filing a ticket.

Almost every tool marketed as a "GPU platform" covers exactly one of the three. I run Qovery, so I will be upfront: Qovery lives in layer three only, and I will tell you plainly where the other ten tools beat it. Let me map the whole stack honestly.

Key points:

• "Automatic GPU cluster provisioning" is three different jobs: infrastructure provisioning (creating GPU nodes and clusters), GPU scheduling (queueing and sharing those GPUs across jobs and teams), and developer self-service (getting an engineer a GPU environment without a ticket). Almost every tool marketed as a GPU platform covers only one of the three.
• Best default stack on your own cloud account in 2026: Terraform or OpenTofu for the cluster, Karpenter or Cluster Autoscaler for GPU node pools, the Nvidia GPU Operator for drivers and MIG or time-slicing, and Kueue or Volcano for queueing and quotas.
• Best scheduling picks by workload: Kueue for multi-team quotas and fair sharing, Volcano for gang-scheduled distributed batch, KubeRay for Ray training and inference, HTCondor for high-throughput independent jobs, Slurm for classic HPC clusters.
• Fastest path to raw capacity: GPU marketplaces and specialized clouds (Vast.ai, TensorDock, CoreWeave, Lambda, FluidStack) hand you an instance in minutes at the lowest sticker price, but you still build the cluster, scheduler, and access layer yourself, and you inherit weaker availability and data-residency guarantees.
• Qovery covers the third layer only: it provisions and operates Kubernetes clusters with GPU-enabled node pools inside your own AWS, GCP, Azure, or Scaleway account (or your existing self-managed cluster), so teams self-serve environments with auto-stop and RBAC while the cloud bill, discounts, and reserved capacity stay in your name. It is not a GPU scheduler and not a capacity marketplace.

What does "provisioning GPU clusters automatically" actually mean?

Automatic GPU cluster provisioning is three separate problems stacked on top of each other: (1) infrastructure provisioning creates GPU nodes, drivers, and networking; (2) scheduling fits many jobs onto scarce GPUs with queues and quotas; (3) self-service lets an engineer get a GPU environment on demand. Most published "top GPU platform" lists mix all three, which is why the tools inside them do not talk to each other.

Sort the eleven tools by the layer each one really solves and the confusion disappears:

• Layer 1 - infrastructure: Terraform/OpenTofu, Karpenter, Cluster Autoscaler, cloud-managed GPU node pools, the Nvidia GPU Operator, and bare-metal tooling like vMetal.
• Layer 2 - scheduling: Kueue, Volcano, KubeRay, HTCondor, Slurm, and Run:ai (acquired by Nvidia, its scheduler open-sourced as KAI Scheduler).
• Layer 3 - self-service: internal developer platforms, including Qovery, that turn a provisioned cluster into environments engineers can create themselves.

There is a fourth axis that is not a layer at all: capacity supply. Vast.ai, TensorDock, and FluidStack solve where the GPUs physically come from, not how they are orchestrated. You can plug marketplace capacity underneath any of the three layers, or use it raw with none of them.

Four prerequisites people forget every time they wire this up:

• Driver and device-plugin installation. A GPU node with no driver is a very expensive CPU node.
• GPU quota per team, or the first team to submit eats the whole fleet.
• Node taints and labels, so ordinary CPU pods do not squat on GPU nodes.
• Idle-GPU shutdown. Idle time, not price per hour, is where most GPU budgets die.

A full working stack normally needs one tool from layer 1 and one from layer 2. You only need layer 3 once more than one team consumes GPUs and the tickets start piling up.

Which platforms provision GPU clusters automatically in 2026?

Eleven tools get called "GPU cluster provisioning platforms," and only a few of them actually create GPU infrastructure: Terraform/OpenTofu, Karpenter (paired with the Nvidia GPU Operator), SkyPilot, and bare-metal or capacity provisioners like vMetal and the marketplaces. The rest schedule jobs onto GPUs that already exist, or hand out environments on top of a cluster someone else built. Here is the map, one honest line each.

• Kueue - Kubernetes-native job queueing with ClusterQueues, quotas, and fair sharing, maintained in the kubernetes-sigs org by the Batch Working Group; currently a beta API (v0.19.2 as of August 2026), Apache 2.0 (Kueue releases). It does not provision infrastructure.
• Volcano - a CNCF batch scheduler (incubating since March 2022) with gang scheduling, queue and fair-share policies, and native MPI/PyTorch/TensorFlow job support; adopters include Huawei, Tencent, Baidu, and JD.com (CNCF Volcano project). It schedules; it does not create nodes.
• KubeRay - the Kubernetes operator for Ray clusters, with autoscaling worker groups for distributed training, tuning, and inference; Apache 2.0, and used in production by Uber, which moved its ML workloads to Ray on Kubernetes (Uber Engineering).
• HTCondor - high-throughput computing for large volumes of independent jobs, maintained by the Center for High Throughput Computing at UW-Madison; a long track record in research and grid computing (htcondor.org).
• Slurm - the HPC default from SchedMD, with Slurm-on-Kubernetes bridges now available (see the next bullet on Slinky). Its classic form runs on VMs and bare metal, not Kubernetes.
• SkyPilot - multi-cloud GPU placement that finds and provisions the cheapest available GPU across 20+ clouds and regions, with managed spot and recovery; Apache 2.0 (SkyPilot on GitHub).
• Karpenter + Nvidia GPU Operator - the standard just-in-time GPU node autoscaling path on your own Kubernetes cluster. Karpenter reached v1.0 in August 2024 (AWS); the GPU Operator installs drivers, the device plugin, DCGM metrics, and MIG or time-slicing.
• Terraform / OpenTofu - declarative provisioning of the cluster and GPU node pools; still the base layer under nearly everything above. Terraform moved to the BUSL license; OpenTofu is the MPL 2.0 fork.
• vMetal - bare-metal GPU cluster provisioning and virtualization for hardware you own or colocate. Commercial.
• Vast.ai / TensorDock / FluidStack - on-demand and marketplace GPU capacity: minutes to a running instance, no orchestration included.
• Qovery - provisions and operates GPU-enabled Kubernetes clusters inside your own cloud account and exposes them as self-service environments. It is not a scheduler; you keep Kueue, Volcano, or KubeRay underneath.

Here is the same set as a checkable table.

Footnotes: ¹ Kueue does quotas and fair sharing; all-or-nothing admission exists, but Volcano is the dedicated gang scheduler. ² Schedulers scale GPU nodes to zero only when paired with a node autoscaler (Karpenter or Cluster Autoscaler). ³ Ray autoscaler handles worker groups; gang and quotas come from a Kueue or Volcano integration. ⁴ Provides an API/CLI or dashboard for jobs or instances, not full environment self-service. ⁵ HTCondor does fair-share and quotas but is built for independent jobs, not tightly coupled gang scheduling. ⁶ Classic Slurm runs on VMs and bare metal; Kubernetes integration comes via SchedMD's Slinky (v1.0.0, November 2025) or CoreWeave's SUNK. ⁷ Managed jobs and spot recovery, not multi-team cluster quotas. ⁸ Karpenter's AWS provider is GA and an Azure provider exists; the GPU Operator itself runs on any Kubernetes. ⁹ Declarative, not a live autoscaler. ¹⁰ Runs on hardware you own or colocate, not a public cloud account. ¹¹ Hands you instances, not a managed cluster. ¹² Capacity spread across marketplace hosts, not your own multi-cloud footprint. ¹³ Qovery is not a scheduler; run Kueue, Volcano, or KubeRay underneath for gang scheduling and quotas.

How do you automate GPU node provisioning on your own AWS, GCP, Azure, or Scaleway account?

The working pattern on any hyperscaler or on your own Kubernetes cluster is the same four pieces: a dedicated GPU node pool with taints and labels, autoscaling by Karpenter or Cluster Autoscaler, drivers and GPU sharing installed by the Nvidia GPU Operator, and jobs admitted by Kueue or Volcano with a per-team quota. That combination gives you scale-from-zero GPU capacity with no human in the loop.

The concrete objects look like this:

• Pods request nvidia.com/gpu: 1 (or more). That resource only exists on a node once the device plugin is running.
• GPU nodes carry a taint (for example nvidia.com/gpu=present:NoSchedule) plus a matching toleration on GPU pods, so CPU workloads never land on a GPU box.
• The Nvidia GPU Operator runs as a set of DaemonSets and installs the driver, the container toolkit, the Kubernetes device plugin, DCGM and the DCGM exporter for metrics, GPU Feature Discovery, the MIG Manager, and a validator (Nvidia GPU Operator overview).
• Karpenter or Cluster Autoscaler watches for unschedulable GPU pods and brings up nodes just in time, then scales the pool back to zero when the queue drains.
• Kueue admits work through a ClusterQueue and a per-team LocalQueue so each team gets a fair slice.

Pick the GPU instance family per cloud: AWS P5 (H100) and P4d (A100), GCP A3 (H100) and A2 (A100), Azure ND-series, and Scaleway's H100 and L4 ranges. In 2026, the harder question is not which instance but whether you can get it: capacity reservations (AWS Capacity Blocks, GCP reservations, committed clusters on specialized clouds) now matter more than instance choice, because on-demand H100 capacity is not always there when you ask.

GPU sharing is the lever most teams skip. You have three options:

• Full GPU - one workload owns the card. Simple, wasteful for small models.
• Time-slicing - several pods share a GPU by taking turns. There is no memory or fault isolation between them (Nvidia GPU sharing docs). Good for dev, notebooks, and light inference.
• MIG (Multi-Instance GPU) - hardware partitioning into up to seven isolated instances per A100 or H100, each with its own memory and compute (Nvidia MIG User Guide). Profiles run from 1g.5gb up to 7g.40gb on the A100-40GB, and 1g.10gb to 7g.80gb on H100 and A100-80GB parts.

For small models and inference, a fractional GPU through MIG or time-slicing usually beats buying another node.

Spot and preemptible GPUs, with checkpointing, are the single biggest cost lever. AWS advertises Spot at up to 90% off on-demand (AWS Spot), and Google Cloud Spot VMs float between 60% and 91% off (Google Cloud Spot VMs). The catch is preemption, so checkpoint often enough that losing a node costs you minutes, not a day.

None of this is AWS-specific. The identical stack (node pool, Karpenter or Cluster Autoscaler, GPU Operator, Kueue or Volcano) runs on an existing on-prem cluster, on EKS, GKE, or AKS, or on any provider-managed Kubernetes. Bring your own cluster and the recipe does not change.

Two failure modes to design against: nodes that come up without a working driver (the GPU Operator's validator catches this if you let it), and pending pods on multi-node training because there is no gang scheduling, so half the workers start and sit idle waiting for the rest. That second one is exactly what Volcano and gang-aware Kueue exist to prevent.

Should you rent GPUs from a marketplace, use a specialized GPU cloud, or run your own cluster?

Rent from a marketplace when you need GPUs this afternoon for a bounded experiment. Use a specialized GPU cloud when you need multi-node training with real interconnect and a contract. Run your own cluster in your own cloud account when GPUs are a standing production dependency and the bill, compliance posture, and reserved capacity have to stay in your name.

• Marketplace / on-demand (Vast.ai, TensorDock): the fastest time to first GPU and the lowest hourly price, with the weakest guarantees on availability, interconnect, and data residency. TensorDock says it launches VMs in around 10 to 30 seconds (TensorDock), and Vast.ai advertises spinning up instances in seconds (Vast.ai docs). Great for a bounded experiment, risky as a production dependency.
• Specialized GPU clouds (CoreWeave, Lambda, FluidStack, plus the European options): better interconnect (often InfiniBand), managed Kubernetes offerings, and committed contracts. If you are weighing EU-based providers specifically, I wrote that comparison separately in European GPU Clouds: 8 Alternatives to AWS and Azure for AI Workloads, and I will not re-litigate it here.
• Hyperscaler plus your own cluster: reserved capacity and Savings Plans or CUDs, your existing VPC, IAM, and audit trail, and one FinOps report instead of three. More setup, more control, and the discounts are yours.

A quick decision checklist:

• Job duration - hours (marketplace) versus weeks (own cluster or committed contract).
• Data gravity and egress - if your data already sits in one cloud, moving it out to rent cheaper GPUs can cost more than you save.
• Multi-node interconnect - real distributed training needs InfiniBand or EFA, which marketplaces rarely guarantee.
• Compliance and residency - regulated data usually rules out anonymous marketplace hosts.
• Who owns the bill - and therefore who keeps the volume discounts.
• Who is on call - when a node dies at 3am, whose pager rings.

This is the whole point of bring-your-own-cloud (BYOC): a platform like Qovery deploys into your own account, so discounts and commitments stay in your name instead of being resold to you at a markup.

How do you give engineers self-service GPU environments without a ticket queue?

Self-service is the layer no scheduler covers. Kueue will queue a job fairly and Karpenter will bring up the node, but neither creates the environment, wires the secrets and the database, exposes an endpoint, or shuts the whole thing down when nobody is using it. That gap is where platform teams still spend their week answering tickets.

What self-service actually means in practice:

• Git-push deployment of a training or inference service, no handwritten YAML per app.
• An environment per pull request, so a reviewer sees the change running on real GPUs before merge.
• Per-environment RBAC, so teams touch only their own.
• Managed databases and object storage sitting next to the GPU workload, not in a separate ticket.

Idle non-production is the second biggest source of wasted GPU spend, right after over-provisioned nodes. The fix is environment auto-stop: dev and staging GPU environments that turn themselves off out of hours and come back on demand. This matters because self-reported GPU utilization is already low. In a 2024 survey of 1,000 companies by ClearML, the AI Infrastructure Alliance, and FuriosaAI, only 7% said their GPUs exceed 85% utilization at peak, and roughly two-thirds never exceed 70% (ClearML State of AI Infrastructure at Scale 2024).

Guardrails are what make self-service safe: Kueue or Volcano quotas underneath so no one starves a neighbor, RBAC and audit above, and node pool limits so a single team cannot drain the fleet.

Where Qovery fits. Qovery provisions and upgrades the Kubernetes clusters on AWS, GCP, Azure, Scaleway, or your existing self-managed cluster, and exposes GPU-enabled environments to teams. On AWS it automates GPU node pools directly: you enable GPU nodes in cluster settings, pick instance types (mixing On-Demand and Spot), set cluster-wide limits, and Qovery installs the Nvidia Kubernetes device plugin and drives instance selection through its Karpenter integration (Qovery GPU docs). You keep Kueue, Volcano, or KubeRay for scheduling underneath.

Let me be equally clear about what Qovery is not: it is not a GPU scheduler, not a capacity marketplace, not a Slurm replacement, and not a model-serving framework. If you need gang scheduling, you run Volcano. If you need the cheapest hourly card, you go to a marketplace.

Fair alternatives in the same layer: Run:ai (now Nvidia-owned, with its KAI Scheduler open-sourced under Apache 2.0 in April 2025) for GPU-centric orchestration, and Backstage plus custom Terraform modules for teams that would rather build the self-service layer themselves.

What does a GPU cluster actually cost, and where does the money leak?

Most GPU budgets are lost to idle time, not to price per hour. Published utilization figures sit well below what teams assume (only 7% of surveyed companies said their GPUs exceed 85% utilization at peak, per the ClearML/AIIA/FuriosaAI 2024 survey of 1,000 companies), and the fixes that move the bill most are shutting down idle environments and running spot with checkpointing, not renegotiating the hourly rate.

Cost levers, ranked by impact:

1. Stop idle non-prod environments. The cheapest GPU-hour is the one you never bill.
2. Spot or preemptible plus checkpointing - up to 90% off on AWS, 60 to 91% off on Google Cloud.
3. MIG or time-slicing for small models and inference, so one card serves many jobs.
4. Committed or reserved capacity for steady baseload you know you will run.
5. Node right-sizing last, once the four above are in place.

A worked comparison for a single H100, on published list prices checked September 2026:

Two things jump out. AWS cut P5 on-demand pricing by 44% in June 2025 (AWS), so old comparisons are stale. And a marketplace card at a third of the hyperscaler price is genuinely cheaper for a bounded job, but the hyperscaler numbers are pre-discount list prices, so compare against your actual Savings Plan or committed rate, not the sticker.

Keeping the cloud bill inside your own account matters for three unglamorous reasons: commitments and reservations accrue to you, chargeback per team comes out of one billing source, and FinOps sees a single view instead of stitching together invoices from three vendors.

One caution worth repeating: the cheapest hourly price is often the most expensive per finished job once you count queue waits, preemptions, and egress. Price the job, not the hour.

Which GPU platform should you pick for your use case?

Match the tool to the job: KubeRay for distributed Ray training and serving, Volcano for gang-scheduled batch, Kueue for multi-team quotas on Kubernetes, HTCondor or Slurm for HPC and research clusters, SkyPilot for cheapest-GPU-anywhere, a marketplace like Vast.ai or TensorDock for short experiments, Terraform plus Karpenter plus the Nvidia GPU Operator for the underlying infrastructure, and an internal developer platform like Qovery when several teams need self-service GPU environments in your own cloud account.

• One team fine-tuning occasionally - a marketplace or on-demand instances, no cluster at all. Vast.ai or TensorDock, stop it when you are done.
• Multi-team ML platform with quotas - Kubernetes plus Kueue or Volcano plus Karpenter and the GPU Operator, with self-service through an IDP on top.
• Distributed Ray training and online inference - KubeRay on your own cluster.
• Research or HPC with existing schedulers - HTCondor for high-throughput independent jobs, Slurm for tightly coupled HPC, optionally bridged to Kubernetes via Slinky or SUNK.
• GPUs already owned on-prem or colocated - bare-metal provisioning with vMetal, plus bring-your-own-Kubernetes and the GPU Operator.
• Budget-driven and the workload can move - SkyPilot across clouds with managed spot.

The verdict, in one liftable sentence: the default 2026 stack is Terraform or OpenTofu plus Karpenter and the Nvidia GPU Operator for infrastructure (layer 1), Kueue or Volcano for scheduling (layer 2), and an internal developer platform like Qovery for self-service in your own cloud account (layer 3) - three layers, one tool each, not one tool pretending to be all three.

Kubernetes is where this is heading regardless: in the 2025 CNCF Annual Cloud Native Survey (628 respondents, fielded September 2025), 66% of organizations hosting generative AI run some or all of their inference on Kubernetes (CNCF). The GPU-as-a-service market is growing to match, with Grand View Research projecting it to reach USD 14.4 billion by 2033 at a 16% CAGR (Grand View Research) and MarketsandMarkets forecasting an even steeper climb. Wherever the market lands, the three-layer split is what keeps your stack sane.

If you want the self-service layer without building it yourself, try Qovery free or come argue with me about the stack in our Discord. I would rather you pick the right tool than the Qovery-shaped one.