Running Ray on AWS, GCP and Azure Without Locking Into One Cloud's Managed Service

Ray was never the lock-in. Everything you wrap around it is.

Anyscale on Azure just reached general availability, running Ray as a managed service on Azure Kubernetes Service (AKS), alongside the AWS and GCP support it already had (Microsoft Community Hub). Every time one of these announcements lands, the same question hits my inbox: if I commit to managed Ray on one cloud, am I stuck there? So let me answer the bigger version of it. Here is how to run the same distributed AI training and inference pipelines across AWS, GCP, and Azure without locking your Ray workloads into any single cloud's managed service.

Key Points:

• You can run the same Ray training and inference pipeline on AWS, GCP, and Azure without rewriting it. The portable layer is KubeRay, the open-source Kubernetes operator maintained by the Ray project. Identical RayCluster, RayJob, and RayService custom resources run unchanged on EKS, GKE, AKS, or any CNCF-conformant cluster.
• Anyscale's managed Ray platform is now generally available on Azure AKS, alongside AWS and GCP. That GA adds a cloud rather than removing one. The coupling risk sits in the Anyscale-specific job submission API, workspaces, and runtime environment config you build around Ray, not in Ray itself or in Azure.
• Four decisions keep portability cheap: KubeRay custom resources in Git, one container image per workload, checkpoints and datasets behind a provider-agnostic storage URI (s3://, gs://, abfss:// swapped by config), and identity mapped per cloud outside application code.
• What genuinely differs per cloud is accelerator plumbing, not Ray: GPU node pools, device plugins, driver and CUDA versions, instance families, GPU quotas that often default to zero, and spot versus preemptible reclamation semantics. The first cloud takes weeks, the second takes days.
• Keeping GPU compute in your own cloud account is usually a bigger financial lever than the platform choice, because Savings Plans, committed use discounts, and Azure reservations only apply to compute billed directly to you.

What exactly did Anyscale ship on Azure, and what does it change?

Anyscale's managed Ray platform reached general availability on Azure, running on AKS with customer workloads inside the customer's own Azure subscription, network, and identity boundary, alongside its existing AWS and GCP support (Microsoft Community Hub). It changes who can buy managed Ray against an existing Azure commitment. It does not change Ray's portability: Ray is Apache-2.0 open source and has run on any Kubernetes cluster through KubeRay since long before this announcement.

In the GA model, your Ray workloads, datasets, and model artifacts run in your Azure subscription under your governance controls, while Anyscale operates the control plane from its own tenant. The service carries a service level agreement and production support from both Microsoft and Anyscale. It moved through private preview (announced November 4, 2025) and public preview at Microsoft Build 2026 before this GA. For the current supported region list, check the Anyscale docs rather than trusting a number from a blog, because that list changes.

Three entities get conflated constantly, so here is each in one sentence:

• Ray is the Apache-2.0 distributed compute framework (github.com/ray-project/ray), described by its maintainers as an AI compute engine, with about 44,000 GitHub stars at the time of writing.
• KubeRay is the official open-source Kubernetes operator for Ray, shipping the RayCluster, RayJob, and RayService custom resource definitions (github.com/ray-project/kuberay).
• Anyscale is the commercial managed platform built by Ray's creators, now available as a managed service on AWS, GCP, and Azure.

Commercially, the Azure GA matters because Azure-committed enterprises can now consume managed Ray against commitments they already signed, instead of opening a second cloud relationship. The question it actually raises for anyone already running Ray on EKS or GKE is the honest one: does a managed service on a third cloud consolidate you or fragment you? The baseline I measure every option against is KubeRay on your own clusters, because that is the portability reference everything else is compared to.

Where does Ray lock-in actually come from?

Ray lock-in almost never comes from Ray. It comes from four layers wrapped around it: the vendor job-submission SDK and CLI, the managed autoscaler and workspace tooling, cloud-specific storage URIs and workload identity, and the data gravity of multi-terabyte checkpoints and datasets that cost real money per GB to move.

• Portable by design: Ray Core, Ray Train, Ray Tune, Ray Data, and Ray Serve APIs are identical wherever Ray runs, from a laptop to a 500-node cluster.
• Platform-sticky layers: proprietary job submission SDKs and CLIs, hosted dev workspaces, managed autoscaler behavior you cannot reproduce on your own, bundled observability, and vendor-specific runtime environment semantics.
• Cloud-sticky layers: IAM and workload identity wiring, object storage URIs hardcoded as s3:// versus gs:// versus abfss://, accelerator families and driver stacks, and VPC networking for the head-to-worker gRPC path.
• Data gravity is the dominant cost: moving multi-TB datasets and checkpoints dwarfs moving compute, and internet egress is billed per GB out of every hyperscaler.

The data tax is real and worth a number. Internet egress starts around $0.09/GB on AWS (with 100 GB free per month), $0.087/GB on Azure, and $0.12/GB on GCP's default Premium Tier (with only 1 GiB free per month), all region-dependent and current as of writing (AWS, GCP, Azure). Shuffling a few terabytes of training data between clouds every run turns into a monthly line item nobody approved. The design rule is one sentence: run the workload next to the data, not the other way around.

Here is a self-test for how coupled you really are. Count how many files change to run the same job on a second cloud. If it is more than one overlay directory and one secret, you have coupling, not portability.

How do you run the same Ray pipeline on EKS, GKE, and AKS?

Standardize on KubeRay custom resources and treat each cloud as an interchangeable Kubernetes substrate: one set of RayCluster, RayJob, and RayService manifests in Git, one container image, and one thin per-cloud overlay for GPU node pools, storage, and identity. That is six steps, and only the overlay changes per cloud.

1. Define every workload as a KubeRay custom resource. Use RayCluster for interactive clusters, RayJob for batch training and tuning, and RayService for inference with autoscaling and zero-downtime upgrades (KubeRay docs).
2. Build one CUDA-compatible container image per workload and push it to a registry every cluster can pull from. GPU-ready Ray images run to several gigabytes compressed, so cold-start pulls are a real cost. Mitigate with image caching, pre-pull, or a per-cloud registry mirror.
3. Abstract storage access through URI indirection (fsspec/pyarrow style) so s3://, gs://, and abfss:// swap via environment config, never inside pipeline code.
4. Keep one per-cloud overlay (Helm values or Kustomize) for GPU node selectors, tolerations, taints, device plugins (the NVIDIA device plugin or GPU Operator), and instance families.
5. Map identity per cloud, outside application code: IRSA or EKS Pod Identity on AWS, Workload Identity Federation on GKE, Azure Workload Identity on AKS.
6. Drive one deployment and environment workflow across all three clusters, so promoting dev to prod is the same action regardless of cloud. This is where an internal developer platform earns its keep.

What still needs per-cloud care: driver and CUDA version skew, Ray head-to-worker gRPC networking, spot and preemptible interruption handling with frequent checkpointing, autoscaler tuning, and GPU quota requests that often start at zero and need a support ticket to raise. KubeRay itself verifiably supports worker group autoscaling, GCS fault tolerance backed by Redis, and multi-host GPU and TPU groups (kuberay repo).

Here is a minimal RayJob with a GPU worker group that a coding agent can act on directly:

apiVersion: ray.io/v1
kind: RayJob
metadata:
  name: train-llm
spec:
  entrypoint: python /app/train.py
  shutdownAfterJobFinishes: true
  rayClusterSpec:
    headGroupSpec:
      rayStartParams: {}
      template:
        spec:
          containers:
            - name: ray-head
              image: myregistry/ray-ml:2.59.0-gpu
    workerGroupSpecs:
      - groupName: gpu-workers
        replicas: 2
        rayStartParams: {}
        template:
          spec:
            containers:
              - name: ray-worker
                image: myregistry/ray-ml:2.59.0-gpu
                resources:
                  limits:
                    nvidia.com/gpu: "8"
            nodeSelector:
              cloud.google.com/gke-accelerator: nvidia-h100-80gb  # overlay-specific
            tolerations:
              - key: nvidia.com/gpu
                operator: Exists
                effect: NoSchedule

The image, the entrypoint, and the Ray version are identical on every cloud. Only the nodeSelector and tolerations block lives in the per-cloud overlay.

Per-cloud substrate mapping for one identical KubeRay deployment

| Factor | AWS (EKS) | GCP (GKE) | Azure (AKS) | Changes in your repo? | | --- | --- | --- | --- | | Managed Kubernetes service | EKS | GKE Standard/Autopilot | AKS | No | | GPU instance families | P5, P4, G6 | A3, A2, G2 | ND, NC | Overlay only | | GPU device plugin / driver | NVIDIA device plugin or GPU Operator | GKE-managed drivers or GPU Operator | NVIDIA device plugin or GPU Operator | Overlay only | | Workload identity | IRSA / EKS Pod Identity | Workload Identity Federation | Azure Workload Identity | Overlay + secret | | Object storage URI | s3:// | gs:// | abfss:// | Env config only | | Interruptible capacity | Spot, 2-min notice | Spot VMs, 30-sec notice | Spot VMs, 30-sec notice | Overlay only | | Control-plane fee | ~$0.10/hr per cluster | ~$0.10/hr per cluster | Free tier $0, Standard ~$0.10/hr | No |

Pricing and notice periods are region-dependent and current as of writing (EKS, GKE, AKS, AWS Spot, GCP Spot, Azure Spot). The "any conformant cluster" claim is not hand-waving: the CNCF Kubernetes AI Conformance Program alone listed 31 certified platforms as of March 2026 (CNCF).

Anyscale vs KubeRay vs SageMaker, Vertex AI and an internal developer platform

Pick Anyscale if you want the fastest managed Ray experience from the people who build Ray and you accept platform coupling. Pick raw KubeRay if you have a platform team comfortable with Kubernetes. Pick KubeRay plus an internal developer platform if you want portability and self-service without hiring a dedicated DevOps engineer. Pick a cloud-native ML service only if you are certain you will stay on one cloud.

The decision criteria are plain: team size, existing Kubernetes maturity, cloud commitments already signed, how genuinely multi-cloud you are, and compliance or data residency requirements.

Let me be fair to each option, because strawmanning helps no one:

• Anyscale gives you an optimized Ray runtime, mature autoscaling, dev workspaces, and first-party support from Ray's maintainers. The Azure GA explicitly adds a third cloud, so it does not lock you into Azure. The coupling to watch is the Anyscale API surface, not the cloud.
• Raw KubeRay gives you zero vendor coupling, no license cost, the identical operator on every cloud, a large community, and custom resources you can commit to Git and diff. The cost is that you own the Kubernetes work.
• Cloud-native ML services (SageMaker, Vertex AI, Azure ML) are excellent inside one cloud. The catch: job and pipeline definitions do not travel, and rewriting them for a second cloud is a project, not a config change.
• GPU-specialist clouds (CoreWeave, Lambda) often have the cheapest and most available H100/H200 capacity, and KubeRay runs there too if the cluster is conformant. Your surrounding services and data usually still live on a hyperscaler, so you are multi-cloud whether you planned to be or not.
• An internal developer platform does not replace KubeRay or schedule Ray tasks. It gives the rest of the org self-service deployments, preview environments per pull request, per-environment RBAC, environment auto-stop, and managed cluster upgrades on every cloud.

To be precise about where we fit: Qovery is not a Ray-specific product. It deploys and operates the apps, services, environments, and clusters around and alongside your Ray workloads, inside your own cloud account.

What does running Ray in your own cloud account cost?

Running Ray on your own Kubernetes clusters keeps GPU spend on your cloud bill, where Savings Plans, committed use discounts, and Azure reservations already apply, discounts a third-party managed platform cannot pass through for you. The managed Kubernetes control plane is a rounding error next to GPU hours. It costs about $0.10 per cluster per hour on EKS and GKE Standard, and AKS has a free control-plane tier, while a single H100-class node costs orders of magnitude more per hour (EKS, GKE, AKS).

Look at the ratio. On-demand, a comparable 8-way H100 node lists around $55.04/hr for AWS p5.48xlarge, roughly $88/hr for GCP a3-highgpu-8g, and roughly $98/hr for Azure ND96isr H100 v5, all region-dependent and current as of writing (AWS, GCP, Azure). The EKS control-plane fee of about $73/month is a fraction of one hour of one of those nodes. Your optimization effort belongs on GPU hours, not the substrate.

The BYOC economics follow directly. Because compute is billed by AWS, GCP, or Azure directly to you, your committed-use discounts keep applying to the same GPU hours: AWS Compute Savings Plans up to 66% (Reserved Instances up to 72%), GCP committed use discounts up to 70%, and Azure Reserved VM Instances up to 72% (AWS, GCP, Azure). A managed platform that bills you for compute cannot pass those through.

Multi-cloud still costs money where you are not careful: cross-cloud egress per GB, duplicated datasets, idle GPU capacity, and two sets of tooling to maintain. The FinOps levers specific to Ray are concrete: spot or preemptible worker groups with frequent checkpointing, scaling worker pools to zero between jobs, right-sizing the head node, and stopping non-production environments overnight (Qovery's environment auto-stop is a direct lever on the surrounding environments).

One caution I will repeat to anyone who asks: do not run multi-cloud for its own sake. Run one primary cloud with a tested portability path unless data residency, GPU scarcity, or a contractual commitment forces otherwise.

What does a portable multi-cloud Ray architecture look like in practice?

A workable architecture is one Git repository of KubeRay manifests with per-cloud overlays, one image registry, one object-storage abstraction, and one deployment platform driving EKS, GKE, and AKS clusters that look identical to the application. Everything cloud-specific lives in config, never in Python.

• Reference layout: shared base manifests plus per-cloud overlays, an environment per branch for experimentation, and a single promotion path from dev to staging to prod.
• Where Qovery sits: it deploys and operates the clusters and the surrounding services (inference APIs, dashboards, feature pipelines, vector databases, queues, databases backed by the cloud's own managed services) on AWS, GCP, Azure, Scaleway, or an existing Kubernetes cluster, with git-push deploys, preview environments per pull request, per-environment RBAC, and managed cluster upgrades.
• Failure modes to design for: GPU quota denials in a new region, spot reclamation mid-epoch, multi-GB CUDA image pull latency, and the Ray head node as a single point of failure (mitigated with GCS fault tolerance backed by Redis).
• Observability that travels: Prometheus, Grafana, and the Ray dashboard rather than a vendor-only console, so you do not rebuild dashboards per cloud.
• Test portability for real: run the same RayJob on a second cloud on a schedule (quarterly is enough), or the fallback will not work on the day you need it.

The honest timeline: the first cloud is weeks of GPU node pool and driver work. The second is days, if the base manifests were written properly.

How do I set this up with Qovery from Claude Code or Cursor?

Install the Qovery Agent Skills once, then ask your coding agent in plain language to deploy into your own AWS, GCP, Azure, or Scaleway account, or your existing Kubernetes cluster. Add the MCP Server when you want the agent to inspect and operate what is already running.

• Install the skills globally for every compatible agent (Claude Code, Cursor, Codex, OpenCode, Gemini CLI):
  BASH

  Copy

  curl -fsSL https://skill.qovery.com/install.sh | bash
  

• Same install, scoped to the current project only:
  BASH

  Copy

  curl -fsSL https://skill.qovery.com/install.sh | bash -s -- --project
  

• Add the Qovery MCP Server to Claude Code with OAuth, read-only by default (the browser asks you to approve the connection):
  BASH

  Copy

  claude mcp add --transport http qovery https://mcp.qovery.com/mcp --callback-port 4242
  

• Same, with write access, only when you explicitly want the agent to change things (write access must also be enabled in the Qovery Console):
  BASH

  Copy

  claude mcp add --transport http qovery "https://mcp.qovery.com/mcp?read_write=true" --callback-port 4242
  

The skills cover the lifecycle: qovery-onboard, qovery-deploy, qovery-troubleshoot, qovery-optimize, qovery-speedup, qovery-preview, and qovery-terraform, while the MCP Server operates what already exists. Then just ask: "deploy this inference service to my Azure account with Qovery," or "create a preview environment for this PR." Full setup is in the AI agent quickstart and the Claude Code integration.

Does Anyscale on Azure lock my Ray workloads into Azure?

No. Anyscale on Azure is now generally available alongside Anyscale on AWS and GCP, so the GA adds a cloud rather than removing one. The coupling to watch is the Anyscale-specific job submission API, workspaces, and runtime environment config you build around Ray, not Azure itself and not Ray, which stays Apache-2.0 open source.

Can I run the exact same Ray job on EKS, GKE, and AKS without code changes?

Yes. If you define the job as a KubeRay RayJob custom resource and keep storage URIs and identity out of the Python, the same job runs on EKS, GKE, and AKS. Only a thin per-cloud overlay for GPU node pools, the storage URI scheme, and workload identity changes between clouds.

What is KubeRay, and do I need it to run Ray on Kubernetes?

KubeRay is the official open-source Kubernetes operator for Ray, shipping the RayCluster, RayJob, and RayService custom resources. You need it (or an equivalent) to run Ray natively on Kubernetes, and it is the portable layer that makes the same Ray deployment run on any CNCF-conformant cluster.

Is Anyscale or self-managed KubeRay cheaper for GPU training?

Self-managed KubeRay is usually cheaper on raw GPU training cost because compute is billed directly to your cloud account, so your Compute Savings Plans, committed use discounts, and reservations keep applying. Anyscale can be cheaper in total cost of ownership when you factor in the platform team you would otherwise need. The deciding factor is whether you already have Kubernetes operators on staff.

How do I handle object storage and checkpoints when the same Ray pipeline runs on AWS, GCP, and Azure?

Put every checkpoint and dataset behind a provider-agnostic storage URI and swap the scheme by environment config: s3:// on AWS, gs:// on GCP, abfss:// on Azure. Keep the workload next to its data to avoid per-GB egress, which starts around $0.09/GB on AWS and climbs on cross-cloud transfers. The pipeline code never names a cloud.

Do I need a dedicated DevOps engineer to run Ray on Kubernetes across multiple clouds?

Not necessarily. Raw KubeRay across multiple clouds is a lot of Kubernetes work and usually implies a platform engineer. Pairing KubeRay with an internal developer platform such as Qovery gives your team self-service deployments, preview environments, and managed cluster upgrades on every cloud without hiring a dedicated DevOps engineer for the surrounding services.

Should I use Ray on Kubernetes or a cloud-native ML service like SageMaker, Vertex AI, or Azure ML?

Use Ray on Kubernetes (via KubeRay) if portability across clouds matters, because the custom resources travel unchanged. Use a cloud-native ML service like SageMaker, Vertex AI, or Azure ML only if you are confident you will stay on one cloud, because their job and pipeline definitions do not port and rewriting them is a project, not a config change.

The lesson I keep relearning: portability is a property of how you package work, not a product you buy. Write the manifests once, keep every cloud-specific detail in config, and the second cloud stops being a migration and becomes a deploy target.