GCP vs AWS for ML without an MLOps team: 7 setups ranked for small teams in 2026

Key Points:

• For a greenfield team under about 20 engineers with no MLOps engineer, Google Cloud Vertex AI is the easier start. Custom training, Model Registry, Pipelines and online endpoints sit behind one service and one IAM model, instead of the SageMaker plus execution role plus VPC assembly most AWS teams build first.
• Amazon SageMaker AI wins the moment your data, compliance footprint or committed spend already lives on AWS. Moving ML to a second cloud costs more in egress, identity and networking work than the console ease you gain on Google Cloud.
• Idle GPUs drive the bill, not the choice of hyperscaler. Vertex AI online prediction and SageMaker AI real-time endpoints both bill per provisioned node-hour whether a request arrives or not. The on-demand price gap between comparable L4 instances on AWS and Google Cloud is about 14%. A GPU idle two thirds of the day wastes 67% of its cost.
• The third path is managed Kubernetes (GKE Autopilot or Amazon EKS with Karpenter) operated by an internal developer platform such as Qovery inside your own AWS, GCP, Azure or Scaleway account, or your existing Kubernetes cluster. You keep your cloud bill, your committed-use discounts and your portability. Qovery is not an ML platform and does not replace managed training, AutoML or a feature store.
• Decide on three questions, in order: where your training data already sits, whether you need managed training or only inference of your own models, and who is on call at 2am when the endpoint fails.

Small teams keep getting told to "just ship the model" with nobody on staff who has ever run a GPU node pool. According to Gartner, only 48% of AI projects reach production, and the ones that do take 8 months on average to get from prototype to production. Most of that gap is operations work.

I talk to CTOs of 5 to 30 person teams every week who are stuck on the same question: GCP vs AWS for machine learning, when nobody does MLOps full time. Below I rank seven setups against six criteria, with the pricing I checked and the arithmetic behind each verdict.

All prices were checked in October 2026, mostly for us-east-1 and us-central1. Cloud pricing changes often, so re-check the linked pages before you commit.

GCP or AWS for a small ML team with no MLOps engineer: what is the short answer?

Google Cloud Vertex AI is easier for a greenfield small team, and Amazon SageMaker AI wins when your data and committed spend already live on AWS. Managed Kubernetes plus an internal developer platform is the third path, and it keeps the choice reversible.

The decisive variable is where your training data already lives. Model quality and GPU list price matter far less. Both clouds are credible defaults: in Q2 2026, AWS held 28% of the cloud infrastructure market and Google Cloud held 15% (Synergy Research Group).

"Easier" needs a concrete definition, so here is mine:

• Services touched before v1 serves traffic. On Google Cloud Vertex AI that is usually Vertex AI, Cloud Storage and maybe BigQuery. On Amazon SageMaker AI it is usually SageMaker AI, S3, IAM, VPC, ECR and CloudWatch.
• IAM concepts to learn. A project and a service account on Google Cloud. An execution role, a trust policy and VPC subnets on AWS.
• Components that can page you at 2am. Fewer is better when nobody owns infrastructure.

A wrong-cloud migration costs more than the difference between the two clouds. Moving a 10 TB training set out of S3 to the internet costs about $900 at $0.09 per GB for the first 10 TB (AWS Data Transfer pricing). That is before you rebuild identity, networking and pipelines on the other side.

How did we rank these ML setups, and what counts as 'easy' without an MLOps team?

We scored seven setups against six criteria that only matter when nobody does MLOps full time, from time to first endpoint to who answers the page on day 2. The ranking is fit-based. There is no single winner, only a best match for your data gravity, traffic shape and headcount.

We compare complete setups (cloud + ML service + operating model):

• Time to first deployed endpoint: hours, days or weeks, including GPU quota approval.
• Infrastructure you must own: IAM and execution roles, VPC and subnets, node pools, NVIDIA drivers and device plugins, autoscaling.
• GPU cost control: scale-to-zero, idle environment shutdown, spot capacity, committed-use discounts and Savings Plans.
• Portability: how much of the workload is a container image versus vendor SDK code.
• Runs in your own cloud account: your bill, your discounts, your data residency, your private networking.
• Day-2 operations owner: the vendor, you, or a platform that operates it for you.

The scoring assumes zero full-time MLOps headcount, because that is the reality for most small teams. Operating model beats tooling choice. 90% of organizations have adopted at least one internal platform according to the 2025 DORA report. In the CNCF Annual Cloud Native Survey, 36% of respondents name lack of training as a top challenge.

"Just hire someone" is expensive. The median US salary for a DevOps engineer is $165,000 in the Stack Overflow 2025 Developer Survey, and MLOps specialists are rarer still.

Which ML setup should a small team pick? Seven options compared

Google Cloud Vertex AI and Amazon SageMaker AI are the fastest managed paths, GKE Autopilot and Amazon EKS with Karpenter are the cheapest at steady GPU load, and Modal, Replicate and Hugging Face Inference Endpoints win on spiky traffic. Qovery keeps Kubernetes-level portability inside your own cloud account without a platform engineer.

Use this table as a fit map for your own situation. The right row changes with your data gravity.

Cost shape by traffic pattern:

The break-even sits around 50% GPU utilization against a 3-year commitment and around 88% against on-demand. I show the arithmetic in the serverless section below.

Is Google Cloud Vertex AI really the easiest place to start for a small team?

Yes, for a small greenfield team: Google Cloud Vertex AI puts custom training, Model Registry, Pipelines, Feature Store and online and batch prediction behind one service and one IAM model, so a five-person team can ship a first endpoint in days. It is the wrong pick if your warehouse and production services already sit on AWS, or if you need a custom serving stack like vLLM, Triton or Ray Serve.

What Google Cloud Vertex AI bundles:

• Custom training jobs, Model Registry, online and batch prediction endpoints.
• Vertex AI Pipelines, Feature Store and Model Garden.
• A direct path from BigQuery to Vertex AI, which removes a whole data-plumbing project for a small team.
• Gemini and Model Garden as a first-party route to foundation models, with no serving infrastructure to stand up.

The honest downsides:

• Online prediction bills per node-hour while a model is deployed, whether traffic arrives or not (Google Cloud Vertex AI pricing). A g2-standard-4 prediction node with one L4 costs $0.81293 per hour in us-central1, about $593 a month for a single always-on node.
• GPU quota can block your first deployment. The Vertex AI quotas page lists a default of 0 for several custom training GPU types, including L4 and A100 80GB, and you request increases through the console.
• Training is billed per machine type plus accelerator: n1-standard-4 at about $0.22 per hour plus an L4 at about $0.64 per hour (Google Cloud Vertex AI pricing).
• Accelerator availability varies by region, so check your target region before you design around a GPU type.

Lock-in is real but bounded. Vertex AI Pipelines and Feature Store code does not move to AWS unchanged. Containerized training and serving code does.

When is Amazon SageMaker AI the better choice despite the extra complexity?

Amazon SageMaker AI is the better choice whenever your data and production services already run on AWS, because moving data, identity and networking to a second cloud costs more than any console ease you gain on Google Cloud. Serverless Inference and scale-to-zero on inference components close most of the small-team gap.

Strengths:

• Breadth: Studio, training jobs, real-time endpoints, Serverless Inference, Batch Transform, and the deepest instance and accelerator selection of any cloud.
• SageMaker Savings Plans reduce costs by up to 64% (AWS).
• Amazon Bedrock next door when you would rather not serve a model at all. Amazon Nova Micro costs $0.000035 per 1,000 input tokens and $0.00014 per 1,000 output tokens on demand (Amazon Bedrock pricing).

The small-team features most people miss:

• Serverless Inference bills per second by memory size, e.g. $0.00008 per second for 4 GB, plus $0.016 per GB of data processed (Amazon SageMaker AI pricing). It runs on CPU only, so it fits small models.
• Scale-to-zero on real-time endpoints works only when the endpoint hosts inference components with a minimum instance count of 0 (AWS documentation). Scaling back up takes several minutes, and requests fail during that window.
• Free tier: 125 hours of m4.xlarge or m5.xlarge real-time inference and 150,000 seconds of Serverless Inference in the first two months.

The friction is execution roles, IAM policies, VPC and subnet configuration, and the number of services you touch before v1 serves traffic. The cost traps are always-on real-time endpoints (an ml.g5.xlarge at $1.408 per hour is about $1,028 a month), forgotten Studio and notebook instances, and provisioned concurrency.

Skip Amazon SageMaker AI if you are a five-person team with no AWS experience and no AWS footprint. The breadth works against you there.

Should you skip the managed ML platform and just run containers on GKE or EKS?

If your workload is mostly inference of your own containerized models plus some batch jobs, managed Kubernetes with GPU nodes is cheaper at steady load and far more portable than Google Cloud Vertex AI or Amazon SageMaker AI endpoints. In exchange, you take back every day-2 operation those services were hiding: cluster upgrades, NVIDIA drivers, autoscaler tuning and on-call.

• GKE Autopilot charges a $0.10 per cluster-hour management fee, with a $74.40 monthly credit per billing account that covers one Autopilot or zonal cluster (GKE pricing). GPU pods on Autopilot use node-based billing, so you pay for the whole node.
• Amazon EKS with Karpenter charges $0.10 per cluster-hour, rising to $0.60 per cluster-hour on extended support (Amazon EKS pricing). Karpenter handles GPU node provisioning, consolidation and spot interruptions.
• Spot capacity is the big lever for training and batch: up to 90% off on AWS EC2 Spot and up to 91% off on Google Cloud Spot VMs.
• This path wins for custom serving stacks (vLLM, Triton, Ray Serve) and for packing several models on one GPU.

The inter-cloud GPU gap is smaller than the idle gap. An AWS g6.xlarge with one L4 costs $0.8048 per hour in us-east-1 (AWS EC2 pricing). A Google Cloud g2-standard-4 with one L4 costs $0.7068 per hour in us-central1 (Google Cloud).

• Gap: ($0.8048 - $0.7068) / $0.7068 = about 14%, or $72 a month per always-on GPU (730 hours).
• Idle cost: a g2-standard-4 busy 8 hours a day wastes two thirds of $516 a month, about $344, nearly five times the inter-cloud gap.

The industry data says idle GPUs are normal. In the ClearML State of AI Infrastructure at Scale 2024 survey, only 7% of companies exceed 85% GPU utilization at peak. Organizations estimate 29% of their cloud spend is wasted according to the Flexera 2026 State of the Cloud Report.

Where it hurts with no MLOps function:

• GPU plumbing. GKE installs NVIDIA drivers only if you ask it to (GKE documentation). The EKS AL2023 NVIDIA AMIs leave out the NVIDIA Kubernetes device plugin, which you install separately (Amazon EKS documentation).
• Cluster and node upgrades, autoscaler tuning, GPU observability and an on-call rotation.

Kubernetes is usually the right substrate for steady-load inference. What breaks small teams is the operating model.

How does Qovery fit between managed ML platforms and raw Kubernetes?

Qovery is the pick when you want Kubernetes-level portability and your own cloud bill without hiring a platform or MLOps engineer: Qovery deploys and operates your containerized ML services inside your own AWS, GCP, Azure or Scaleway account, or your existing Kubernetes cluster. Qovery is not an ML platform and does not replace Google Cloud Vertex AI or Amazon SageMaker AI for managed training, AutoML, a feature store or a hosted model registry.

Qovery is an internal developer platform that runs in your own cloud account (BYOC), so the bill, the committed-use discounts and the Savings Plans stay in your name. That matters for ML because GPU commitments are where the money is:

• Google Cloud flexible CUDs cut G2 (L4) pricing by 21% for 1 year and 43% for 3 years, and resource-based CUDs reach up to 55% on most machine series (Google Cloud).
• AWS Compute Savings Plans reduce costs by up to 66% (AWS).

What Qovery does for ML teams:

• Environment auto-stop for non-production. Idle GPU staging and evaluation environments are the most common avoidable ML spend, and auto-stop shuts them down when nobody is using them. With only 7% of companies above 85% GPU utilization, this is where the savings are.
• Git-push deployments of inference services and batch jobs.
• Preview environments per pull request, so you can test a new model version against real infrastructure.
• Managed cluster upgrades and per-environment RBAC.
• Databases backed by managed cloud services.

The strongest point in a GCP vs AWS debate is portability. The same Qovery setup runs on AWS, GCP, Azure, Scaleway or your existing Kubernetes cluster, so the hyperscaler choice stops being irreversible.

If you need managed training jobs, AutoML, a feature store or a hosted model registry, use Google Cloud Vertex AI or Amazon SageMaker AI, alongside Qovery or on their own. Qovery fits teams that already containerize their inference services and have nobody to operate Kubernetes.

What about serverless GPU platforms like Modal, Replicate or Hugging Face Inference Endpoints?

Modal, Replicate and Hugging Face Inference Endpoints get an inference endpoint live faster than anything else, often in under an hour, and they are the right call for spiky traffic, demos and pre-product-market-fit teams with light compliance needs. They stop being the cheap option once a GPU is busy for much of the day, and they run in the vendor's account.

• Modal bills an L4 at $0.000222 per second and an A100 80GB at $0.000694 per second, scaling to zero by default (Modal pricing). Modal charges CPU and memory separately, and region pinning costs extra.
• Replicate bills an A100 80GB at $0.0014 per second, or $5.04 per hour (Replicate pricing).
• Hugging Face Inference Endpoints deploy open models into AWS or GCP regions from a UI, at $0.80 per hour for an L4 on AWS and $0.70 on GCP, billed per minute (Hugging Face).

The crossover arithmetic, for one L4:

• Modal L4: $0.000222 × 3,600 = $0.80 per hour, paid only while running.
• Google Cloud g2-standard-4 on demand: $0.7068 per hour. Break-even: 0.7068 / 0.80 = about 88% utilization.
• With a 1-year flexible CUD ($0.5584 per hour), break-even drops to about 70%.
• With a 3-year flexible CUD ($0.4029 per hour), break-even drops to about 50% (Google Cloud).

Below roughly 50% GPU utilization, per-second serverless is cheaper than even committed capacity in your own account. Above 70%, owning the GPU wins.

The trade-offs: the workload runs on the vendor's account, VPC peering and data residency control are limited, your committed-spend discounts do not apply, and scale-to-zero adds cold-start latency. Pick serverless GPU for pre-PMF products, demos and bursty batch inference. Avoid it for regulated data and steady high-traffic inference.

How do you choose in practice? A five-question decision path

Answer five questions in order and the choice follows. For most small teams, only the first two change the outcome: where your data already lives, and whether you need managed training or only inference of your own models.

1. Where does your training data live? BigQuery points to Google Cloud Vertex AI. An S3 lake or Redshift points to Amazon SageMaker AI.
2. Do you need managed training, AutoML, a feature store or a model registry? If yes, pick a managed ML service. If you only serve your own models, containers on Kubernetes are an option.
3. Is traffic spiky or steady? Spiky points to Modal, Replicate, Hugging Face Inference Endpoints or scale-to-zero endpoints. Steady points to committed capacity in your own account.
4. Do compliance, data residency or a committed-spend agreement require your own cloud account? That rules out vendor-account platforms and points to hyperscaler-native services or to managed Kubernetes operated by Qovery.
5. Who is on call at 2am? If nobody owns infrastructure, pick the setup with the fewest components you must operate.

How do I deploy an ML service to my own AWS or GCP account from Claude Code or Cursor?

Install the Qovery Agent Skills once, then ask your coding agent in plain language to deploy the service into your own cloud account. The agent runs the Qovery workflow and asks you to approve each action.

Install the skills globally for every compatible agent (Claude Code, Cursor, Codex, OpenCode, Gemini CLI):

curl -fsSL https://skill.qovery.com/install.sh | bash

Or scope them to a single project:

curl -fsSL https://skill.qovery.com/install.sh | bash -s -- --project

Add the MCP Server to Claude Code. It uses OAuth and is read-only by default:

claude mcp add --transport http qovery https://mcp.qovery.com/mcp --callback-port 4242

Enable write access only when you explicitly want it. This also requires enabling write access in the Qovery Console:

claude mcp add --transport http qovery "https://mcp.qovery.com/mcp?read_write=true" --callback-port 4242

The skills (qovery-onboard, qovery-deploy, qovery-troubleshoot, qovery-optimize, qovery-speedup, qovery-preview, qovery-terraform) deploy. The MCP Server operates what already exists.

Then ask: "deploy this inference service to my GCP account with Qovery and give me a preview environment per pull request." Setup details are in the AI agent quickstart and on the Claude Code integration page.