How to Self-Host an 800B+ Parameter Open Model (Like Meta's "Muse Spark Open") on Kubernetes in Your Own Cloud

Key Points:

• Muse Spark Open is real and you can download it. Meta Superintelligence Labs released it on October 15, 2026: 816B total parameters, about 42B active per token, 256 experts, a 1M-token context, BF16 and NVFP4 weights, under the Muse Model License (Hugging Face model card).
• An 816B model is a memory problem first. Weights take about 1.63 TB at BF16, about 816 GB at FP8, and about 530 GB in Meta's NVFP4 build. Meta's own floor is "at least 700GB of VRAM for NVFP4 and 2TB for BF16" once you add KV cache and runtime overhead (model card).
• The entry point is one 8-GPU node with more than 1 TB of HBM. 8xH200 (1,128 GB), 8xB200 (1,440 GB), or 8xMI300X (1,536 GB). BF16 needs multiple nodes on InfiniBand or EFA, and that node boundary is where most attempts break.
• The Kubernetes stack is standard. NVIDIA GPU Operator, vLLM or SGLang, LeaderWorkerSet for multi-pod replicas, DCGM metrics, and a shared filesystem for the weights.
• Most teams should call an API. Together AI lists Muse Spark Open at $1.25 input and $4.25 output per million tokens as of October 2026 (Together AI pricing). An 8xH200 node on AWS costs $63.30 per hour on demand. Self-hosting only wins with sustained high utilization, data that cannot leave your VPC, or customization an endpoint cannot give you.

People keep asking me how to self-host Meta's 816-billion-parameter model. Ten days ago I would have answered that Meta had not released one. Now it has, and the harder problem shows up: most teams asking cannot keep the node busy enough to pay for it.

This is the playbook I would follow. The math works for Muse Spark Open and for any other 400B-1T open-weights model, so it stays useful when the next one ships.

Does Meta's "Muse Spark Open" (816B parameters) actually exist?

Yes. Meta Superintelligence Labs released Muse Spark Open on October 15, 2026 as "the open weights version of Muse Spark", an 816B-parameter sparse mixture-of-experts model that activates roughly 42B parameters per token (Meta developer blog). The weights sit in the gated meta-models/Muse-Spark-Open-816B-A42B repository on Hugging Face, which is a different org from the meta-llama org that hosts Llama.

The original Muse Spark, announced in April 2026, stayed closed and app-only (Meta newsroom). The open model comes with 62 layers, grouped-query attention (128 query heads, 16 KV heads, head dimension 64), and text, image, video, and audio input (model card).

Dense vs MoE is the sizing fact that matters most. Total parameters decide how much HBM you need. Active parameters decide how many FLOPs each token costs. An 816B MoE with 42B active needs the same memory as a dense 816B model, but each token costs about the compute of a 42B dense model. Memory is what you end up paying for.

Read the license before you design anything. The Muse Model License requires a "Built with Muse Spark Open" notice, and anyone selling it as a model-as-a-service with more than $20M revenue in 12 months has to sign a separate agreement with Meta (model card). Internal use is exempt from that clause.

Sources: Llama 4 announcement, Llama 3.1 405B card and license, DeepSeek-V3, Kimi K2, Qwen3-235B-A22B. Llama 4 Behemoth (288B active, nearly 2T total) was announced as "still training" and has no public weights.

How much GPU memory and how many GPUs does an 816B-parameter model need?

Weight memory is parameters x bytes per parameter. For 816B that gives 1.63 TB at BF16, 816 GB at FP8, and about 408 GB at a pure 4-bit format. Meta's NVFP4 build lands at about 530 GB because only the experts are quantized (Meta developer blog).

Then add the rest of the bill. Meta's stated floor of 700 GB for NVFP4 and 2 TB for BF16 is 23-32% above the weight size, which matches the 20-40% overhead I budget for KV cache, activations, and CUDA graphs.

KV cache per token = 2 (K and V) x layers x KV heads x head dim x bytes. For Muse Spark Open at BF16: 2 x 62 x 16 x 64 x 2 = about 254 KB per token. One 128K-token conversation holds about 33 GB of KV cache, and ten concurrent ones hold about 333 GB. Those are upper bounds, because Meta says the model uses sliding-window attention, which caps the cache on the layers that use it.

Attention design changes this by an order of magnitude:

• Llama 3.1 405B (126 layers, 8 KV heads, GQA): about 516 KB per token, or 68 GB for one 128K sequence. Without GQA, with 128 KV heads, it would be 16x larger.
• DeepSeek-V3 (61 layers, MLA with a 512 + 64 compressed latent): about 70 KB per token, or 9 GB at 128K. DeepSeek reported that MLA cut KV cache by 93.3% versus its earlier dense model (DeepSeek-V2 paper).

Model configs: Llama 3.1 405B architecture, DeepSeek-V3 config. Use vLLM's PagedAttention so this memory actually gets used: the paper found earlier systems used only 20.4-38.2% of KV memory for real token state (vLLM paper).

Keep tensor parallelism inside a node and put pipeline or expert parallelism across nodes. NVLink gives each Hopper GPU 900 GB/s and each Blackwell GPU 1.8 TB/s (NVIDIA NVLink). An AWS p5en node gets 3,200 Gbps of EFA for all eight GPUs (AWS P5), which works out to 50 GB/s per GPU, 18x less than NVLink. Run tensor parallelism over that link and throughput collapses without any error message.

Quantization costs less quality than people fear. Red Hat ran over 500,000 evaluations on quantized Llama 3.1 models up to 405B and found every scheme kept over 99% of the BF16 baseline on the Open LLM Leaderboard v1 tasks, with INT4 W4A16 recovering 98.9% on HumanEval (Red Hat). Run your own evals on Muse Spark Open before you trust that number for it.

HBM per GPU: H100 80 GB, H200 141 GB, B200 180 GB, MI300X 192 GB, GB200 NVL72 13.4 TB. AWS us-east-1 Linux on-demand prices as of October 2026 come from the EC2 on-demand pricing page. Meta also says NVFP4 runs on four Blackwell Ultra GPUs.

Which cloud GPU instances can run an 800B model, and what does it cost per hour?

Four instance families hold Muse Spark Open on a single node: AWS p5en, GCP A3 Ultra, GCP A4, and Azure ND H200 v5, all 8xH200 or 8xB200. Scaleway's largest H100 instance needs several nodes. As of October 2026, the GPU cost per hour runs from about $42 to about $114 per node, so the reservation model decides your bill more than the inference engine does.

Sources: AWS P5, AWS Capacity Blocks pricing, GCP accelerator machines, GCP DWS pricing, Azure ND H200 v5, Azure retail prices API, Scaleway GPU pricing. Prices as of October 2026, Linux, before tax.

Plan capacity in weeks. EC2 Capacity Blocks can be booked up to eight weeks ahead for up to six months (AWS), and GCP sells 8-GPU shapes through Dynamic Workload Scheduler. Our GPU capacity guide covers where to look when your region is sold out.

Avoid spot for this model. Every reclaim means pulling 530 GB to 1.6 TB of weights back into HBM before you serve a token, and that takes minutes even from fast storage.

Break-even: divide the node price by your measured throughput. Say your benchmark shows 5,000 output tokens per second at full load on one 8xH200 node (a placeholder: measure your own):

At that throughput you break even at 83% utilization on AWS on demand and at 55% on GCP Flex-start. Double the throughput and the threshold halves. Also count egress, a shared filesystem (FSx for Lustre, Filestore, or Azure NetApp Files), and nights and weekends when the node sits idle. In ClearML's 2024 survey, only 7% of companies said their GPUs exceed 85% utilization at peak (ClearML).

What does the Kubernetes serving stack look like for a model this size?

The stack has five layers, all open source: the NVIDIA GPU Operator, vLLM or SGLang, LeaderWorkerSet when one replica spans several pods, DCGM exporter with Prometheus, and a shared read-only filesystem for the weights.

• GPU Operator installs the driver, container toolkit, device plugin, GPU feature discovery, and DCGM monitoring (NVIDIA docs).
• LeaderWorkerSet is "an API for deploying a group of pods as a unit of replication" and reached v1.0.0 in October 2026 (kubernetes-sigs/lws). A Deployment or StatefulSet cannot express "one model, two nodes, restart together", so rolling updates and scaling break without it. vLLM documents a 2-node, 16-GPU LWS example (vLLM LWS docs).
• Node pool: a dedicated, tainted GPU pool in one placement group or cluster network, plus the EFA or RDMA device plugin.
• Probes: a startupProbe with a long failure threshold. A default liveness probe kills the pod mid-load and you get CrashLoopBackOff forever.
• Metrics: HBM used, tokens per second, time to first token, inter-token latency, queue depth, KV cache usage.
• Weights: stage them once on a shared filesystem or node-local NVMe, or stream them. Run:ai Model Streamer loaded an 8B model from S3 into GPU memory in 4.88 seconds versus 37 seconds for Tensorizer (benchmarks), and vLLM supports it natively (vLLM docs).

Sources: vLLM parallelism docs, SGLang multi-node, Meta developer blog, NVIDIA Dynamo, llm-d, KServe.

vLLM vs SGLang multi-node: start with vLLM for its documentation and the LWS path. Move to SGLang when MoE throughput dominates your bill. LMSYS ran DeepSeek on 12 nodes of 8xH100 with prefill/decode disaggregation and large-scale expert parallelism and measured 22,282 output tokens per second per decode node (LMSYS).

What are the step-by-step deployment steps on your own Kubernetes cluster?

There are seven steps: GPU node pool, GPU Operator, weights on shared storage, engine as a LeaderWorkerSet, private endpoint, benchmark, auto-stop.

1. Node pool. 8xH200 or 8xB200 instances in one placement group, tainted, with EFA or InfiniBand enabled.
2. GPU Operator. Check that nodes advertise nvidia.com/gpu: 8 and the RDMA resource.
3. Weights. Accept the license on Hugging Face, download once, verify checksums, and store them on a filesystem inside your VPC.
4. Engine. Set tensor parallelism to GPUs per node and pipeline parallelism to node count, as vLLM recommends. Then tune gpu-memory-utilization and max-model-len, the two settings that actually control memory. Don't default to 1M context.
5. Endpoint. Expose /v1/chat/completions behind internal ingress with auth and rate limits. Never put it on a public LoadBalancer.
6. Benchmark. Use GuideLLM or vLLM's serving benchmark. Record tokens per second, TTFT, inter-token latency, and p99 under real concurrency.
7. Auto-stop. Schedule non-production GPU environments to stop at night before the first monthly bill arrives.

Three failures eat a week each. OOM during load comes from setting gpu-memory-utilization or the context length too high. A misconfigured NCCL or EFA setup across nodes gives a silent throughput collapse with no error in the logs. A liveness probe or Helm timeout kills the pod before the weights finish loading.

Should you self-host an 800B model at all, or just call a hosted API?

No, for most teams. Muse Spark Open is listed at $1.25 per million input tokens and $4.25 per million output tokens on Together AI, at the same list price through OpenRouter's Fireworks, Baseten, Modal, and Nebius endpoints, and with self-deploy options in SageMaker AI, Google Cloud Model Garden, and Microsoft Foundry (Together AI, OpenRouter, Meta). That ships this week.

Self-hosting is defensible for three reasons:

• Regulated data that contractually cannot leave your VPC, which matters for health, finance, and public-sector workloads under GDPR or the EU AI Act.
• Sustained utilization high enough to pass the break-even point above, measured on your own traffic.
• Customization an endpoint will not give you: your own LoRA stack, speculative decoding with a custom draft model, request routing, or modified attention.

Most teams land on a middle path: self-host a 30B-235B model that fits one node, and route the hardest queries to a frontier API. Meta's own Muse Glimmer 30B, distilled from Muse Spark (OpenRouter), and Qwen3-235B-A22B both fit that role. Quality per dollar often favors a well-served 235B MoE over a half-idle 816B.

This is not a Qovery pitch. If an API is right for you, use an API.

How does Qovery fit into self-hosting a large open model?

Qovery deploys and operates the serving stack inside your own AWS, GCP, Azure, or Scaleway account, or on your existing Kubernetes cluster. The weights, the VPC, and the GPU reservations stay yours, and your team deploys without a dedicated DevOps engineer. Qovery does not sell GPUs, does not host models, and is not an inference platform. vLLM or SGLang serves the model, and Qovery runs the cluster and the deployments around it.

• Your cloud account (BYOC). Capacity Blocks, DWS reservations, and committed-use discounts stay in your name, and that is the whole economic case for self-hosting.
• Kubernetes without managing it directly. Qovery-managed EKS clusters have a GPU node pool option, and you can attach an existing cluster on any provider, so the GPU Operator and LeaderWorkerSet slot in normally.
• Git-push deployments for the serving layer and the app in front of it, plus preview environments for the application tier.
• Access control per environment, so only the ML platform team touches the GPU environment while product teams deploy against its endpoint.
• Environment auto-stop. Deployment rules start and stop environments on a schedule. When a node bills $63-114 per hour and nobody works at 3am, this is the biggest saving you have.
• Managed cluster upgrades, because CUDA, driver, and Kubernetes version drift is how a working GPU cluster stops working.

For a smaller model on the same pattern, see our guide to running multiple open-weight models on one GPU cluster.

How do I set this up with Qovery from Claude Code or Cursor?

Install the Qovery Agent Skills with one curl command, then ask your coding agent in plain language to deploy the inference service into your own cloud account. The skills handle onboarding, deployment, troubleshooting, and cost optimization. The MCP Server operates what already exists and is read-only by default.

curl -fsSL https://skill.qovery.com/install.sh | bash

# Or scope the install to the current project
curl -fsSL https://skill.qovery.com/install.sh | bash -s -- --project

# Add the Qovery MCP Server to Claude Code (OAuth, read-only by default)
claude mcp add --transport http qovery https://mcp.qovery.com/mcp --callback-port 4242

# Write access, only when you want it (also enable it in the Qovery Console)
claude mcp add --transport http qovery "https://mcp.qovery.com/mcp?read_write=true" --callback-port 4242

The skills that matter here are qovery-onboard to connect your cloud account, qovery-deploy to ship, qovery-troubleshoot for failing pods, qovery-optimize and qovery-speedup for cost and build time, qovery-preview for preview environments, and qovery-terraform for infrastructure as code. Then ask: "deploy this vLLM service to my AWS account with Qovery and schedule it on the GPU node pool."

Details: AI agent quickstart and Claude Code integration.