Reflection AI's Beam Is 501B Parameters and Apache 2.0. What Would It Take to Self-Host in Your Own Cloud Account?

Key Points:

• A 501B-parameter model does not fit on one GPU. Weights alone need roughly 1 TB at BF16, about 500 GB at FP8/8-bit, and about 250 GB at 4-bit, before KV cache and activations. The practical floor is one 8-GPU node of 141 GB-class accelerators (H200), or two 8xH100 80 GB nodes with tensor plus pipeline parallelism.
• Beam is a sparse Mixture-of-Experts model: 501B total, 23B active per token (Reflection AI). That cuts compute per token, not memory. You still hold every expert resident, so the memory math is driven by the full 501B, not the 23B.
• Apache 2.0 is the part that unblocks procurement. You can self-host, fine-tune, and run Beam commercially with no per-token fee, no monthly-active-user threshold, and no acceptable-use rider, unlike the Llama Community License's 700M MAU cap (llama.com). Your entire cost moves into GPU hours.
• Self-hosting only beats a hosted API above sustained, high-utilization traffic. An 8-GPU H100/H200-class node bills tens of dollars per hour whether it serves one request or ten thousand, so break-even = (node cost per hour) / (tokens served per hour) compared against the API's per-million-token price.
• The hard part is the platform, not the model: GPU node pools, driver and device-plugin lifecycle, multi-node RDMA networking, 500 GB to 1 TB weight caching, autoscaling on queue depth, and cluster upgrades. Qovery provisions and operates a Kubernetes cluster inside your own AWS, GCP, Azure, or Scaleway account, or plugs into your existing cluster, with git-push deploys, per-PR preview environments, environment auto-stop, managed cluster upgrades, and per-environment RBAC.

A reader asked me last week how hard it would be to pull a frontier open-weight model into their own AWS account and serve it, now that the licenses are finally sane. Reflection AI announcing Beam made that question concrete. So I did the sizing the way I would for our own infrastructure, and this is the write-up: the memory math, the node shapes on every major cloud, the serving stack, the real cost comparison, and the part everybody underestimates, which is the platform underneath.

One caution before any numbers. Beam was announced on October 5, 2026, and as I write this the weights are not downloadable yet. Reflection says the weights, technical report, and model card arrive later in October under Apache 2.0 (Reflection AI). So I verified the architecture facts against Reflection's own announcement, and where a spec is not public yet (released checkpoint precisions, exact layer counts) I give the sizing parametrically so you can re-derive it from the model card the day it lands.

What exactly did Reflection AI release with Beam, and what does Apache 2.0 let you do?

Reflection AI announced Beam as a 501-billion-parameter open-weight model with 23 billion active parameters, a sparse Mixture-of-Experts architecture, to be released under the Apache 2.0 license (Reflection AI). Apache 2.0 permits commercial use, modification, redistribution, and self-hosting with no per-token fee and no field-of-use restriction (apache.org). Open weight is not the same as open source: you get the weights, not the training data or the training code.

Here is what Reflection has stated directly. Beam is a sparse MoE with 501B total and 23B active parameters. It was pretrained to a 256K-token context during reinforcement learning and extended toward a 1M-token effective context in midtraining. Reflection reports it as competitive with larger open models like GLM 5.2 on coding and agentic tasks while using 3 to 4 times less inference compute on reasoning benchmarks (Reflection AI). The RL run used 10,500 NVIDIA GB300 GPUs over four weeks. Released checkpoint precisions are not published yet, so confirm BF16 versus FP8 versus quantized on the model card before you size hardware.

The MoE detail matters more than any benchmark. Twenty-three billion active parameters means each token only routes through a fraction of the network, so your compute per token is far below what a dense 501B model would cost. Your memory bill does not get that discount. Every expert has to be resident in VRAM because you never know which expert the next token needs. Size for 501B, budget compute for 23B.

Apache 2.0 is the reason a model this size is interesting to an enterprise rather than just to a lab. Compare the alternatives. The Llama Community License caps free commercial use at 700 million monthly active users and then requires a separate license from Meta "in its sole discretion," aggregated across your whole corporate group (llama.com). Google moved Gemma onto Apache 2.0, removing the earlier custom terms that let the licensor modify or terminate downstream grants (Google Gemma terms). Apache 2.0 removes the legal review that usually stalls self-hosting for a quarter. What open weights still do not give you: the training data, the training code, and reproducibility. You own the inference, not the provenance.

Check the exact license and parameter count on each model's own card before you commit. License terms and checkpoints change, and the MoE active-parameter figures move with each release.

How much GPU memory and how many GPUs does a 501-billion-parameter model actually need?

Weight memory is parameters times bytes per parameter. So 501B needs about 1 TB at BF16 (2 bytes), about 500 GB at FP8 or INT8 (1 byte), and about 250 GB at 4-bit (0.5 byte), before KV cache, activations, and allocator fragmentation, which realistically add 20 to 40 percent depending on context length and concurrency. That puts the floor at one 8xH200 141 GB node at FP8, or two 8xH100 80 GB nodes at FP8 with tensor plus pipeline parallelism.

Here is the arithmetic in plain text so you can re-run it when the model card drops:

• 501e9 params x 2 bytes = ~1.00 TB at BF16
• 501e9 params x 1 byte = ~501 GB at FP8 / INT8
• 501e9 params x 0.5 byte = ~250 GB at INT4

Usable VRAM is always below the spec sheet. Once the runtime, CUDA graphs, and the KV cache pool are allocated, plan on losing a chunk of every card. An 8xH100 node is 640 GB of raw VRAM, an 8xH200 node is 1,128 GB, and an 8xMI300X node is 1,536 GB (NVIDIA H200, AMD Instinct). Those headline numbers shrink fast under load.

The weights are the fixed cost. What blows up a production deployment is the KV cache, which scales with context and concurrency:

kv_cache_bytes = 2 x layers x kv_heads x head_dim x bytes x seq_len x batch_size

At a 1M-token effective context, the KV cache per request can rival or exceed the weights. The mitigations are grouped-query or multi-head latent attention to shrink kv_heads, FP8 KV cache to halve bytes, PagedAttention to stop fragmentation waste, and prefix caching to reuse shared prompt prefixes. Long context and high concurrency, not the weights, are what force you from one node to two.

Translating to real hardware: the NVIDIA H100 carries 80 GB at 3.35 TB/s, the H200 carries 141 GB at 4.8 TB/s (NVIDIA), the AMD MI300X carries 192 GB at 5.3 TB/s and the MI325X carries 256 GB at 6.0 TB/s (AMD). More VRAM per card means fewer cards and fewer network hops, which is the whole game at this size.

Parallelism is where deployments go wrong. Tensor parallelism splits each layer across GPUs and is bandwidth-hungry, so you want it inside one NVLink domain (8 GPUs in one node). Pipeline parallelism splits layers across nodes and tolerates slower links. Expert parallelism spreads MoE experts across GPUs. Staying inside a single 8-GPU NVLink node is dramatically faster than crossing the network, which is why one 8xH200 node beats two 8xH100 nodes for the same model even when total VRAM is similar.

If you do go multi-node, the interconnect sets your ceiling. AWS exposes Elastic Fabric Adapter on p5-class instances, Azure runs InfiniBand NDR on its ND-series, and GCP offers GPUDirect-TCPX and RDMA on A3 Ultra and A4 (AWS EFA, GCP GPUs). A multi-node replica over a slow link can lose most of the throughput you paid for.

On quantization, FP8 is close to lossless on most frontier models today, while INT4 (AWQ or GPTQ) buys memory at a measurable quality cost. Benchmark on your own evals, not a public leaderboard. And do not forget cold start: moving 500 GB to 1 TB of weights from object storage onto a node takes minutes, so cache weights on local NVMe or a shared filesystem, because naive autoscaling that cold-pulls the checkpoint on every scale-up will time out.

GPU counts assume weights plus a working KV cache and runtime overhead, not just the raw weight figure. Long context and high concurrency push every row up a tier.

Which cloud GPU instances can run Beam, and what do they cost per hour on AWS, GCP, Azure, and Scaleway?

The realistic single-node options are AWS p5/p5e/p5en (8xH100 or H200), GCP A3 Ultra and A4 (8xH200 or B200), Azure ND H100 v5 and ND H200 v5, Scaleway H100 instances, and Azure ND MI300X v5 on the AMD side. On-demand list prices for 8-GPU H100-class nodes sit in the tens of dollars per hour, so commitment discounts and utilization, not the sticker price, decide the economics.

Treat every number below as a moving target and confirm it on the provider's own page before you budget. As published rates, GCP lists the a3-ultragpu-8g (8xH200) and a3-highgpu-8g (8xH100) on its accelerator-optimized pricing page (GCP), Azure lists ND H100 v5 and ND H200 v5 on its Linux VM pricing page (Azure), AWS publishes p5 and p6 rates on its on-demand and Capacity Blocks pages (AWS on-demand, AWS Capacity Blocks), and Scaleway lists H100 PCIe and H100 SXM on its GPU pricing page (Scaleway). One structural note on AWS: on-demand H200 is scarce, and most H200 capacity is sold through Capacity Blocks for ML (1-day or 14-day reservations), which is a scheduling item, not a checkbox.

The discount ladder is where the real money is. On-demand is the worst rate you will ever pay. One-year and three-year commitments cut it hard through AWS Savings Plans and Capacity Blocks, GCP Committed Use Discounts and Dynamic Workload Scheduler, and Azure Reservations (AWS Savings Plans, GCP CUDs, Azure Reservations). Spot and preemptible are cheaper still, with one caveat that kills them for this workload: a multi-node inference replica does not survive preemption gracefully, so spot is fine for batch jobs and wrong for a live endpoint.

Capacity is its own constraint. H200 and B200 are quota-gated and region-limited. Quota approval and reservation lead time are weeks, not minutes, so a serious plan starts the quota request before the model card is even out.

Neoclouds (CoreWeave, Lambda, Nebius, RunPod, Together) often price GPU-hours below hyperscaler list, and if raw GPU price is the only thing that matters, they usually win. The tradeoff is that your weights, prompts, and responses leave your primary cloud account and VPC, which may be the exact thing you were trying to avoid by self-hosting in the first place. And watch the lines people forget: repeatedly pulling a 1 TB checkpoint, cross-AZ traffic between multi-node replicas, object storage for weights and logs, and egress.

The BYOC argument is concrete. When the GPU nodes run in your own account, the spend lands on your existing bill under your committed-use discounts, Savings Plans, or enterprise agreement, with no third-party inference markup layered on top.

Prices move weekly and vary by region and reservation term. The provider's own pricing page is the only authority; use this table to compare shapes, not to lock a budget.

What serving stack should you run, and how do you deploy it on Kubernetes?

For a 501B open-weight model the production-grade choices are vLLM, SGLang, or NVIDIA TensorRT-LLM behind Triton, orchestrated on Kubernetes with the NVIDIA GPU Operator for drivers and the device plugin, plus LeaderWorkerSet or Ray Serve when a single replica spans more than one node. Ollama and llama.cpp are great on a laptop and wrong for this.

The three serious engines, honestly compared. vLLM is Apache 2.0, uses PagedAttention, and has the widest and fastest-moving model coverage, which makes it the safe default for a brand-new model (vLLM docs). SGLang is Apache 2.0, uses RadixAttention, and is strong on structured output and high concurrency (SGLang). TensorRT-LLM delivers the highest tuned throughput on NVIDIA hardware but is NVIDIA-only and carries the longest engine-build and ops cycle (TensorRT-LLM). Pick vLLM if you want day-one model support and the least friction. Pick SGLang if concurrency and structured decoding dominate. Pick TensorRT-LLM only if you have the engineers to tune it and you need every last token per second. Do not start with TensorRT-LLM on a model whose checkpoint shape you have not even seen yet.

Multi-node serving is the part a plain Kubernetes Deployment cannot express. One logical replica spread across two nodes is not two pods, it is one coordinated unit. The LeaderWorkerSet API models exactly that, and Ray Serve is the other common pattern (LeaderWorkerSet). If Beam needs two nodes at your chosen precision, you need one of these, not an HPA pointed at a Deployment.

The GPU plumbing checklist is non-negotiable: the NVIDIA GPU Operator or device plugin for drivers and CUDA, pinned driver and CUDA versions, node taints and tolerations so only GPU workloads land on GPU nodes, topology-aware scheduling, RDMA or EFA device exposure for multi-node, and hugepages (NVIDIA GPU Operator). Miss one and the pod either will not schedule or will run at a fraction of the hardware's capability.

Autoscaling for LLMs does not look like web autoscaling. Scale on queue depth or time-to-first-token, not CPU, using KEDA or an HPA wired to custom metrics. And respect the cold start: a multi-hundred-gigabyte weight load means a new replica is minutes away from serving, so keep warm headroom and pre-stage weights.

The rest of day-one: track tokens per second, time to first token, inter-token latency, GPU utilization, and memory headroom, and run a regression eval suite on every model or engine upgrade. Put an OpenAI-compatible gateway in front with per-team API keys, rate limits, and prompt/response logging with PII controls; the Kubernetes Gateway API Inference Extension is the emerging standard for this. Then accept the day-two reality: Kubernetes ships about three minor releases a year and supports each for roughly 14 months (kubernetes.io), GPU driver upgrades drain nodes, and inference engines ship breaking changes monthly. Somebody owns that calendar.

Verify current multi-node and quantization support on each project's own docs; these engines change fast and a new model often needs the latest release.

Is self-hosting Beam cheaper than calling a hosted API, and at what token volume does it flip?

Self-hosting is a fixed hourly cost and an API is a variable per-token cost, so break-even is (GPU node cost per hour x hours) divided by (tokens served per hour), compared against the API's per-million-token price. Below steady, high-utilization traffic the hosted endpoint wins outright, because an idle 8-GPU node bills exactly the same as a saturated one.

The formula, explicitly:

self_hosted_cost_per_1M_tokens = (node_$ per_hour) / (tokens_per_hour / 1e6)

Work one example with published numbers. Say an 8xH200 node and a comparable MoE on a hosted API at roughly $1.25 per 1M tokens combined, the kind of rate Together lists for DeepSeek-V3 (Together AI). If your node costs on the order of $40 to $90 per hour depending on commitment, and a well-tuned vLLM deployment of a 23B-active MoE sustains, say, a few thousand tokens per second, then it serves on the order of 10 million-plus tokens per hour. Divide and you land near or below the per-token API price only when the node is busy. At 10 percent utilization that same node costs roughly 10 times as much per token. Plug in your own measured throughput and your own quoted node price; do not trust my round numbers, trust the arithmetic.

Anchor the per-token side with real list prices from Together AI, Fireworks AI, Groq, Amazon Bedrock, Google Vertex AI, and Azure AI Foundry for comparable open-weight models (Together, Fireworks, Bedrock, Vertex AI). For context, Fireworks buckets generic MoE models from about $0.50 to $1.20 per 1M tokens by size (Fireworks). Those are the numbers your self-hosted cost has to beat after you account for utilization.

Utilization is the entire argument. A node at 10 percent utilization costs roughly 10 times per token what the same node costs at 90 percent. Cast AI's 2026 report found average GPU utilization across analyzed Kubernetes clusters at about 5 percent, which is roughly 20 times more capacity than workloads consumed (Cast AI). Continuous batching, prefix caching, and request queuing are cost levers before they are performance features, because they are what move you from 5 percent to something defensible.

Then the costs nobody puts in the spreadsheet: platform engineering time to build and maintain the serving stack, on-call, GPU quota wrangling, eval maintenance, and checkpoint storage and transfer. And the quiet leak, which is non-production. Dev and staging GPU environments left running overnight and on weekends burn the most expensive idle resource in the company. Auto-stopping non-production environments is the single fastest saving after launch.

Some reasons override the math entirely: data residency, regulated data that cannot leave your VPC, fine-tuning on proprietary data, and pinning a version nobody can deprecate out from under you. If any of those apply, self-hosting wins even when the per-token number does not.

My decision rule: prototype on a hosted API, measure real sustained tokens per second for two weeks, then self-host only if projected utilization clears break-even with margin. Most teams should start on an API and move to self-hosting once volume and data requirements justify it, not before.

The crossover point is yours to measure, not mine to guess. Build the two-week utilization data first.

How do you run GPU inference in your own cloud account without building a platform team first?

The model is the easy part. The platform underneath it, GPU node pools, drivers, multi-node networking, autoscaling, secrets, RBAC, and cluster upgrades, is what turns a two-week project into a two-quarter one. Qovery provisions and operates a Kubernetes cluster inside your own AWS, GCP, Azure, or Scaleway account, or plugs into your existing Kubernetes cluster, so your team deploys the vLLM or SGLang service instead of the stack beneath it.

Look at what the platform layer actually has to cover for GPU inference: GPU node pools with taints and tolerations, driver and device-plugin lifecycle, storage for multi-hundred-gigabyte weights, network policy, per-environment secrets, cost visibility, and rolling cluster upgrades on Kubernetes' three-releases-a-year clock. None of that is the model, and all of it is required before the model serves a single token.

Here is what Qovery does, stated only as the capabilities I can verify: it runs on AWS, GCP, Azure, and Scaleway in your own account, or on a Kubernetes cluster you already operate (self-managed, on-prem, any distribution). It gives you git-push deployments, preview and ephemeral environments per pull request, environment auto-stop for non-production, managed cluster upgrades, per-environment RBAC, and databases backed by managed cloud services. The cloud bill and any committed-use discounts or Savings Plans stay in your own name. I am not going to claim GPU-specific magic that is not documented; what I will say is that these are the exact Kubernetes and day-two capabilities a GPU inference deployment needs, and they are the parts teams usually hand-build.

BYOC matters specifically for GPU spend. Your weights, prompts, and responses stay inside your account and VPC, and the GPU bill plus any discounts stay under your contracts rather than a vendor's markup. Auto-stop and preview environments pay off most here precisely because GPU environments are the most expensive idle resource you own, and per-PR environments let teams test prompt and engine changes without standing up a permanent second cluster.

Be fair about the alternatives, because even-handed honesty is the only thing that makes this section worth reading. Fully managed inference (Bedrock, Vertex AI, Azure AI Foundry, Together, Fireworks) wins when the data does not need to stay in your account. Kubernetes-native ML stacks (KServe, Ray Serve, KubeAI, NVIDIA NIM) win when you already have a platform team who wants to own every layer. Neoclouds win when raw GPU price dominates every other concern. And Qovery is the wrong choice if you need bare-metal, custom-kernel, or exotic interconnect tuning below the Kubernetes layer, or if you already run a mature internal platform with GPU scheduling solved.

Match the row to your real constraint: data residency, speed to production, or raw price. There is no single right answer, only the one that fits your data and your team.

How many GPUs do you need to run a 501-billion-parameter model like Reflection AI's Beam?

At FP8, roughly 500 GB of weights plus KV cache fit on one 8-GPU node of 141 GB-class accelerators (8xH200, 1,128 GB) or one 8xMI300X node (1,536 GB). At BF16, plan for about 1 TB of weights, which usually means two 8xH100 80 GB nodes with tensor plus pipeline parallelism. At INT4, about 250 GB, you can drop to roughly 4 GPUs. Confirm Beam's released precisions on the model card before committing.

Can you self-host Beam commercially under the Apache 2.0 license?

Yes. Reflection AI states Beam's weights will be released under Apache 2.0 (Reflection AI), which permits commercial use, modification, and redistribution with no per-token fee, no user-count threshold, and no field-of-use restriction (apache.org). You keep attribution and the included notices. That is a different world from the Llama Community License's 700M MAU cap (llama.com).

What is the cheapest cloud instance that can serve a 501B-parameter open-weight model?

At FP8 a single 8-GPU node is the floor, so the cheapest option is whichever 8xH200-class or 8xMI300X node you can get at the best committed rate in a region with quota. Published list prices sit in the tens of dollars per hour; GCP, Azure, AWS, and Scaleway each post their own rates (GCP, Azure, AWS, Scaleway). Neoclouds often beat hyperscaler list price if the data can leave your account.

Should you use vLLM, SGLang, or TensorRT-LLM to serve Beam?

Start with vLLM for a brand-new model: Apache 2.0, PagedAttention, and the fastest model support (vLLM). Choose SGLang if high concurrency or structured output dominates (SGLang). Choose TensorRT-LLM only when you need maximum tuned throughput on NVIDIA hardware and have the engineering time for its build cycle (TensorRT-LLM).

Is self-hosting an open-weight model cheaper than using an inference API?

Only above sustained, high-utilization traffic. Break-even is (node cost per hour) divided by (tokens served per hour), compared to the API's per-million-token price. An idle node bills the same as a busy one, and real-world GPU utilization averages about 5 percent (Cast AI), so most teams should prototype on an API and self-host only once measured volume clears break-even.

Do you need a platform team to run GPU inference on Kubernetes in your own cloud account?

You need the platform work done, by a team or by a tool. GPU node pools, driver lifecycle, multi-node networking, autoscaling, RBAC, and cluster upgrades are required regardless. A managed platform like Qovery provisions and operates the cluster inside your own account so your engineers deploy the inference service rather than building and maintaining the stack beneath it.

Beam is a genuinely exciting release, and Apache 2.0 weights at this scale change what a self-hosted strategy can look like. The model is the easy decision. The infrastructure is where the quarters go, and that is the part Qovery takes off your plate: deploy your vLLM or SGLang service with a git push inside your own AWS, GCP, Azure, or Scaleway account, get a preview environment on every pull request, and auto-stop non-production GPU environments so they stop burning money overnight. Try Qovery free.