The first post on this fleet's inference stack dropped a number without fully explaining it: a 30B-class MoE model running at roughly 40 tokens/second, next to a 27B dense model limping along at 4.4. Same card, same quantization philosophy, almost a 9x gap. That's not a rounding difference — it's two architectures hitting completely different bottlenecks on the same hardware, and the "why" is worth walking through slowly, because it changes how you should pick a model for a memory-constrained card.
the number everyone gets wrong: parameter count isn't the cost
A 30B MoE model and a 27B dense model sound like they should perform similarly — they're within 10% of each other in total parameters. They don't perform similarly, because total parameter count is the wrong number to look at for inference speed. The number that matters is active parameters per token — how much of the model actually has to move through compute to produce the next token.
A dense model activates all of its parameters on every single token. A 27B dense model does roughly 27B parameters worth of matrix multiplication for every token it generates, no exceptions. A mixture-of-experts model routes each token through only a handful of its experts — typically 2 out of dozens available. The 30B MoE on this fleet activates something in the neighborhood of 3-4B parameters per token, even though the full model, experts included, adds up to 30B.
// the number that actually predicts speed 27B active parameters per token (dense) vs. ~3-4B active parameters per token (MoE), despite the MoE model having more total parameters. Active parameters, not total parameters, is what the GPU has to compute for every single token — and that's the number that maps directly to tokens/second.
why this matters more on a budget card than a data-center one
On a card with abundant compute and bandwidth, the gap between "compute 4B parameters" and "compute 27B parameters" per token still matters, but it's proportionally smaller — a fast enough card can brute-force the dense model's cost. The Arc Pro B60 is not that card. It's a ~$500 consumer/workstation part with a fixed, modest amount of compute and memory bandwidth, and every cycle spent multiplying dense-model weights that a MoE model would have skipped is a cycle that shows up directly in tokens/second.
This is the part that's easy to miss if you're benchmarking on borrowed data-center intuition: MoE's advantage isn't a nice-to-have, it's the difference between usable and not on hardware this size. Vulkan compute on the B60 is throughput-bound in a way that punishes wasted matrix multiplies harder than a card with bandwidth to spare.
VRAM tells the same story from a different angle
Total parameters still cost VRAM — a 30B MoE model has to keep all of its experts resident even though it only uses a few per token, because it doesn't know in advance which experts the next token will need. So the MoE model isn't cheaper on memory; quantized at IQ4_XS, it occupies roughly what you'd expect for a 30B-parameter model on disk and in VRAM.
What changes is what happens after the weights are loaded. Loading 30B quantized parameters into VRAM is a one-time cost paid at model load. Computing against them is the recurring cost paid on every single token, forever, for the life of the session — and that's exactly the number MoE routing cuts by routing around 85-90% of the model on every step.
// the tradeoff, stated plainly MoE costs you VRAM you have to pay once (all experts resident) to save compute you'd otherwise pay every token (only a few experts active). On a card with more VRAM than raw compute — which describes a lot of budget and workstation-class hardware, the B60 included — that trade is close to free money.
the practical takeaway for anyone picking a model on similar hardware
If you're choosing between a dense and MoE model of similar total size for a card in this class, the MoE model is very likely to win on throughput, sometimes by a wide margin, as long as it fits in VRAM at your target quantization. Total parameter count is a marketing number. Active parameters per token is the engineering number, and it's the one that actually predicts what happens when you hit "generate."
The 9x gap on this fleet isn't an outlier or a benchmarking fluke — it's the mechanism working exactly as designed, on hardware that makes the mechanism's advantage impossible to ignore.