In a dense model every parameter takes part in every token. A mixture-of-experts (MoE) model splits much of the network into many “experts” and a router that picks a few of them for each token. Names like 30B-A3B mean 30 billion parameters in total and about 3 billion active per token.
What you gain
Speed follows the weights read per token (bandwidth). Only the active experts are read, so an MoE model can generate nearly as fast as a dense model the size of its active part, while drawing on the knowledge of its much larger total. That is why a 30B-A3B model can feel like a small model and answer like a bigger one.
What it still costs
- Memory for all of it. Every expert must be stored somewhere, so a 30B MoE needs roughly the memory of a 30B dense model. MoE saves speed, not memory.
- Mixed quality comparisons. An MoE model is usually better than a dense model of its active size and a little behind a dense model of its total size. Check the grade on the model page rather than guessing.
When it does not fit: expert offload
llama.cpp can keep the parts used on every token (attention and shared layers) on the GPU and move the routed experts to system RAM. Because only a few experts are touched per token, this hurts far less than offloading layers of a dense model, and it lets a modest card run a large MoE model. In llama.cpp the relevant options are --cpu-moe (all experts on the CPU) and --n-cpu-moe N (experts for the first N layers on the CPU), used together with a high -ngl. Exact behavior varies by version, so check the current llama.cpp documentation. Apps built on it may expose the same idea as a setting.
Our estimates account for MoE: the model pages show total and active parameters, and the speed estimate reads only the active share. Find them in the LLM list, where the parameters column shows, for example, “30B (3B active)”.