Hardware · Clusters

Run bigger AI models on the devices you already own

Join a Mac, a gaming PC, a mini PC or a Raspberry Pi into one cluster, and a model too big for any of them can run across all of them. Free tools such as EXO and llama.cpp RPC do the splitting. Here is what that gets you, what it doesn’t, and what your own devices could run together.

Build your cluster Compare the tools Read the guide

How a model is split

Layers in turn: more memory, not faster

Pipeline split. Each device holds some layers; a token passes through them in turn. You get their memory together, and typing is no faster than the devices’ average. Most tools work this way.

Every layer shared: faster, needs a fast link

Tensor split. Every device holds a slice of every layer, and all of them work on each token at once, swapping results twice per layer. Typing gets faster, but only over a very fast link such as Thunderbolt 5 with RDMA between Macs.

What gets faster, and what doesn’t

One Mac Studio6.42 s

Reads the prompt in 5.57 s, then types the answer in 0.85 s.

Spark reads, Mac types2.32 s

The DGX Spark reads the prompt in 1.47 s; the Mac types in the same 0.85 s. 2.8× sooner

8,192-token prompt, 32-token answer (EXO, Llama 3.1 8B): the typing takes the same 0.85 s in both, so all of the gain, 6.42 s down to 2.32 s, is in the wait for the first word.

Memory adds up. Models that fit nowhere else can run, usually at a slower speed than a single big machine would give.
Typing gets faster only with a tensor split on a fast link. Four Mac Studios typed a dense 123B model 2.9 times faster than one in Apple’s test; a mixture-of-experts model gained 1.6 times.
The wait for the first word can shrink when a high-compute device reads the prompt. EXO’s demo cut 6.4 s to 2.3 s, all of it in that wait, for a long prompt and a two-sentence answer.
The slowest device sets the pace. A Raspberry Pi holding part of a model makes every word wait for it: leave slow devices out unless the model needs their memory.
Wi-Fi works, but slowly. Use Ethernet or Thunderbolt cables.

Example clusters

More examples

Or add your own devices and see what they run together.

Models only a cluster runs

The best of the models that no single device in an example cluster can run, with the fastest example cluster for each. These are our estimates for the best case: the one published run of Kimi K3 on four Mac Studios (at 4-bit) typed about 2 tokens per second.

Model	Size	Cluster	Quant	Speed	Runs	Tool
Kimi K3.1 Pro (2.8T)Moonshot AI	2,780B (104.0B active)	Four Mac Studios (M3 Ultra, 512 GB) with RDMA	`Q3_K_M`	31.7 tok/s	Runs well	EXO
Qwen3.8 Max (2.4T-A95B)Alibaba	2,446B (95.0B active)	Four Mac Studios (M3 Ultra, 512 GB) with RDMA	`Q3_K_M`	34.1 tok/s	Runs well	EXO
DeepSeek V4.1 Flash 1017 (552B-A16B)DeepSeek	763B (16.0B active)	Two Mac Studios (M3 Ultra, 256 GB)	`Q2_K`	55.7 tok/s	Runs well	llama.cpp RPC
GLM-5.3 (753B-A40B)Zhipu AI	753B (40.0B active)	Two Mac Studios (M3 Ultra, 256 GB)	`Q2_K`	28.7 tok/s	Runs well	llama.cpp RPC
Kimi K3 (2.8T)Moonshot AI	2,780B (104.0B active)	Four Mac Studios (M3 Ultra, 512 GB) with RDMA	`Q3_K_M`	31.9 tok/s	Runs well	EXO
Ling 3.1 Flash 1019 (560B-A25B)Ant Group (inclusionAI)	572B (25.0B active)	Two Mac Studios (M3 Ultra, 256 GB)	`Q4_K_M`	50.7 tok/s	Runs well	EXO
MiniMax M3.1 Flash (428B-A23B)MiniMax	429B (23.0B active)	Two DGX Sparks	`Q2_K`	30.6 tok/s	Runs great	llama.cpp RPC
Step 5 Preview (600B-A27B)StepFun	604B (27.0B active)	Two Mac Studios (M3 Ultra, 256 GB)	`Q3_K_M`	52.1 tok/s	Runs well	EXO
GLM-5.3 Flash (320B-A18B)Zhipu AI	321B (18.0B active)	Two DGX Sparks	`Q4_K_M`	30.9 tok/s	Runs well	llama.cpp RPC
MiMo V2.6 Flash (310B-A15B)Xiaomi	311B (15.0B active)	Two DGX Sparks	`Q4_K_M`	36.1 tok/s	Runs well	llama.cpp RPC
Qwen3.8 Flash Next (125B-A6B)Alibaba	180B (6.0B active)	Two Mac minis (M4 Pro, 64 GB)	`Q2_K`	40.3 tok/s	Runs great	llama.cpp RPC
GLM-5.2 (753B-A40B)Zhipu AI	753B (40.0B active)	Two Mac Studios (M3 Ultra, 256 GB)	`Q2_K`	28.7 tok/s	Runs well	llama.cpp RPC
Muse Spark Open (816B-A42B)Meta	822B (42.0B active)	Two Mac Studios (M3 Ultra, 256 GB)	`Q2_K`	27.6 tok/s	Runs well	llama.cpp RPC
Hunyuan Hy4 Preview (780B)Tencent	780B (55.0B active)	Two Mac Studios (M3 Ultra, 256 GB)	`Q2_K`	22.0 tok/s	Runs well	llama.cpp RPC
DeepSeek V4 Pro (1.6T-A49B)DeepSeek	1,651B (49.0B active)	Four Mac Studios (M3 Ultra, 512 GB) with RDMA	`Q4_K_M`	33.8 tok/s	Runs great	EXO

The tools

EXOActive · v1.0.71, Apr 2026Two or more Apple silicon Macs, best of all Mac Studios or Mac minis joined by Thunderbolt 5 cables, when you want big models with almost no setup.
- Mac: Yes
- NVIDIA: Partly
- Linux CPU: Partly
llama.cpp RPCActive · v0.9.0, Nov 2026Mixed devices you already own, a Mac, a gaming PC, a mini PC or a Jetson, when one GGUF model won't fit on any of them alone.
- Mac: Yes
- NVIDIA: Yes
- AMD: Yes
- Linux CPU: Yes
- Pi: Yes
- Windows: Yes
- Android: Partly
MLX distributedActive · mlx v0.32.4, mlx-lm v0.32.2, Oct 2026Mac owners comfortable with a terminal who want the most direct, lowest-overhead way to run one model on several Macs, especially Mac Studios on Thunderbolt 5.
- Mac: Yes
- NVIDIA: Partly
- Linux CPU: Partly
Mesh LLMActive · v0.78.1, Oct 2026A household or office with mixed Macs and NVIDIA or AMD PCs that wants one shared local endpoint, and now and then a model too big for any one machine.
- Mac: Yes
- NVIDIA: Yes
- AMD: Yes
- Linux CPU: Yes
- Pi: Partly
- Windows: Yes
Distributed LlamaMaintained · v0.16.5, Feb 2026Identical small machines on a wired switch, such as four Raspberry Pi 5s or a few Mac minis, where adding boards really speeds up typing.
- Mac: Partly
- NVIDIA: Partly
- AMD: Partly
- Linux CPU: Yes
- Pi: Yes
- Windows: Yes
prima.cppExperimental · no releasesGetting a 30B-70B model running at all on a few ordinary laptops, desktops and Android phones whose memory together is still too small.
- Mac: Yes
- NVIDIA: Yes
- AMD: Partly
- Linux CPU: Yes
- Pi: Partly
- Android: Partly
PooledExperimental · v2.0.0, Oct 2026Trying clustering in a few minutes with what's at hand, including an iPhone, for one of the handful of models it supports.
- Mac: Yes
- NVIDIA: Yes
- AMD: Partly
- Windows: Yes
- Android: Partly
- iPhone: Yes
ParallaxQuiet · v0.1.2, Dec 2025A mix of NVIDIA PCs and Macs, possibly on different networks, serving one of the models on its list.
- Mac: Yes
- NVIDIA: Yes
- Windows: Partly
LocalAI (P2P workers)Active · v4.12.1, Oct 2026People already running LocalAI who want workers to join with a token instead of typing addresses.
- Mac: Partly
- NVIDIA: Yes
- AMD: Yes
- Linux CPU: Yes
- Pi: Yes
- Windows: Partly
SwarmLLMExperimental · v0.3.276-alpha, Nov 2026Sharing GPUs with friends or volunteers over the internet.
- Mac: Yes
- NVIDIA: Yes
- Linux CPU: Yes
- Windows: Yes

Every tool compared, including server tools (vLLM, SGLang, GPUStack), quiet projects (Petals, Cake), and why Ollama can’t cluster.

The fine print

Frontier models still need a data center. A 2.8-trillion-parameter model like Kimi K3 ships in native 4-bit and needs about 4.8 TB of memory as released, more than twice the 2 TB four 512 GB Mac Studios hold. Quantizing it again to make it fit stacks one lossy compression on another.
EXO’s DGX Spark + Mac demo isn’t in the public app. Macs and Linux machines can’t join one EXO cluster today, and the 2.8× it reported came from an 8K-token prompt with a 32-token answer.
Our speeds are estimates for one person chatting, worked out from each device’s memory speed and compute, the connection, and costs fitted to published cluster runs. Check them against the measured speeds on each tool’s page.

Sources: The Stack, “EXO: Can Frontier AI Models Now Run Locally?”; EXO’s README and blog; Apple’s MLX benchmarks; each tool’s repository.

Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.