Hardware · Clusters · Tools

GPUStack

by GPUStack · Apache-2.0 · 5,984 GitHub stars · Active · v2.3.0, Oct 2026

A manager for a team's Linux GPU servers that deploys vLLM or SGLang and can run one model across several of them.

Mac: No
NVIDIA PC: Yes
AMD PC: Yes
Linux, processor only: No
Raspberry Pi and boards: No
Windows: Partly
Android: No
iPhone, iPad: No

Go to GPUStack Source code

At a glance

Best for: Teams with several Linux GPU servers who want a managed, shared model service.
Runs on: Since v2.0 workers are Linux with Docker only (Macs, native Windows and the llama.cpp backend were dropped).
How it splits a model: Through vLLM or SGLang: tensor and pipeline (experimental across workers).
Connections: Workers just need to reach each other.
How devices join: Workers join the server with a token.
Models: Hugging Face models through vLLM or SGLang.
API: OpenAI and Anthropic compatible.
Rules: Same GPU count and model path on every worker in a split.

Get started

Linux

docker run -d -p 80:80 gpustack/gpustack

Then:

docker run -d --name gpustack -p 80:80 gpustack/gpustack

Limits to know

Not for Macs or mixed home devices since v2.0.

We don’t estimate speeds for GPUStack: it runs vLLM or SGLang underneath, so see their figures.

Sources:

All clustering tools

Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.