vLLM and SGLang (multi-node)

by vLLM project, SGLang project · Apache-2.0 · 96,479 GitHub stars · Active · vLLM v0.33.0, SGLang v0.5.23, Nov 2026

The data-center serving engines can also spread one model across several Linux GPU servers, with tensor split inside each server and pipeline split between them.

Mac: No
NVIDIA PC: Yes
AMD PC: Yes
Linux, processor only: No
Raspberry Pi and boards: No
Windows: No
Android: No
iPhone, iPad: No

Go to vLLM and SGLang (multi-node) Source code

At a glance

Best for: Racks or labs of matching Linux NVIDIA or AMD GPU servers on fast links, serving many users at once. Two DGX Sparks are the one desk-sized fit.
Runs on: Linux GPU servers only for multi-node; no Macs, phones or mixed GPU makers.
How it splits a model: Tensor, pipeline, expert and data parallel.
Connections: Wants InfiniBand-class or RDMA networking for tensor split; pipeline split between servers works on ordinary Ethernet. Traffic between nodes isn't encrypted.
How devices join: Manual: a head node and workers (Ray for vLLM, launch flags for SGLang).
Models: Hugging Face models (BF16, FP8, AWQ and GPTQ 4-bit).
API: OpenAI compatible.
Rules: Identical software and model paths on every node; containers recommended.

Get started

Linux

docker pull vllm/vllm-openai   (or pip install sglang)

Then:

vllm serve <model> --tensor-parallel-size 8 --pipeline-parallel-size 2 --distributed-executor-backend ray

Limits to know

Not for mixed home devices.
Manual head and worker setup.

Sources:

All clustering tools

Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.