LLMs · Ant Group (inclusionAI)

Ling 2.6 1T

Parameters: 1,025B (63.0B active)
Architecture: Mixture of experts
Max context: 256K tokens
License: mit
Released: 2026-04

How to run it

Original weights · Hugging Face

Full-precision files from Ant Group (inclusionAI), for fine-tuning, vLLM, Transformers.

huggingface.co/inclusionAI/Ling-2.6-1T (opens in a new tab)

No sign-in needed.

Official repo · GitHub

Reference code, inference scripts and docs from Ant Group (inclusionAI). The weights themselves download from Hugging Face.

github.com/inclusionAI/Ling-V2 (opens in a new tab)

git clone https://github.com/inclusionAI/Ling-V2

Links checked 5 days ago against Hugging Face, Ollama and GitHub.

Apps that run Ling 2.6 1T

Our pick for each kind of computer, assuming the model fits in its memory. Pick your own GPU or device in the Model Finder for an exact answer.

Mac (Apple silicon)

Docker Model Runner

Handy if you already work in Docker.

Also good:

Text Generation WebUI

Windows PC, NVIDIA GPU

Docker Model Runner

Handy if you already work in Docker.

Also good:

Text Generation WebUI

Windows PC, AMD Radeon

Docker Model Runner

Handy if you already work in Docker.

Also good:

Text Generation WebUI

Linux, NVIDIA GPU

Docker Model Runner

Handy if you already work in Docker.

Also good:

Text Generation WebUI

Linux server, 80 GB+ GPUs

Docker Model Runner

Handy if you already work in Docker.

Also good:

Text Generation WebUI

App	Type	Opens it as	Systems	Engine	Start
vLLM	Server	Original weights	WSLWindowsLinux	vLLM	`vllm serve inclusionAI/Ling-2.6-1T`
SGLang	Server	Original weights	WSLWindowsLinux	SGLang	`python3 -m sglang.launch_server --model-path inclusionAI/Ling-2.6-1T`
Docker Model Runner	CLI + API	Original weights	macOSWindowsLinux	llama.cpp, vLLM	`docker model run hf.co/inclusionAI/Ling-2.6-1T`
Text Generation WebUI	Desktop app	Original weights	macOSWindowsLinux	llama.cpp, Transformers, ExLlamaV3	Get Text Generation WebUI

Which apps can open a model depends on the files published for it: GGUF for llama.cpp-based apps, MLX for Apple’s engine, the original weights for vLLM and SGLang. Compare every app.

Memory needed

Weights plus KV cache and runtime overhead. Leave a little extra free for the operating system and other apps. Try another context length.

KV cache: hybrid: 10 of 80 layers keep a (latent) KV cache; the rest use linear attention.

Quantization	Quality	Weights	Total at 8K context	Total at 32K context	Download
`Q2_K`	Low	410 GB	411 GB	411 GB	All files (opens in a new tab)
`Q3_K_M`	Low	502 GB	503 GB	503 GB	All files (opens in a new tab)
`Q4_K_M`	Good	626 GB	626 GB	626 GB	All files (opens in a new tab)
`Q5_K_M`	High	728 GB	729 GB	729 GB	All files (opens in a new tab)
`Q6_K`	High	841 GB	842 GB	842 GB	All files (opens in a new tab)
`Q8_0`	Very high	1,087 GB	1,088 GB	1,088 GB	All files (opens in a new tab)
`F16`	Full	2,051 GB	2,052 GB	2,052 GB	Original weights (opens in a new tab)

Best hardware for Ling 2.6 1T

Quality	Hardware	Quant	Memory	Speed	Runs
64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	Apple M3 Ultra (512 GB)	`Q2_K`	442 GB / 384 GB	17.9 tok/s	Slowfar over GPU memory limit
64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	AMD EPYC 7002 / 7003 series NAS (512 GB)	`Q2_K`	449 GB / 471 GB	3.6 tok/s	Slowtight fit
64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	Apple M5 Ultra (512 GB)	`Q2_K`	442 GB / 384 GB	25.5 tok/s	Slowfar over GPU memory limit
64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	Intel Xeon Silver (4100 to 4300 series) NAS (512 GB)	`Q2_K`	449 GB / 471 GB	1.8 tok/s	Very slowtight fit
64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	Intel Xeon W-2100 series NAS (512 GB)	`Q2_K`	449 GB / 471 GB	1.7 tok/s	Very slowtight fit
64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	Intel Xeon D-2700 series NAS (512 GB)	`Q2_K`	449 GB / 471 GB	1.3 tok/s	Very slowtight fit

Check it on your own hardware

Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.

Was this page helpful?