Intel Xeon Silver (4100 to 4300 series) NAS (512 GB) vs AMD Radeon RX 7600 XT for local AI

Pick two to four systems or devices to see which runs more models, faster, for the kind of work you do.

Hardware to compare, two to four

Winner: Intel Xeon Silver (4100 to 4300 series) NAS (512 GB)

For general chat: It runs 124 more LLMs (614 vs 490). On the other hand, the AMD Radeon RX 7600 XT is about 18.5× faster on its own top models.

Side by side

Swap sides All models on the Xeon Silver (4100 to 4300 series) NAS All models on the RX 7600 XT

  QNAP TDS-h6889AXU-RPIntel Xeon Silver (4100 to 4300 series) NAS (512 GB) AMD Radeon RX 7600 XTAMD Radeon RX 7600 XT
Memory 512 GB unified 16.0 GB VRAM
Usable for models 471 GB 16.0 GB
Memory bandwidth 115 GB/s 288 GB/s
FP16 compute 1.0 TFLOPS 45.1 TFLOPS
Launch price — $329
Released 2017 2024
LLMs that run 614 of 619 490 of 619
LLMs that run well 366 456
Image models that run 60 of 60 55 of 60
Video models that run 43 of 43 17 of 43
Decision models that run 70 of 70 68 of 70
Top-10 LLM average score 90 90
Top-10 LLM average speed 3.2 tok/s 59.9 tok/s
Best LLM MiMo V2.6 Flash (310B-A15B) Gemma 4 26B-A4B

Highlighted values are the best of the two. Model results assume 8K of context and the default macOS GPU memory limit on Macs.

LLMs

614 of 619 LLMs run on the Intel Xeon Silver (4100 to 4300 series) NAS (512 GB). 471 GB usable for models.

490 of 619 LLMs run on the AMD Radeon RX 7600 XT. 16.0 GB usable for models.

Model QualityXeon Silver (4100 to 4300 series) NAS SpeedXeon Silver (4100 to 4300 series) NAS RunsXeon Silver (4100 to 4300 series) NAS QualityRX 7600 XT SpeedRX 7600 XT RunsRX 7600 XT
MiMo V2.6 Flash (310B-A15B)Xiaomi · 311B (15.0B active) 95Excellent quality (A+) · 95 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Intern-S2 (397B-A17B)Shanghai AI Lab · 403B (17.0B active) 92Excellent quality (A+) · 92 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Nex N2 Pro (397B-A17B)Nex-AGI · 397B (17.0B active) 92Excellent quality (A+) · 92 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3.5 397B-A17BAlibaba · 403B (17.0B active) 92Excellent quality (A+) · 92 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Naive N0.5 Flash (309B-A15.5B)Naive AI · 309B (15.5B active) 94Excellent quality (A+) · 94 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slow 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V4 Flash (284B-A13B)DeepSeek · 304B (13.0B active) 93Excellent quality (A+) · 93 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 3.6 tok/s Slow 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiMo V2.5 (310B-A15B)Xiaomi · 311B (15.0B active) 93Excellent quality (A+) · 93 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ornith 1.5 397B-A17BDeepReinforce · 403B (17.0B active) 91Excellent quality (A+) · 91 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-5.3 Flash (320B-A18B)Zhipu AI · 321B (18.0B active) 93Excellent quality (A+) · 93 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiMo V2 Flash (309B-A15B)Xiaomi · 310B (15.0B active) 89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Inkling Small (276B-A12B)Thinking Machines Lab · 266B (12.0B active) 92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slow 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Nemotron 3 Ultra 550B-A55BNVIDIA · 561B (55.0B active) 88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 1.2 tok/s Very slow 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V4.1 Flash 1017 (552B-A16B)DeepSeek · 763B (16.0B active) 82Great quality (A) · 82 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 5.4 tok/s Slowtight fit 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Step 3.7 Flash (198B-A11B)StepFun · 201B (11.0B active) 92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 4.3 tok/s Slow 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3.8 Flash Next (125B-A6B)Alibaba · 180B (6.0B active) 91Excellent quality (A+) · 91 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Solar Open 2 250BUpstage · 250B (15.0B active) 89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling 3.1 Flash 1019 (560B-A25B)Ant Group (inclusionAI) · 572B (25.0B active) 81Great quality (A) · 81 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V4.1 Flash (552B-A16B)DeepSeek · 763B (16.0B active) 81Great quality (A) · 81 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 5.4 tok/s Slowtight fit 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Step 5 Preview (600B-A27B)StepFun · 604B (27.0B active) 81Great quality (A) · 81 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Step 3.5 Flash (196B-A11B)StepFun · 199B (11.0B active) 91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 4.3 tok/s Slow 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Hunyuan Hy3 (295B-A21B)Tencent · 299B (21.0B active) 89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling 3.1 Flash (560B-A25B)Ant Group (inclusionAI) · 572B (25.0B active) 80Great quality (A) · 80 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Mistral Large 3 (675B)Mistral AI · 675B (41.0B active) 85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 1.7 tok/s Very slowtight fit 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-5.3 (753B-A40B)Zhipu AI · 753B (40.0B active) 82Great quality (A) · 82 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 2.2 tok/s Very slowtight fit 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Llama 3.1 Nemotron Ultra 253B v1NVIDIA · 253B 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 0.1 tok/s Very slow 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Beam 1.0 (501B-A23B)Reflection AI · 501B (23.4B active) 79Great quality (A) · 79 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 3.7 tok/s Slow 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V4 Flash (April 2026)DeepSeek · 291B (13.0B active) 91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 3.6 tok/s Slow 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-5.2 (753B-A40B)Zhipu AI · 753B (40.0B active) 80Great quality (A) · 80 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 2.2 tok/s Very slowtight fit 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3.6 35B-A3BAlibaba · 36B (3.0B active) 88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
MiniMax-M1 80KMiniMax · 456B (45.9B active) 80Great quality (A) · 80 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 1.2 tok/s Very slow 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Muse Spark Open (816B-A42B)Meta · 822B (42.0B active) 80Great quality (A) · 80 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 2.1 tok/s Very slowtight fit 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniCPM-V 4.7 35B-A3BOpenBMB · 35B (3.0B active) 88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
Qwen3.5 35B-A3BAlibaba · 36B (3.0B active) 88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
MiniMax M3.1 Flash (428B-A23B)MiniMax · 429B (23.0B active) 81Great quality (A) · 81 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slow 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-5.1 (744B-A40B)Zhipu AI · 754B (40.0B active) 79Great quality (A) · 79 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 2.2 tok/s Very slowtight fit 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Flint (40B-A2.5B)Recursive · 40B (2.5B active) 85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 25.1 tok/s Runs well 75Great quality (A) · 75 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 70.7 tok/s Runs welloffloads to RAM
MiniMax-M1 40KMiniMax · 456B (45.9B active) 79Great quality (A) · 79 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 1.2 tok/s Very slow 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Hunyuan Hy4 Preview (780B)Tencent · 780B (55.0B active) 80Great quality (A) · 80 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 1.6 tok/s Very slowtight fit 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Nemotron 3.5 Super VL 120B-A12BNVIDIA · 124B (12.0B active) 92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slow 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
NorthCUA Probeipo-ai · 397B (17.0B active) 75Great quality (A) · 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kolibri 1 (78B-A3.5B)Aleph Alpha · 78B (3.5B active) 86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 18.6 tok/s Runs well 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 49.6 tok/s Slowheavy RAM offload
GLM-5 (744B-A40B)Zhipu AI · 754B (40.0B active) 79Great quality (A) · 79 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 2.2 tok/s Very slowtight fit 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Nex N2 Mini (35B-A3B)Nex-AGI · 35B (3.0B active) 87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
ERNIE 4.5 300B-A47BBaidu · 300B (47.0B active) 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 0.7 tok/s Very slow 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Sage-A1.5 35B-A3BInternScience · 36B (3.0B active) 86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
LongCat Flash Chat (560B)Meituan · 562B (27.0B active) 73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax M3 (427B)MiniMax · 427B (25.0B active) 79Great quality (A) · 79 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ornith 1.5 35B-A3BDeepReinforce · 36B (3.0B active) 85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
Gemma 4 26B-A4BGoogle · 26B (3.8B active) 91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 12.0 tok/s Runs well 77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 77.5 tok/s Runs welltight fit
Qwen3-Next 80B-A3B ThinkingAlibaba · 81B (3.0B active) 81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 54.9 tok/s Slowheavy RAM offload
Apodex 1.1 Mini (35B-A3B)Apodex · 36B (3.0B active) 84Great quality (A) · 84 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
Ornith 1.0 35B-A3BDeepReinforce · 36B (3.0B active) 84Great quality (A) · 84 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
gpt-oss 120BOpenAI · 117B (5.1B active) 84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 13.7 tok/s Runs well 84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3 235B-A22B Instruct 2507Alibaba · 235B (22.0B active) 85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Hermes 4 405BNous Research · 406B 79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 0.1 tok/s Very slow 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
K2-Horizon (375B-A23B)Institute of Foundation Models · 379B (23.0B active) 79Great quality (A) · 79 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slow 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-4.7 Flash (30B-A3B)Zhipu AI · 31B (3.0B active) 83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
Qwen3-VL 235B-A22B InstructAlibaba · 236B (22.0B active) 84Great quality (A) · 84 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3 Next 80B-A3B InstructAlibaba · 81B (3.0B active) 79Great quality (A) · 79 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 54.9 tok/s Slowheavy RAM offload
Agents-A1 35B-A3BInternScience · 35B (3.0B active) 82Great quality (A) · 82 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
Qwen AgentWorld 35B-A3BAlibaba · 35B (3.0B active) 82Great quality (A) · 82 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
MiniMax-Text-01MiniMax · 456B (45.9B active) 73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 1.2 tok/s Very slow 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3 235B-A22BAlibaba · 235B (22.0B active) 83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Nemotron 3 Super 120B-A12BNVIDIA · 124B (12.0B active) 87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slow 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Llama 4 Maverick (17B x 128E)Meta · 402B (17.0B active) 69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Trinity Large Thinking (400B-A13B)Arcee AI · 399B (13.0B active) 79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 3.6 tok/s Slow 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Raon-1 mini Preview (30B-A4B)KRAFTON · 29B (4.3B active) 87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 10.6 tok/s Runs well 73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 70.6 tok/s Runs welltight fit
Tülu 3 405BAi2 · 406B 75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 0.1 tok/s Very slow 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling 3.0 Flash (127B-A5B)Ant Group (inclusionAI) · 127B (5.1B active) 77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 12.9 tok/s Runs well 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Baichuan M3 235B-A22BBaichuan · 235B (22.0B active) 79Great quality (A) · 79 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Laguna XS 2.1 (33B-A3B)Poolside · 33B (3.0B active) 77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 68Good quality (B) · 68 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 67.5 tok/s Runs welloffloads to RAM
DeepSeek V3.2 (671B)DeepSeek · 685B (37.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slow 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K2.6 (1T)Moonshot AI · 1,027B (32.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slowtight fit 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiMo V2.6 Pro (1T-A42B)Xiaomi · 1,024B (42.0B active) 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 2.7 tok/s Very slowtight fit 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V3.2 ExpDeepSeek · 685B (37.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slow 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K2.5 (1T)Moonshot AI · 1,027B (32.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slowtight fit 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek R1 (671B)DeepSeek · 671B (37.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slow 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiMo V2.5 Pro (1T-A42B)Xiaomi · 1,023B (42.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 2.7 tok/s Very slowtight fit 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-4.6VZhipu AI · 108B (12.0B active) 82Great quality (A) · 82 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slow 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V3.1 TerminusDeepSeek · 685B (37.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slow 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Llama 3.1 405B InstructMeta · 406B 71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 0.1 tok/s Very slow 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Mistral Large 4 (1T-A52B)Mistral AI · 1,053B (52.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 2.2 tok/s Very slowtight fit 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Jamba 1.5 LargeAI21 Labs · 399B (94.0B active) 69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V3.1 (671B)DeepSeek · 685B (37.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slow 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Mistral Small 4 (119B-A6B)Mistral AI · 119B (6.0B active) 76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
KAT-Coder V2.5 Dev (35B-A3B)Kuaishou · 35B (3.0B active) 86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
Inkling (975B-A41B)Thinking Machines Lab · 952B (41.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 2.7 tok/s Very slowtight fit 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Nemotron Cascade 2 30B-A3BNVIDIA · 32B (3.5B active) 76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 18.4 tok/s Runs well 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 98.8 tok/s Runs welltight fit
Kimi K2 Instruct 0905 (1T)Moonshot AI · 1,026B (32.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slowtight fit 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3 235B-A22B Thinking 2507Alibaba · 235B (22.0B active) 86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3 Coder Next (80B-A3B)Alibaba · 80B (3.0B active) 81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 55.3 tok/s Slowheavy RAM offload
GLM-4.5VZhipu AI · 108B (12.0B active) 79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slow 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K2 Instruct (July 2025)Moonshot AI · 1,026B (32.0B active) 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slowtight fit 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Solar Open 100BUpstage · 103B (12.0B active) 79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slow 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling 2.6 1TAnt Group (inclusionAI) · 1,025B (63.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 1.8 tok/s Very slowtight fit 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LFM2.5 24B-A2BLiquid AI · 24B (2.5B active) 70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 25.6 tok/s Runs well 62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 106 tok/s Runs great
dots.llm1 Instructrednote · 143B (14.0B active) 75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3 30B-A3B Instruct 2507Alibaba · 31B (3.3B active) 73Great quality (A) · 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 19.4 tok/s Runs well 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 103 tok/s Runs welltight fit
Mistral Medium 3.5 (128B)Mistral AI · 128B 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 0.3 tok/s Very slow 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi Linear 48B-A3BMoonshot AI · 49B (3.0B active) 70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 74.8 tok/s Runs welloffloads to RAM
Nemotron 3 Nano 30B-A3BNVIDIA · 32B (3.5B active) 73Great quality (A) · 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 18.4 tok/s Runs well 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 98.8 tok/s Runs welltight fit
Ling 1TAnt Group (inclusionAI) · 1,000B (50.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 2.2 tok/s Very slowtight fit 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-4.5 Air (106B-A12B)Zhipu AI · 110B (12.0B active) 77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slow 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3-VL 30B-A3B ThinkingAlibaba · 31B (3.3B active) 72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 19.4 tok/s Runs well 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 103 tok/s Runs welltight fit
Mistral Large 2.1 (2411)Mistral AI · 123B 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 0.3 tok/s Very slow 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DiffusionGemma 26B-A4B ITGoogle · 26B (4.0B active) 74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 16.2 tok/s Runs well 65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 74.6 tok/s Runs welltight fit
K-EXAONE 236B-A23BLG AI Research · 237B (23.0B active) 73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slow 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling Flash 2.0 (100B-A6B)Ant Group (inclusionAI) · 103B (6.1B active) 72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 10.8 tok/s Runs well 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V3 0324 (671B)DeepSeek · 685B (37.0B active) 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slow 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Hunyuan Hy-Memory 30B-A3BTencent · 30B (3.0B active) 80Great quality (A) · 80 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 59Good quality (B) · 59 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
Command A (111B)Cohere · 111B 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 0.3 tok/s Very slow 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3-VL 30B-A3B InstructAlibaba · 31B (3.3B active) 70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 19.4 tok/s Runs well 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 103 tok/s Runs welltight fit
Qwen3 30B-A3BAlibaba · 31B (3.3B active) 70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 19.4 tok/s Runs well 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 103 tok/s Runs welltight fit
Sarvam 30BSarvam AI · 32B (2.4B active) 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 26.1 tok/s Runs well 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 126 tok/s Runs welltight fit
Command A+ (218B-A25B)Cohere · 219B (25.0B active) 73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Pixtral Large 124B (2411)Mistral AI · 124B 78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 0.3 tok/s Very slow 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Mistral Large 2 (123B)Mistral AI · 123B 78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 0.3 tok/s Very slow 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Leanstral 119B-A6BMistral AI · 119B (6.5B active) 70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 10.2 tok/s Runs well 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Nemotron 3.5 Lightning 30B-A3BNVIDIA · 32B (3.0B active) 68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
InternVL3.5 241B-A28BShanghai AI Lab · 241B (28.0B active) 72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
gpt-oss 20BOpenAI · 21B (3.6B active) 70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 19.1 tok/s Runs well 70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 69.4 tok/s Runs great
Gemma 4 12BGoogle · 12B 82Great quality (A) · 82 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 4.0 tok/s Slow 78Great quality (A) · 78 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 25.5 tok/s Runs well
DeepSeek V2.5 1210DeepSeek · 236B (21.0B active) 68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax M2.7 (230B-A10B)MiniMax · 229B (10.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 10.9 tok/s Runs well 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Devstral 2 (123B)Mistral AI · 125B 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 0.3 tok/s Very slow 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax M2.5 (230B-A10B)MiniMax · 229B (10.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 10.9 tok/s Runs well 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek R1 0528 (671B)DeepSeek · 685B (37.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slow 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3-Omni 30B-A3B InstructAlibaba · 35B (3.3B active) 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 19.4 tok/s Runs well 58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 61.6 tok/s Runs welloffloads to RAM
Ling 3.0 flash SanteAnt Group (inclusionAI) · 127B (5.1B active) 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 12.9 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Groot 65B A6B 261023GrootTeam · 118B (6.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V2.5DeepSeek · 236B (21.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K2.7 Code (1T)Moonshot AI · 1,027B (32.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slowtight fit 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax M2.1 (230B-A10B)MiniMax · 229B (10.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 10.9 tok/s Runs well 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling 3.0 Tiny (8B-A1.3B)Ant Group (inclusionAI) · 7.9B (1.3B active) 58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 37.8 tok/s Runs great 60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 82.0 tok/s Runs great
North Mini Code 1.0 (30B-A3B)Cohere · 30B (3.0B active) 76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
GLM-4.7 (355B-A32B)Zhipu AI · 358B (32.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3.8 27BAlibaba · 28B 80Great quality (A) · 80 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 80Great quality (A) · 80 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 14.2 tok/s Runs welltight fit
Ling 2.6 Flash (104B-A7.4B)Ant Group (inclusionAI) · 107B (7.4B active) 67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 11.5 tok/s Runs well 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V3 (671B)DeepSeek · 671B (37.0B active) 52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slow 52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K2 Thinking (1T)Moonshot AI · 1,026B (32.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slowtight fit 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Hunyuan A13B (80B-A13B)Tencent · 80B (13.0B active) 73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 3.6 tok/s Slow 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 14.3 tok/s Slowheavy RAM offload
Llama 3.3 Nemotron Super 49B v1.5NVIDIA · 50B 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 0.7 tok/s Very slow 67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slowheavy RAM offload
Maple Preview (20B-A1B)DeepGrove · 20B (1.5B active) 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 39.9 tok/s Runs great 67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 106 tok/s Runs welltight fit
Qwen3.6 27BAlibaba · 28B 78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 14.2 tok/s Runs welltight fit
MiniMax M2 (230B-A10B)MiniMax · 229B (10.0B active) 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 10.9 tok/s Runs well 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3.5 27BAlibaba · 28B 78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 14.2 tok/s Runs welltight fit
Trinity Mini (26B-A3B)Arcee AI · 26B (3.0B active) 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 92.1 tok/s Runs welltight fit
Qwen3 30B-A3B Thinking 2507Alibaba · 31B (3.3B active) 74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 19.4 tok/s Runs well 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 103 tok/s Runs welltight fit
Qwen3 Coder 30B-A3BAlibaba · 31B (3.3B active) 74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 19.4 tok/s Runs well 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 103 tok/s Runs welltight fit
Gemma 4 E2BGoogle · 5.1B 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 29.8 tok/s Runs well 62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 67.4 tok/s Runs great
Llama 4 Scout 17B-16EMeta · 109B (17.0B active) 69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ornith 1.5 122B-A10BDeepReinforce · 125B (10.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 10.9 tok/s Runs well 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Nemotron 3.5 Bolt E4BNVIDIA · 8.0B 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 15.2 tok/s Runs well 69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 46.9 tok/s Runs great
Qwen3.5 122B-A10BAlibaba · 125B (10.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 10.9 tok/s Runs well 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Snowflake Arctic InstructSnowflake · 479B (17.0B active) 48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ring Flash 2.0 (100B-A6B)Ant Group (inclusionAI) · 103B (6.1B active) 74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 10.8 tok/s Runs well 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
InternVL3 78BShanghai AI Lab · 78B 74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 0.4 tok/s Very slow 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 2.4 tok/s Very slowheavy RAM offload
Nemotron Labs Dory 1.0 4BNVIDIA · 4.0B 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 17.2 tok/s Runs well 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great
Command A Reasoning (111B)Cohere · 111B 82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 0.3 tok/s Very slow 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
TMax 27BAi2 · 27B 75Great quality (A) · 75 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 75Great quality (A) · 75 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 14.4 tok/s Runs welltight fit
ERNIE 4.5 21B-A3BBaidu · 22B (3.0B active) 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 75.6 tok/s Runs welltight fit
LFM3 1.7BLiquid AI · 1.7B 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 39.7 tok/s Runs great 56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 41.3 tok/s Runs great
Nanbeige4.2 3BBOSS Zhipin (Nanbeige) · 4.2B 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 16.5 tok/s Runs well 67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 50.7 tok/s Runs great
Nemotron 3.5 Bolt 74B-A9BNVIDIA · 75B (9.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 12.1 tok/s Runs well 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 21.6 tok/s Slowheavy RAM offload
Llama 3.3 70B InstructMeta · 71B 74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 2.8 tok/s Very slowheavy RAM offload
Hermes 4 70BNous Research · 71B 74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 2.8 tok/s Very slowheavy RAM offload
Gemma 4 E4BGoogle · 8.0B 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 15.2 tok/s Runs well 67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 46.9 tok/s Runs great
Axion 1.0 35BAxion · 35B (3.0B active) 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
Qwen2.5-VL 72B InstructAlibaba · 73B 73Great quality (A) · 73 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 2.6 tok/s Very slowheavy RAM offload
Qwen2.5 72B InstructAlibaba · 73B 73Great quality (A) · 73 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 2.6 tok/s Very slowheavy RAM offload
Command R+ (08-2024) 104BCohere · 104B 70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 0.3 tok/s Very slow 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Fara 1.5 27BMicrosoft · 27B 73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 14.4 tok/s Runs welltight fit
Laguna S 2.1 (118B-A8B)Poolside · 118B (8.0B active) 72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 10.7 tok/s Runs well 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
InternVL3.5 38BShanghai AI Lab · 38B 75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 1.2 tok/s Very slow 72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slowheavy RAM offload
MiniCPM5 2BOpenBMB · 2.5B 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 27.2 tok/s Runs well 60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 61.5 tok/s Runs great
Spark-X2.6 4BSparkLLM · 4.1B 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 16.7 tok/s Runs well 65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 51.4 tok/s Runs great
GLM-4.6 (355B-A32B)Zhipu AI · 357B (32.0B active) 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek Coder V2 (236B)DeepSeek · 236B (21.0B active) 70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek R1 Distill Llama 70BDeepSeek · 71B 72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 2.8 tok/s Very slowheavy RAM offload
Tülu 3 70BAi2 · 71B 72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 2.8 tok/s Very slowheavy RAM offload
DeepSeek V2 ChatDeepSeek · 236B (21.0B active) 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling Mini 2.0 (16B-A1.4B)Ant Group (inclusionAI) · 16B (1.4B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 42.1 tok/s Runs great 50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs great
UI-Mate 27BTencent · 27B 72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 14.4 tok/s Runs welltight fit
GLM-4.5 (355B-A32B)Zhipu AI · 358B (32.0B active) 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LFM2.5 8B-A1BLiquid AI · 8.5B (1.5B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great 52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 74.2 tok/s Runs great
Qwen3 Coder 480B-A35BAlibaba · 480B (35.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
NorthCUA Minibeipo-ai · 35B (3.0B active) 56Good quality (B) · 56 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 41Fair quality (C) · 41 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs welltight fit
Mellum2.1 Thinking (12B-A2.5B)JetBrains · 12B (2.5B active) 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 25.1 tok/s Runs well 69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 64.7 tok/s Runs great
DeepSeek Coder V2 Instruct (0614)DeepSeek · 236B (21.0B active) 68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi-VL A3B Thinking 2506Moonshot AI · 16B (3.0B active) 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 64.8 tok/s Runs great
LFM2 24B-A2BLiquid AI · 24B (2.3B active) 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 27.1 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 110 tok/s Runs great
Apriel 1.5 15B ThinkerServiceNow · 15B 71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 20.5 tok/s Runs well
Llama 3.1 70B InstructMeta · 71B 70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 2.8 tok/s Very slowheavy RAM offload
Grok-1 (314B)xAI · 314B (86.0B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 0.6 tok/s Very slow 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi Dev 72BMoonshot AI · 73B 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 2.6 tok/s Very slowheavy RAM offload
ERNIE 4.5 21B-A3B ThinkingBaidu · 22B (3.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 75.6 tok/s Runs welltight fit
QVQ 72B PreviewAlibaba · 73B 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 2.6 tok/s Very slowheavy RAM offload
Phi-4 Reasoning Vision 15BMicrosoft · 15B 69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 20.1 tok/s Runs well
Hermes 3 (Llama 3.1 70B)Nous Research · 71B 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 2.8 tok/s Very slowheavy RAM offload
Llama 3.2 90B Vision InstructMeta · 89B 66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 0.4 tok/s Very slow 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Groot 19B A3B 261023GrootTeam · 19B (3.0B active) 54Fair quality (C) · 54 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 54Fair quality (C) · 54 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 75.6 tok/s Runs great
TMax 4BAi2 · 4.2B 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 16.3 tok/s Runs well 60Good quality (B) · 60 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 50.2 tok/s Runs great
Qwen3.5 4BAlibaba · 4.7B 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 14.7 tok/s Runs well 61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 45.3 tok/s Runs great
Command R+ 104B (04-2024)Cohere · 104B 64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 0.3 tok/s Very slow 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LFM3-VL 2BLiquid AI · 1.9B 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 35.7 tok/s Runs great 50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 80.7 tok/s Runs great
InternVL3 38BShanghai AI Lab · 38B 69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 1.2 tok/s Very slow 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 8.6 tok/s Slowoffloads to RAM
Nanbeige4.3 33BBOSS Zhipin (Nanbeige) · 33B 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 14.0 tok/s Runs welloffloads to RAM
Apertus v1.5 70BSwiss AI · 72B 66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 2.7 tok/s Very slowheavy RAM offload
Qwen3 14BAlibaba · 15B 67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
Llama 3.1 Nemotron 70B InstructNVIDIA · 71B 66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 2.8 tok/s Very slowheavy RAM offload
Agents-A1 4BInternScience · 4.5B 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 15.1 tok/s Runs well 59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 46.5 tok/s Runs great
Granite 4.0 H Tiny (7B-A1B)IBM · 6.9B (1.0B active) 43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs great 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 97.5 tok/s Runs great
Qwen2-VL 72B InstructAlibaba · 73B 64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 2.6 tok/s Very slowheavy RAM offload
Qwen2 72B InstructAlibaba · 73B 64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 0.5 tok/s Very slow 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 2.6 tok/s Very slowheavy RAM offload
Hermes 4 14BNous Research · 15B 65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
Muse Glimmer 30BMeta · 30B 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slow 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 16.8 tok/s Runs well
Fara 1.5 4BMicrosoft · 4.5B 55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 15.1 tok/s Runs well 57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 46.5 tok/s Runs great
Devstral Small 2 (24B)Mistral AI · 24B 63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 3.7 tok/s Slow 63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 16.4 tok/s Runs welltight fit
Gemma 4 31BGoogle · 31B 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 3.6 tok/s Slow 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 16.0 tok/s Runs welltight fit
K2 Horizon 7BInstitute of Foundation Models · 9.0B 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 12.5 tok/s Runs well 76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 33.8 tok/s Runs great
Granite 4.2 30BIBM · 29B 63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slow 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 17.1 tok/s Runs welltight fit
Sarvam 105BSarvam AI · 106B (10.0B active) 52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 10.9 tok/s Runs well 52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ministral 3 14BMistral AI · 14B 63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 21.8 tok/s Runs well
Granite 4.2 8BIBM · 8.8B 57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 10.1 tok/s Runs well 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 34.6 tok/s Runs great
Phi-4 (14B)Microsoft · 15B 63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 20.7 tok/s Runs well
InternVL3 14BShanghai AI Lab · 15B 63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 20.1 tok/s Runs well
Mage-VL 4.7BMicrosoft · 4.7B 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 14.5 tok/s Runs well 55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 44.6 tok/s Runs great
Qwen3 4B Instruct 2507Alibaba · 4.0B 51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 17.1 tok/s Runs well 53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 52.5 tok/s Runs great
SenseNova Loop 9B PreviewSenseTime · 9.7B 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 11.7 tok/s Runs well 74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 31.5 tok/s Runs great
Mixtral 8x22B InstructMistral AI · 141B (39.0B active) 54Fair quality (C) · 54 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 1.2 tok/s Very slow 38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi-VL A3B InstructMoonshot AI · 16B (3.0B active) 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 64.8 tok/s Runs great
Phi-3.5 MoE (42B-A6.6B)Microsoft · 42B (6.6B active) 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 10.0 tok/s Runs well 39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 43.0 tok/s Runs welloffloads to RAM
Granite 4.2 3BIBM · 3.7B 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 18.7 tok/s Runs well 52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 42.3 tok/s Runs great
Qwen3-VL 4B InstructAlibaba · 4.4B 51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 15.5 tok/s Runs well 53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 47.6 tok/s Runs great
Ornith 1.5 9BDeepReinforce · 9.7B 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 11.7 tok/s Runs well 73Great quality (A) · 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 31.5 tok/s Runs great
Qwen2.5 14B InstructAlibaba · 15B 61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 20.7 tok/s Runs well
Qwen3.5 9BAlibaba · 9.7B 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 11.7 tok/s Runs well 72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 31.5 tok/s Runs great
Gemma 3 27B ITGoogle · 27B 60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 14.4 tok/s Runs welltight fit
Granite 4.1 30BIBM · 29B 60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 17.3 tok/s Runs welltight fit
MiniCPM-V 4.7 1BOpenBMB · 1.6B 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 44.3 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 46.1 tok/s Runs great
Phi-4 Reasoning Plus (14B)Microsoft · 15B 71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 20.8 tok/s Runs well
Gemma 3 12B ITGoogle · 12B 59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slow 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 25.0 tok/s Runs well
Ornith 1.0 9BDeepReinforce · 9.7B 52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 11.7 tok/s Runs well 71Great quality (A) · 71 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 31.5 tok/s Runs great
TMax 9BAi2 · 9.0B 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 12.6 tok/s Runs well 70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 34.0 tok/s Runs great
Devstral Small 1.1 (24B)Mistral AI · 24B 58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slow 58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 16.7 tok/s Runs welltight fit
Magistral Small 2507Mistral AI · 24B 58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slow 58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 16.7 tok/s Runs welltight fit
Qwen2.5 14B Instruct 1MAlibaba · 15B 59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
Apertus 70B InstructSwiss AI · 71B 59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 0.7 tok/s Very slow 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking. 2.8 tok/s Very slowheavy RAM offload
TMax 2BAi2 · 1.9B 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 36.5 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 82.4 tok/s Runs great
MedGemma 27B Text ITGoogle · 27B 58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 14.6 tok/s Runs welltight fit
LFM2 8B-A1BLiquid AI · 8.3B (1.5B active) 36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great 38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 74.2 tok/s Runs great
North Micro VisionCohere · 2.5B 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 27.7 tok/s Runs well 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 62.5 tok/s Runs great
Nemotron 3 Nano 4BNVIDIA · 4.0B 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 17.3 tok/s Runs well 50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 39.0 tok/s Runs great
Qwen3 4BAlibaba · 4.0B 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 17.1 tok/s Runs well 49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 52.5 tok/s Runs great
InternVL3.5 8BShanghai AI Lab · 8.5B 52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 10.4 tok/s Runs well 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 35.7 tok/s Runs great
Qwen3-VL 8B InstructAlibaba · 8.8B 52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 10.1 tok/s Runs well 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 34.7 tok/s Runs great
Phi-4 Reasoning (14B)Microsoft · 15B 69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 20.8 tok/s Runs well
MiniCPM5 1BOpenBMB · 1.1B 35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 44.1 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 66.1 tok/s Runs great
Moonlight 16B-A3B InstructMoonshot AI · 16B (3.0B active) 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 64.8 tok/s Runs great
Magistral Small 2506Mistral AI · 24B 57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slow 57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 16.7 tok/s Runs welltight fit
Mistral Small 3.2 24BMistral AI · 24B 57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.7 tok/s Slow 57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 16.4 tok/s Runs welltight fit
LFM2.5 2.6BLiquid AI · 2.7B 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 25.4 tok/s Runs well 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 57.4 tok/s Runs great
EXAONE 4.5 33BLG AI Research · 34B 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 14.6 tok/s Runs welltight fit
Llama 3 70B InstructMeta · 71B 57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 0.7 tok/s Very slow 40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking. 2.8 tok/s Very slowheavy RAM offload
Qwen3 8BAlibaba · 8.2B 50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 10.8 tok/s Runs well 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 37.2 tok/s Runs great
Qwen1.5 110B ChatAlibaba · 111B 52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 0.3 tok/s Very slow 36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Terminal RSI 27BTeriminal-RSI · 28B 57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 14.2 tok/s Runs welltight fit
Solar Pro Preview 22BUpstage · 22B 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 17.8 tok/s Runs well
MiniCPM-V 4.5 8BOpenBMB · 8.7B 50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 10.2 tok/s Runs well 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 35.0 tok/s Runs great
Granite 4.1 8BIBM · 8.8B 50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 10.1 tok/s Runs well 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 34.6 tok/s Runs great
Granite 4.1 3BIBM · 3.4B 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 20.2 tok/s Runs well 46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 45.6 tok/s Runs great
Hemmingway 1.5Hemmingway · 27B 56Good quality (B) · 56 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 56Good quality (B) · 56 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 14.6 tok/s Runs welltight fit
Qwen3.5 2BAlibaba · 2.3B 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 30.2 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 68.3 tok/s Runs great
Seed-OSS 36B InstructByteDance · 36B 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 10.0 tok/s Runs welloffloads to RAM
Mistral Small 3.1 24BMistral AI · 24B 55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.7 tok/s Slow 55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 16.4 tok/s Runs welltight fit
LFM2.5-VL 3BLiquid AI · 3.1B 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 22.0 tok/s Runs well 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 49.7 tok/s Runs great
Unlimited OCR 2 (3B-A0.6B)Baidu · 3.3B (0.6B active) 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 47.7 tok/s Runs great 44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 78.3 tok/s Runs great
MiMo-VL 7B RLXiaomi · 8.3B 48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 10.7 tok/s Runs well 55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 36.6 tok/s Runs great
Jamba Reasoning 3BAI21 Labs · 3.2B 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 21.4 tok/s Runs well 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 48.4 tok/s Runs great
Hermes 4.3 36BNous Research · 36B 55Good quality (B) · 55 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 55Good quality (B) · 55 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 10.0 tok/s Runs welloffloads to RAM
Mage-VL-Cowork 4.7BMicrosoft · 4.7B 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 14.5 tok/s Runs well 59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 44.6 tok/s Runs great
DeepSeek Coder V2 Lite (16B)DeepSeek · 16B (2.4B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 26.1 tok/s Runs well 50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 76.7 tok/s Runs great
Gemma 3n E4BGoogle · 7.9B 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 17.2 tok/s Runs well 45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 52.8 tok/s Runs great
DeepSeek V2 Lite ChatDeepSeek · 16B (2.4B active) 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 26.1 tok/s Runs well 38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 76.7 tok/s Runs great
Qwen3-VL 32B InstructAlibaba · 33B 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 12.7 tok/s Runs welloffloads to RAM
Nous Hermes 2 Mixtral 8x7B DPONous Research · 47B (12.9B active) 51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 3.6 tok/s Slow 36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 20.7 tok/s Runs welloffloads to RAM
Gemma 3n E2BGoogle · 5.4B 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 34.3 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 77.5 tok/s Runs great
Mistral Small 3 24B (2501)Mistral AI · 24B 53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slow 53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 16.7 tok/s Runs welltight fit
Ministral 3 14B ReasoningMistral AI · 14B 65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 21.8 tok/s Runs well
MiMo 7B RLXiaomi · 7.8B 47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 11.3 tok/s Runs well 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great
AceReason Nemotron 14BNVIDIA · 15B 65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
OpenReasoning Nemotron 14BNVIDIA · 15B 65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
Qwen2.5 Coder 14B InstructAlibaba · 15B 65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
Jamba2 3BAI21 Labs · 3.0B 39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 22.6 tok/s Runs well 42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 51.1 tok/s Runs great
Apriel Nemotron 15B ThinkerServiceNow · 15B 65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 20.3 tok/s Runs well
InternVL3 8BShanghai AI Lab · 7.9B 47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 11.2 tok/s Runs well 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 38.3 tok/s Runs great
LFM2.5 1.2B InstructLiquid AI · 1.2B 31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 40.7 tok/s Runs great 32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 61.0 tok/s Runs great
SmolLM3 3BHugging Face · 3.1B 39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 22.3 tok/s Runs well 42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 50.3 tok/s Runs great
Qwen3 32BAlibaba · 33B 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 13.4 tok/s Runs welloffloads to RAM
Phi-4 Mini (3.8B)Microsoft · 3.8B 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 17.9 tok/s Runs well 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 40.4 tok/s Runs great
Ministral 3 3BMistral AI · 3.9B 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 17.8 tok/s Runs well 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 40.3 tok/s Runs great
Qwen3-VL 2B InstructAlibaba · 2.1B 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 32.2 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 72.8 tok/s Runs great
Qwen2 57B-A14B InstructAlibaba · 57B (14.0B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 11.5 tok/s Slowheavy RAM offload
Qwen3 4B Thinking 2507Alibaba · 4.0B 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 17.1 tok/s Runs well 55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 52.5 tok/s Runs great
Gemma 3 4B ITGoogle · 4.3B 42Fair quality (C) · 42 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 16.0 tok/s Runs well 44Fair quality (C) · 44 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 49.1 tok/s Runs great
PaddleOCR-VL 1.6 0.9BBaidu · 1.0B 41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 49.6 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 74.4 tok/s Runs great
Mixtral 8x7B InstructMistral AI · 47B (12.9B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 3.6 tok/s Slow 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 20.7 tok/s Runs welloffloads to RAM
Dolphin Mistral 24B Venice EditionCognitive Computations · 24B 52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 3.7 tok/s Slow 52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 16.4 tok/s Runs welltight fit
Voxtral Small 24BMistral AI · 24B 52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 3.7 tok/s Slow 52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 16.2 tok/s Runs welltight fit
Jamba 1.5 MiniAI21 Labs · 52B (12.0B active) 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 5.6 tok/s Slow 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 21.4 tok/s Runs welloffloads to RAM
Llama 3.1 Nemotron Nano 8B v1NVIDIA · 8.0B 45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Gemma 2 27B ITGoogle · 27B 52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 18.4 tok/s Runs welltight fit
DeepCoder 14B PreviewAgentica · 15B 63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
MiniCPM4 8BOpenBMB · 8.2B 45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 10.8 tok/s Runs well 51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 37.2 tok/s Runs great
MedGemma 4B ITGoogle · 4.3B 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 16.0 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 49.1 tok/s Runs great
Granite 4.0 H Small (32B-A9B)IBM · 32B (9.0B active) 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 12.1 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 47.3 tok/s Runs welltight fit
GLM-4 32B 0414Zhipu AI · 33B 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 15.4 tok/s Runs welltight fit
MiniCPM-OCR 1.0 1.3BOpenBMB · 1.3B 40Fair quality (C) · 40 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 43.6 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 54.9 tok/s Runs great
Mistral Nemo 12B InstructMistral AI · 12B 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 5.6 tok/s Slow 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 25.0 tok/s Runs well
Llama 3.1 Nemotron Nano 4B v1.1NVIDIA · 4.5B 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 15.2 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 46.8 tok/s Runs great
Unlimited OCR (3B-A0.6B)Baidu · 3.3B (0.6B active) 40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 47.7 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 78.3 tok/s Runs great
Reka Flash 3.1 (21B)Reka AI · 21B 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 14.6 tok/s Runs welltight fit
EXAONE 4.0 1.2BLG AI Research · 1.3B 29Basic quality (D) · 29 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 44.3 tok/s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 55.8 tok/s Runs great
Hunyuan 7B InstructTencent · 7.5B 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 11.8 tok/s Runs well 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs great
Mistral Small 22B (2409)Mistral AI · 22B 51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 17.7 tok/s Runs welltight fit
MiMo V2.6 Distill Qwen 9BXiaomi · 9.4B 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 12.0 tok/s Runs well 74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 32.4 tok/s Runs great
Qwen2.5 7B InstructAlibaba · 7.6B 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 11.6 tok/s Runs well 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 40.0 tok/s Runs great
Dolphin 3.0 R1 Mistral 24BCognitive Computations · 24B 50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slow 50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 16.7 tok/s Runs welltight fit
Granite 4.0 Micro (3B)IBM · 3.4B 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 20.2 tok/s Runs well 40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 45.6 tok/s Runs great
EXAONE 4.0 32BLG AI Research · 32B 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 15.6 tok/s Runs welltight fit
Olmo 3.1 32B InstructAi2 · 32B 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 15.5 tok/s Runs welltight fit
Hunyuan 4B InstructTencent · 4.2B 39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 16.3 tok/s Runs well 41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 50.1 tok/s Runs great
Qwen2.5-VL 7B InstructAlibaba · 8.3B 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 10.7 tok/s Runs well 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 36.7 tok/s Runs great
DeepSeek R1 Distill Qwen 14BDeepSeek · 15B 61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
TeleOCRXingChen-AGI · 1.4B 38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 39.9 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 50.3 tok/s Runs great
JanusCoder 14BShanghai AI Lab · 15B 61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
Nemotron Nano 9B v2NVIDIA · 8.9B 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 12.7 tok/s Runs well 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 34.2 tok/s Runs great
MiniCPM-o 2.6 8BOpenBMB · 8.7B 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 10.2 tok/s Runs well 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 35.1 tok/s Runs great
Phi-3.5 Mini InstructMicrosoft · 3.8B 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 18.0 tok/s Runs well 40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs great
Magistral Small 1.2 (24B)Mistral AI · 24B 60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 3.7 tok/s Slow 60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 16.4 tok/s Runs welltight fit
Olmo 3 7B InstructAi2 · 7.3B 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 12.2 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 41.7 tok/s Runs great
Pixtral 12B (2409)Mistral AI · 12B 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 5.5 tok/s Slow 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 24.6 tok/s Runs well
Phi-4 Multimodal InstructMicrosoft · 5.6B 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 12.3 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Granite 4.0 H MicroIBM · 3.2B 36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 21.5 tok/s Runs well 38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 48.6 tok/s Runs great
Falcon-H1 7B InstructTII · 7.6B 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 11.7 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great
Qwen2.5 7B Instruct 1MAlibaba · 7.6B 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 11.6 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 40.0 tok/s Runs great
Baichuan M2 32BBaichuan · 33B 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 13.4 tok/s Runs welloffloads to RAM
Qwen2.5 32B InstructAlibaba · 33B 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 13.4 tok/s Runs welloffloads to RAM
Granite 4.0 H 1BIBM · 1.5B 27Basic quality (D) · 27 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 38.8 tok/s Runs great 28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 48.9 tok/s Runs great
GLiDE 9BFastino · 7.9B 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 11.2 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 38.3 tok/s Runs great
Foundation-Sec 1.1 8B InstructCisco Foundation AI · 8.0B 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
InternVL2.5 8BShanghai AI Lab · 8.1B 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 37.7 tok/s Runs great
GLM-4.1V 9B ThinkingZhipu AI · 10B 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 29.6 tok/s Runs well
LFM2 2.6BLiquid AI · 2.6B 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 26.7 tok/s Runs well 34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 60.3 tok/s Runs great
Qwen3.5 0.8BAlibaba · 0.9B 26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 40.2 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 82.1 tok/s Runs great
Qwen3 1.7BAlibaba · 2.0B 28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 33.8 tok/s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 76.3 tok/s Runs great
InternLM3 8B InstructShanghai AI Lab · 8.8B 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 10.1 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 34.6 tok/s Runs great
LFM2 1.2BLiquid AI · 1.2B 25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 40.7 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 61.0 tok/s Runs great
Falcon-H1 3B InstructTII · 3.2B 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 21.8 tok/s Runs well 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 49.2 tok/s Runs great
Qwen2.5-VL 32B InstructAlibaba · 33B 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 12.6 tok/s Runs welloffloads to RAM
DeepSeek R1 0528 Qwen3 8BDeepSeek · 8.2B 52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 10.8 tok/s Runs well 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 37.2 tok/s Runs great
InternLM2.5 20B ChatShanghai AI Lab · 20B 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 15.3 tok/s Runs welltight fit
Codestral 22BMistral AI · 22B 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 17.7 tok/s Runs welltight fit
AFM 4.5BArcee AI · 4.6B 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 14.9 tok/s Runs well 39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 45.7 tok/s Runs great
OLMoE 1B-7B InstructAi2 · 6.9B (1.3B active) 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 44.8 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 82.0 tok/s Runs great
Ministral 3 8BMistral AI · 8.9B 39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 12.6 tok/s Runs well 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 34.1 tok/s Runs great
Hermes 3 (Llama 3.1 8B)Nous Research · 8.0B 40Fair quality (C) · 40 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Llama 3.1 8B InstructMeta · 8.0B 40Fair quality (C) · 40 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
MiniCPM-V 2.6 8BOpenBMB · 8.1B 40Fair quality (C) · 40 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 37.6 tok/s Runs great
Falcon-H1 1.5B InstructTII · 1.6B 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 44.3 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 46.1 tok/s Runs great
Phi-3.5 Vision InstructMicrosoft · 4.2B 36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 16.5 tok/s Runs well 37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 50.9 tok/s Runs great
MiniCPM3 4BOpenBMB · 4.1B 36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 16.9 tok/s Runs well 37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 51.9 tok/s Runs great
Llama 3.2 3B InstructMeta · 3.2B 33Basic quality (D) · 33 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 21.4 tok/s Runs well 35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 48.3 tok/s Runs great
Qwen2.5 3B InstructAlibaba · 3.1B 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 22.2 tok/s Runs well 34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 50.2 tok/s Runs great
EXAONE 3.5 2.4BLG AI Research · 2.4B 25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 36.8 tok/s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 64.3 tok/s Runs great
Falcon 3 7B InstructTII · 7.5B 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 11.9 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 40.8 tok/s Runs great
Qwen2.5-Math 72B InstructAlibaba · 73B 57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 0.7 tok/s Very slow 40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking. 2.8 tok/s Very slowheavy RAM offload
Phi-3 Mini 128K InstructMicrosoft · 3.8B 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 18.0 tok/s Runs well 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs great
Phi-3 Mini 4K InstructMicrosoft · 3.8B 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 18.0 tok/s Runs well 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs great
EXAONE 3.5 7.8BLG AI Research · 7.8B 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 11.3 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great
Qianfan-OCR 4BBaidu · 4.7B 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 14.5 tok/s Runs well 49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 44.6 tok/s Runs great
Ministral 8B (2410)Mistral AI · 8.0B 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 11.1 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 38.0 tok/s Runs great
DeepHermes 3 Llama 3 8B PreviewNous Research · 8.0B 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Dolphin 3.0 Llama 3.1 8BCognitive Computations · 8.0B 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Tülu 3 8BAi2 · 8.0B 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Granite 3.2 2B InstructIBM · 2.5B 25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 35.1 tok/s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 61.3 tok/s Runs great
Granite 3.3 2BIBM · 2.5B 25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 35.1 tok/s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 61.3 tok/s Runs great
Granite 3.2 8B InstructIBM · 8.2B 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 10.9 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 37.3 tok/s Runs great
Granite 3.3 8BIBM · 8.2B 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 10.9 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 37.3 tok/s Runs great
UI-Mate 9BTencent · 9.4B 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 12.0 tok/s Runs well 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 32.4 tok/s Runs great
UI-Venus 2 9BAnt Group (inclusionAI) · 9.4B 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 12.0 tok/s Runs well 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 32.4 tok/s Runs great
Phi-3 Medium 128K InstructMicrosoft · 14B 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 4.9 tok/s Slow 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 21.8 tok/s Runs well
Phi-3 Medium (14B)Microsoft · 14B 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 4.9 tok/s Slow 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 21.8 tok/s Runs well
Qwen2.5 1.5B InstructAlibaba · 1.5B 22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 44.6 tok/s Runs great 24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 46.4 tok/s Runs great
LFM2-VL 1.6BLiquid AI · 1.6B 22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 43.4 tok/s Runs great 24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 45.2 tok/s Runs great
Falcon-H1 34B InstructTII · 34B 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 14.7 tok/s Runs welloffloads to RAM
Devstral Small 2505Mistral AI · 24B 55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slow 55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 16.7 tok/s Runs welltight fit
Phi-3 Small 8K InstructMicrosoft · 7.4B 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 12.0 tok/s Runs well 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 41.2 tok/s Runs great
S5 nanoMaritaca AI · 4.7B 35Basic quality (D) · 35 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 14.7 tok/s Runs well 36Basic quality (D) · 36 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 45.3 tok/s Runs great
Seed-Coder 8B ReasoningByteDance · 8.3B 48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 10.8 tok/s Runs well 55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 36.9 tok/s Runs great
Qwen2 7B InstructAlibaba · 7.6B 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 11.6 tok/s Runs well 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 40.0 tok/s Runs great
SenseNova 6.8 Code2VideoSenseTime · 27B 55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 14.4 tok/s Runs welltight fit
LightOnOCR-3 4BLightOn · 4.5B 45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 15.1 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 46.5 tok/s Runs great
Phi-4 Mini Reasoning (3.8B)Microsoft · 3.8B 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 17.9 tok/s Runs well 46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 40.4 tok/s Runs great
Qwen2.5-VL 3B InstructAlibaba · 3.8B 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 18.3 tok/s Runs well 34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 41.3 tok/s Runs great
GLM-4 9B 0414Zhipu AI · 9.4B 36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 12.0 tok/s Runs well 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 32.4 tok/s Runs great
Hermes 3 Llama 3.2 3BNous Research · 3.2B 30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 21.4 tok/s Runs well 32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 48.3 tok/s Runs great
NorthCUA Nanobeipo-ai · 9.4B 36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 12.0 tok/s Runs well 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 32.4 tok/s Runs great
Dolphin 2.9.4 Llama 3.1 8BCognitive Computations · 8.0B 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Hermes 2 Pro Llama 3 8BNous Research · 8.0B 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
OpenChat 3.6 8BOpenChat · 8.0B 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
InternVL2 8BShanghai AI Lab · 8.1B 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 37.7 tok/s Runs great
Command R7BCohere · 8.0B 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Granite 3.1 2B InstructIBM · 2.5B 23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 35.1 tok/s Runs great 28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 61.3 tok/s Runs great
LLM4Decompile 22B V2LLM4Binary · 22B 55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slow 48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 17.7 tok/s Runs welltight fit
QwQ 32BAlibaba · 33B 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 13.4 tok/s Runs welloffloads to RAM
Granite 3.1 8B InstructIBM · 8.2B 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 10.9 tok/s Runs well 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 37.3 tok/s Runs great
OLMo 2 13B InstructAi2 · 14B 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 5.0 tok/s Slow 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 22.2 tok/s Runs well
Voxtral Mini 3BMistral AI · 4.7B 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 14.7 tok/s Runs well 35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 45.1 tok/s Runs great
Llama 3.2 1B InstructMeta · 1.2B 21Basic quality (D) · 21 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 45.7 tok/s Runs great 22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 57.6 tok/s Runs great
Command R (08-2024) 32BCohere · 32B 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 15.5 tok/s Runs welltight fit
Qwen2-VL 7B InstructAlibaba · 8.3B 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 10.7 tok/s Runs well 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 36.7 tok/s Runs great
WeVisOne VL 2BTencent · 2.4B 22Basic quality (D) · 22 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 36.4 tok/s Runs great 27Basic quality (D) · 27 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 63.5 tok/s Runs great
Gemma 2 2BGoogle · 2.6B 23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 34.0 tok/s Runs great 28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 59.4 tok/s Runs great
Falcon 3 10B InstructTII · 10B 36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 10.9 tok/s Runs well 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 29.5 tok/s Runs well
Falcon-H1R 7BTII · 7.6B 47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 11.7 tok/s Runs well 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great
Aya Vision 8BCohere · 8.6B 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 10.3 tok/s Runs well 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 35.3 tok/s Runs great
Phi-3 Vision 128K InstructMicrosoft · 4.2B 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 16.5 tok/s Runs well 33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 50.9 tok/s Runs great
EXAONE Deep 7.8BLG AI Research · 7.8B 47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 11.3 tok/s Runs well 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great
Gemma 2 9B ITGoogle · 9.2B 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 12.2 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 32.9 tok/s Runs great
Chandra OCR 2Datalab · 5.3B 45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 12.9 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 39.9 tok/s Runs great
Seed-Coder 8B InstructByteDance · 8.3B 47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 10.8 tok/s Runs well 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 36.9 tok/s Runs great
Hunyuan 1.8B InstructTencent · 1.8B 20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 38.3 tok/s Runs great 22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 86.6 tok/s Runs great
Qwen2-VL 2B InstructAlibaba · 2.2B 20Basic quality (D) · 20 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 70.1 tok/s Runs great
SmolVLM2 2.2B InstructHugging Face · 2.3B 20Basic quality (D) · 20 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 39.4 tok/s Runs great 24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 68.9 tok/s Runs great
Qwen2.5 Coder 7B InstructAlibaba · 7.6B 46Fair quality (C) · 46 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 11.6 tok/s Runs well 52Fair quality (C) · 52 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 40.0 tok/s Runs great
OpenReasoning Nemotron 32BNVIDIA · 33B 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 13.4 tok/s Runs welloffloads to RAM
Skywork OR1 32BSkywork · 33B 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 13.4 tok/s Runs welloffloads to RAM
Granite 3.1 3B-A800M InstructIBM · 3.3B (0.8B active) 19Basic quality (D) · 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 49.0 tok/s Runs great 20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 64.2 tok/s Runs great
Olmo 3.1 32B ThinkAi2 · 32B 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 15.5 tok/s Runs welltight fit
Aya Expanse 32BCohere · 32B 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 15.5 tok/s Runs welltight fit
OLMo 2 32B InstructAi2 · 32B 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 15.5 tok/s Runs welltight fit
Llama 3 8B InstructMeta · 8.0B 35Basic quality (D) · 35 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Aya Vision 32BCohere · 33B 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 15.1 tok/s Runs welltight fit
Command R7B ArabicCohere · 8.0B 35Basic quality (D) · 35 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Gemma 3 1BGoogle · 1.0B 19Basic quality (D) · 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 47.6 tok/s Runs great 20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 71.4 tok/s Runs great
Olmo 3 7B ThinkAi2 · 7.3B 45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 12.2 tok/s Runs well 51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 41.7 tok/s Runs great
Falcon 3 3B InstructTII · 3.2B 25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 27.5 tok/s Runs well 30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 48.0 tok/s Runs great
Mistral-NeMo-Minitron 8B InstructNVIDIA · 8.4B 35Basic quality (D) · 35 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 10.5 tok/s Runs well 39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 36.2 tok/s Runs great
OpenReasoning Nemotron 7BNVIDIA · 7.6B 45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 11.6 tok/s Runs well 51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 40.0 tok/s Runs great
Granite 3.0 2B InstructIBM · 2.6B 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 33.7 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 58.9 tok/s Runs great
SmolLM2 1.7B InstructHugging Face · 1.7B 19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 41.8 tok/s Runs great
Falcon 180B ChatTII · 180B 33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 0.3 tok/s Very slow 23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-4 9B Chat (2024)Zhipu AI · 9.4B 33Basic quality (D) · 33 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 12.0 tok/s Runs well 45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 32.4 tok/s Runs great
GLM-4 9B Chat 1MZhipu AI · 9.5B 33Basic quality (D) · 33 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 11.9 tok/s Runs well 45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 32.1 tok/s Runs great
EXAONE Deep 32BLG AI Research · 32B 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 14.5 tok/s Runs welloffloads to RAM
InternLM2.5 7B ChatShanghai AI Lab · 7.7B 33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 11.5 tok/s Runs well 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 39.3 tok/s Runs great
GLM-Z1 32B (0414)Zhipu AI · 33B 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 15.4 tok/s Runs welltight fit
WeVisOne-VL 4BTencent · 4.8B 42Fair quality (C) · 42 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 14.2 tok/s Runs well 44Fair quality (C) · 44 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 43.7 tok/s Runs great
SmolVLM 2.2B InstructHugging Face · 2.3B 18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 39.4 tok/s Runs great 22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 68.9 tok/s Runs great
Aya Expanse 8BCohere · 8.0B 33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Apertus 8B InstructSwiss AI · 8.1B 33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 37.8 tok/s Runs great
Granite 3.0 3B-A800M InstructIBM · 3.4B (0.8B active) 17Basic quality (D) · 17 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 49.0 tok/s Runs great 18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 64.2 tok/s Runs great
Granite 3.0 8B InstructIBM · 8.2B 33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 10.9 tok/s Runs well 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 37.3 tok/s Runs great
Apertus v1.5 8BSwiss AI · 8.9B 32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 12.7 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 34.2 tok/s Runs great
LFM2 700MLiquid AI · 0.7B 18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 47.2 tok/s Runs great 18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 96.5 tok/s Runs great
InternLM2 Chat 20BShanghai AI Lab · 20B 39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 35Basic quality (D) · 35 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 19.8 tok/s Runs well
DeepSeek MoE 16B ChatDeepSeek · 16B (2.8B active) 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 28.7 tok/s Runs well 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 79.6 tok/s Runs great
Mistral 7B Instruct v0.3Mistral AI · 7.3B 31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 12.2 tok/s Runs well 36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Granite Vision 3.2 2BIBM · 3.0B 18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 37.8 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 52.0 tok/s Runs great
Falcon3 Mamba 7B InstructTII · 7.3B 31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 12.2 tok/s Runs well 36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 41.9 tok/s Runs great
DeepSeek R1 Distill Qwen 32BDeepSeek · 33B 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 13.4 tok/s Runs welloffloads to RAM
Zamba2 7B InstructZyphra · 7.4B 31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 12.1 tok/s Runs well 36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 41.4 tok/s Runs great
Foundation-Sec 8B ReasoningCisco Foundation AI · 8.0B 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
DeepSeek LLM 67B ChatDeepSeek · 67B 39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 0.7 tok/s Very slow 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 40/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slowheavy RAM offload
EXAONE 3.0 7.8B InstructLG AI Research · 7.8B 31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 11.3 tok/s Runs well 36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great
Dolphin 2.9 Llama 3 8BCognitive Computations · 8.0B 31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Command R 35B (v01)Cohere · 35B 37Basic quality (D) · 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 37Basic quality (D) · 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking. 4.8 tok/s Slowheavy RAM offload
Qwen1.5 MoE A2.7B ChatAlibaba · 14B (2.7B active) 20Basic quality (D) · 20 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 29.7 tok/s Runs well 23Basic quality (D) · 23 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 70.2 tok/s Runs great
Qwen2.5-Omni 7BAlibaba · 11B 32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 10.5 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 28.4 tok/s Runs well
OpenHands LM 32BAll Hands AI · 33B 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 13.4 tok/s Runs welloffloads to RAM
OLMo 2 7B InstructAi2 · 7.3B 30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 12.2 tok/s Runs well 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 41.7 tok/s Runs great
Hermes 2 Pro Mistral 7BNous Research · 7.2B 30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 12.3 tok/s Runs well 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Starling LM 7B BetaNexusflow · 7.2B 30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 12.3 tok/s Runs well 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Zamba2 2.7B InstructZyphra · 2.7B 18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 33.4 tok/s Runs great 22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 58.3 tok/s Runs great
InternLM2.5 1.8B ChatShanghai AI Lab · 1.9B 17Basic quality (D) · 17 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 36.3 tok/s Runs great 18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 82.0 tok/s Runs great
DeepSeek R1 Distill Llama 8BDeepSeek · 8.0B 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Falcon 3 1B InstructTII · 1.7B 15Basic quality (D) · 15 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 41.1 tok/s Runs great 16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 42.8 tok/s Runs great
Qwen2 1.5B InstructAlibaba · 1.5B 15Basic quality (D) · 15 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 44.6 tok/s Runs great 16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 46.4 tok/s Runs great
Spark X2.6 1.7BSparkLLM · 1.7B 15Basic quality (D) · 15 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 41.8 tok/s Runs great
Fara 7BMicrosoft · 8.3B 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 10.7 tok/s Runs well 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 36.7 tok/s Runs great
OpenChat 3.5 (0106)OpenChat · 7.2B 29Basic quality (D) · 29 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 12.3 tok/s Runs well 33Basic quality (D) · 33 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
GLM-Z1 9B (0414)Zhipu AI · 9.4B 40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 12.0 tok/s Runs well 55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 32.4 tok/s Runs great
Qwen2.5 Coder 3B InstructAlibaba · 3.1B 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 22.2 tok/s Runs well 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 50.2 tok/s Runs great
GLiDE 2BFastino · 1.9B 16Basic quality (D) · 16 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 36.5 tok/s Runs great 17Basic quality (D) · 17 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 82.4 tok/s Runs great
CogVLM2 Llama 3 Chat 19BZhipu AI · 20B 36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 20.2 tok/s Runs welltight fit
Qwen3 0.6BAlibaba · 0.8B 14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 95.2 tok/s Runs great
Mistral 7B Instruct v0.2Mistral AI · 7.2B 28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 12.3 tok/s Runs well 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Nous Hermes 2 Mistral 7B DPONous Research · 7.2B 28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 12.3 tok/s Runs well 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
OpenChat 3.5 (1210)OpenChat · 7.2B 28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 12.3 tok/s Runs well 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Yi 1.5 9B Chat01.AI · 8.8B 30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 10.0 tok/s Runs well 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 34.5 tok/s Runs great
Qwen2.5-Omni 3BAlibaba · 5.5B 28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 12.4 tok/s Runs well 29Basic quality (D) · 29 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 45.4 tok/s Runs great
LFM2.5 350MLiquid AI · 0.4B 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 46.0 tok/s Runs great 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 204 tok/s Runs great
Falcon-H1 0.5B InstructTII · 0.5B 14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 67.2 tok/s Runs great 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 137 tok/s Runs great
Qwen2.5 Coder 32B InstructAlibaba · 33B 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 13.4 tok/s Runs welloffloads to RAM
Aya 23 35BCohere · 35B 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking. 4.8 tok/s Slowheavy RAM offload
Llama 3.2 11B Vision InstructMeta · 11B 29Basic quality (D) · 29 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 10.6 tok/s Runs well 39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 28.5 tok/s Runs well
Qwen2.5 Coder 1.5B InstructAlibaba · 1.5B 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 44.6 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 46.4 tok/s Runs great
SmolLM 1.7B InstructHugging Face · 1.7B 13Basic quality (D) · 13 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 41.8 tok/s Runs great
OLMo 2 1B InstructAi2 · 1.5B 13Basic quality (D) · 13 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 46.4 tok/s Runs great 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 48.3 tok/s Runs great
Aya 23 8BCohere · 8.0B 28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
QwQ 32B PreviewAlibaba · 33B 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 13.4 tok/s Runs welloffloads to RAM
DeepScaleR 1.5B PreviewAgentica · 1.8B 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 38.6 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great
Llama 2 70B ChatMeta · 69B 35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 0.7 tok/s Very slow 25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slowheavy RAM offload
Yi 1.5 34B Chat01.AI · 34B 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 14.1 tok/s Runs welloffloads to RAM
DeepSeek R1 Distill Qwen 7BDeepSeek · 7.6B 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 11.6 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 40.0 tok/s Runs great
OpenHermes 2.5 Mistral 7BTeknium · 7.2B 26Basic quality (D) · 26 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 12.3 tok/s Runs well 30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Starling LM 7B AlphaBerkeley · 7.2B 26Basic quality (D) · 26 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 12.3 tok/s Runs well 30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Nous Hermes 2 SOLAR 10.7BNous Research · 11B 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 10.5 tok/s Runs well 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 28.4 tok/s Runs well
SOLAR 10.7B InstructUpstage · 11B 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 10.5 tok/s Runs well 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 28.4 tok/s Runs well
Qwen1.5 14B ChatAlibaba · 14B 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 4.8 tok/s Slow 28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 27.8 tok/s Runs welltight fit
LFM2 350MLiquid AI · 0.4B 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 46.0 tok/s Runs great 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 204 tok/s Runs great
Granite 4.0 H 350MIBM · 0.3B 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 47.4 tok/s Runs great 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 210 tok/s Runs great
LFM2-VL 450MLiquid AI · 0.5B 12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 77.6 tok/s Runs great 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 159 tok/s Runs great
StableLM Zephyr 3BStability AI · 2.8B 11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 40.3 tok/s Runs great 16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 55.3 tok/s Runs great
Yi 1.5 6B Chat01.AI · 6.1B 26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 11.3 tok/s Runs well 27Basic quality (D) · 27 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 41.5 tok/s Runs great
Phi-2 2.7BMicrosoft · 2.8B 11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 40.5 tok/s Runs great 16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 55.7 tok/s Runs great
GLiDE 0.8BFastino · 0.8B 11Basic quality (D) · 11 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great 11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 95.2 tok/s Runs great
Mistral 7B Instruct v0.1Mistral AI · 7.2B 25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 12.3 tok/s Runs well 28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Falcon Mamba 7B InstructTII · 7.3B 25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 12.2 tok/s Runs well 28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 41.9 tok/s Runs great
SmolVLM 500M InstructHugging Face · 0.5B 10Basic quality (D) · 10 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 68.5 tok/s Runs great 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 140 tok/s Runs great
Qwen2.5 0.5B InstructAlibaba · 0.5B 10Basic quality (D) · 10 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 71.3 tok/s Runs great 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 146 tok/s Runs great
LFM2.5 230MLiquid AI · 0.2B 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 70.0 tok/s Runs great 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 311 tok/s Runs great
Code Llama 70B InstructMeta · 69B 43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 0.7 tok/s Very slow 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 36/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slowheavy RAM offload
StableLM 2 1.6B ChatStability AI · 1.6B 9Basic quality (D) · 9 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 41.8 tok/s Runs great 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 43.6 tok/s Runs great
StableLM 2 Zephyr 1.6BStability AI · 1.6B 9Basic quality (D) · 9 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 41.8 tok/s Runs great 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 43.6 tok/s Runs great
Nemotron Mini 4B InstructNVIDIA · 4.2B 15Basic quality (D) · 15 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 26.9 tok/s Runs well 21Basic quality (D) · 21 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 50.4 tok/s Runs great
DeepSeek R1 Distill Qwen 1.5BDeepSeek · 1.8B 20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 38.6 tok/s Runs great 22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great
Neural Chat 7B v3.3Intel · 7.2B 23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 12.3 tok/s Runs well 26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Qwen1.5 32B ChatAlibaba · 33B 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 13.8 tok/s Runs welloffloads to RAM
RecurrentGemma 9B ITGoogle · 9.6B 23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 11.7 tok/s Runs well 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 31.6 tok/s Runs great
DeepSeek Prover V2 7BDeepSeek · 6.9B 33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 12.8 tok/s Runs well 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 44.1 tok/s Runs great
LLaVA 1.6 Mistral 7BLLaVA · 7.6B 23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 11.7 tok/s Runs well 26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 40.2 tok/s Runs great
LLaVA 1.6 34BLLaVA · 35B 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slow 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 13.6 tok/s Runs welloffloads to RAM
Gemma 2B ITGoogle · 2.5B 8Basic quality (D) · 8 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 44.9 tok/s Runs great 12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 61.7 tok/s Runs great
Hunyuan 0.5B InstructTencent · 0.5B 8Basic quality (D) · 8 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 64.7 tok/s Runs great 8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 132 tok/s Runs great
SmolLM2 360M InstructHugging Face · 0.4B 8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 44.7 tok/s Runs great 8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 198 tok/s Runs great
Gemma 1.1 7B ITGoogle · 8.5B 23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 10.4 tok/s Runs well 26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 35.6 tok/s Runs great
Qwen2.5-Math 1.5B InstructAlibaba · 1.5B 19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 44.6 tok/s Runs great 20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 46.4 tok/s Runs great
Zephyr 7B BetaHugging Face · 7.2B 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 12.3 tok/s Runs well 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Qwen1.5 7B ChatAlibaba · 7.7B 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 11.5 tok/s Runs well 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 39.4 tok/s Runs great
d1 2BLiquid AI · 1.9B 19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 35.7 tok/s Runs great 21Basic quality (D) · 21 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 80.7 tok/s Runs great
StableLM 2 12B ChatStability AI · 12B 26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 5.7 tok/s Slow 23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 32.4 tok/s Runs great
Orca 2 13BMicrosoft · 13B 26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 5.3 tok/s Slow 23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 30.2 tok/s Runs great
CodeGeeX4 All 9BZhipu AI · 9.4B 32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 12.0 tok/s Runs well 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 32.4 tok/s Runs great
Yi Coder 9B Chat01.AI · 8.8B 33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 10.0 tok/s Runs well 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 34.5 tok/s Runs great
Gemma 7B ITGoogle · 8.5B 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 10.4 tok/s Runs well 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 35.6 tok/s Runs great
ERNIE 4.5 0.3BBaidu · 0.4B 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 44.7 tok/s Runs great 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 198 tok/s Runs great
SmolVLM 256M InstructHugging Face · 0.3B 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 61.9 tok/s Runs great 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 275 tok/s Runs great
Qwen2 0.5B InstructAlibaba · 0.5B 6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 71.3 tok/s Runs great 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 146 tok/s Runs great
Gemma 3 270MGoogle · 0.3B 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 59.6 tok/s Runs great 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 265 tok/s Runs great
TinyLlama 1.1B ChatTinyLlama · 1.1B 5Basic quality (D) · 5 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 43.3 tok/s Runs great 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 64.9 tok/s Runs great
Yi 34B Chat01.AI · 34B 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 14.1 tok/s Runs welloffloads to RAM
StarCoder2 15B InstructBigCode · 16B 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 4.3 tok/s Slow 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 19.1 tok/s Runs well
DeepSeek Coder 6.7B InstructDeepSeek · 6.7B 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 10.2 tok/s Runs well 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 45.2 tok/s Runs great
SmolLM 360M InstructHugging Face · 0.4B 5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 44.7 tok/s Runs great 5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 198 tok/s Runs great
Orca 2 7BMicrosoft · 6.7B 20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 10.2 tok/s Runs well 20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 45.2 tok/s Runs great
InternLM2 Math Plus 20BShanghai AI Lab · 20B 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slow 33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 19.8 tok/s Runs well
LLM4Decompile 9B V2LLM4Binary · 8.8B 31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 10.0 tok/s Runs well 36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 34.5 tok/s Runs great
Llama 2 13B ChatMeta · 13B 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 5.3 tok/s Slow 18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great
Codestral Mamba 7BMistral AI · 7.3B 30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 12.2 tok/s Runs well 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 41.8 tok/s Runs great
Baichuan 2 13B ChatBaichuan · 14B 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 4.9 tok/s Slow 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 28.3 tok/s Runs well
Yi Coder 1.5B Chat01.AI · 1.5B 15Basic quality (D) · 15 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 46.4 tok/s Runs great 16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 48.3 tok/s Runs great
Qwen2.5-Math 7B InstructAlibaba · 7.6B 30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 11.6 tok/s Runs well 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 40.0 tok/s Runs great
SmolLM2 135M InstructHugging Face · 0.1B 4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 124 tok/s Runs great 4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 549 tok/s Runs great
Yi 6B Chat01.AI · 6.1B 16Basic quality (D) · 16 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 14.6 tok/s Runs well 19Basic quality (D) · 19 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 41.5 tok/s Runs great
surya ocr 2.5Datalab · 0.7B 15Basic quality (D) · 15 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 50.6 tok/s Runs great 15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 104 tok/s Runs great
LLM4Decompile 6.7B V2LLM4Binary · 6.7B 30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 10.2 tok/s Runs well 30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 45.2 tok/s Runs great
Stable Code Instruct 3BStability AI · 2.8B 14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 40.3 tok/s Runs great 20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 55.3 tok/s Runs great
Llama 2 7B ChatMeta · 6.7B 16Basic quality (D) · 16 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 13.2 tok/s Runs well 19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 45.2 tok/s Runs great
Mathstral 7BMistral AI · 7.3B 28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 12.2 tok/s Runs well 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Falcon-H1 Tiny 90M InstructTII · 0.1B 3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 179 tok/s Runs great 3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 794 tok/s Runs great
Baichuan 2 7B ChatBaichuan · 7.5B 16Basic quality (D) · 16 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 11.8 tok/s Runs well 19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 40.5 tok/s Runs great
SmolLM 135M InstructHugging Face · 0.1B 2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 124 tok/s Runs great 2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 549 tok/s Runs great
CodeGemma 7BGoogle · 8.5B 28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 10.4 tok/s Runs well 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 35.6 tok/s Runs great
SQLCoder 7B v2Defog · 6.7B 28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 10.2 tok/s Runs well 28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 45.2 tok/s Runs great
DeepSeek LLM 7B ChatDeepSeek · 6.9B 15Basic quality (D) · 15 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 12.8 tok/s Runs well 17Basic quality (D) · 17 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 44.1 tok/s Runs great
WizardLM 13B v1.2WizardLM · 13B 20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 5.3 tok/s Slow 15Basic quality (D) · 15 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great
Granite Code 20B InstructIBM · 20B 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 19.6 tok/s Runs well
InternLM2 Math Plus 7BShanghai AI Lab · 7.7B 26Basic quality (D) · 26 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 11.5 tok/s Runs well 30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 39.3 tok/s Runs great
DeepSeek Coder 1.3B InstructDeepSeek · 1.4B 11Basic quality (D) · 11 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 52.9 tok/s Runs great
LLM4Decompile 1.3B V2LLM4Binary · 1.4B 11Basic quality (D) · 11 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 52.9 tok/s Runs great
LLaVA 1.5 13BLLaVA · 13B 20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 5.1 tok/s Slow 15Basic quality (D) · 15 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 37.5 tok/s Runs great
Granite Code 8B Instruct (128K)IBM · 8.1B 26Basic quality (D) · 26 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 11.0 tok/s Runs well 30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 37.8 tok/s Runs great
StarCoder2 3BBigCode · 3.0B 12Basic quality (D) · 12 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 37.2 tok/s Runs great 18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 51.1 tok/s Runs great
Granite Code 3B Instruct (128K)IBM · 3.5B 14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking. 32.4 tok/s Runs great 20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 44.5 tok/s Runs great
LLaVA 1.5 7BLLaVA · 7.1B 11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking. 16.0 tok/s Runs well 15Basic quality (D) · 15 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great
Vicuna 13B v1.5LMSYS · 13B 19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking. 5.3 tok/s Slow 14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great
CodeGemma 2BGoogle · 2.5B 9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking. 44.9 tok/s Runs great 14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 61.7 tok/s Runs great
Vicuna 7B v1.5LMSYS · 6.7B 9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking. 16.7 tok/s Runs well 13Basic quality (D) · 13 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 45.2 tok/s Runs great
OLMo 7B InstructAi2 · 6.9B 9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking. 16.4 tok/s Runs well 13Basic quality (D) · 13 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 44.2 tok/s Runs great
StarCoder2 15B (base)BigCode · 16B 30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking. 4.3 tok/s Slow 30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 19.1 tok/s Runs well
Code Llama 13B InstructMeta · 13B 28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking. 5.3 tok/s Slow 21Basic quality (D) · 21 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great
DeepSeek Coder 33B InstructDeepSeek · 33B 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slow 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 12.9 tok/s Runs welloffloads to RAM
Qwen2.5 Coder 0.5B InstructAlibaba · 0.5B 8Basic quality (D) · 8 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking. 71.3 tok/s Runs great 8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 146 tok/s Runs great
WizardMath 7B v1.1WizardLM · 7.2B 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking. 12.3 tok/s Runs well 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
InternLM 7BShanghai AI Lab · 7.3B 8Basic quality (D) · 8 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking. 15.4 tok/s Runs well 11Basic quality (D) · 11 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 41.6 tok/s Runs great
Code Llama 7B InstructMeta · 6.7B 22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking. 10.2 tok/s Runs well 22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 45.2 tok/s Runs great
Granite Code 34B InstructIBM · 34B 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 14.8 tok/s Runs welltight fit
StarCoder2 7BBigCode · 7.2B 20Basic quality (D) · 20 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking. 12.4 tok/s Runs well 22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 42.5 tok/s Runs great
Code Llama 34B InstructMeta · 34B 26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slow 26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking. 13.4 tok/s Runs welloffloads to RAM
StarCoder 15BBigCode · 16B 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking. 4.3 tok/s Slow 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 24.9 tok/s Runs well
Falcon 7B InstructTII · 7.2B 4Basic quality (D) · 4 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking. 15.6 tok/s Runs well 5Basic quality (D) · 5 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 42.2 tok/s Runs great
Falcon 40B InstructTII · 42B 15Basic quality (D) · 15 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking. 1.6 tok/s Very slow 11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking. 8.6 tok/s Slowoffloads to RAM
Kimi K3.1 Pro (2.8T)Moonshot AI · 2,780B (104.0B active) 69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3.8 Max (2.4T-A95B)Alibaba · 2,446B (95.0B active) 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K3 (2.8T)Moonshot AI · 2,780B (104.0B active) 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V4 Pro (1.6T-A49B)DeepSeek · 1,651B (49.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V4 Pro (April 2026)DeepSeek · 1,599B (49.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run

Image models

60 of 60 image models run on the Intel Xeon Silver (4100 to 4300 series) NAS (512 GB). 471 GB usable for models.

55 of 60 image models run on the AMD Radeon RX 7600 XT. 16.0 GB usable for models.

Model QualityXeon Silver (4100 to 4300 series) NAS SpeedXeon Silver (4100 to 4300 series) NAS RunsXeon Silver (4100 to 4300 series) NAS QualityRX 7600 XT SpeedRX 7600 XT RunsRX 7600 XT
HunyuanImage 3.0 (80B-A13B)Tencent · 83.0B 90Excellent quality (A+) · 90 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 49.3 min Very slow 88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Cosmos3-Super Text2Image (64B)NVIDIA · 64.0B 92Excellent quality (A+) · 92 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 32.9 min Very slow 90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
FLUX.2 [dev]Black Forest Labs · 32.2B 92Excellent quality (A+) · 92 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 29.6 min Very slow 90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 1.7 min Slowtight fit · CPU offload
Krea 2 TurboKrea · 12.8B 92Excellent quality (A+) · 92 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 3.5 min Very slow 92Excellent quality (A+) · 92 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 13 s Runs greatCPU offload
Ideogram 4Ideogram · 18.6B 92Excellent quality (A+) · 92 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 20.5 min Very slow 85Excellent quality (A+) · 85 of 100: how capable the model is at NF4, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 68 s Slowencoder in RAM
Qwen-Image 2.1Alibaba · 7.1B 93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 49.3 min Very slow 86Excellent quality (A+) · 86 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
FLUX 3 Image KleinBlack Forest Labs · 5.6B 91Excellent quality (A+) · 91 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 4.7 min Very slow 89Excellent quality (A+) · 89 of 100: how capable the model is at FP8, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 12 s Runs great
FLUX.2 [klein] 9BBlack Forest Labs · 9.1B 84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 89 s Slow 83Great quality (A) · 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 5.1 s Runs greatencoder in RAM
Fibo 2.0 DevBria AI · 8.1B 93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 18.5 min Very slow 93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 69 s SlowCPU offload
Qwen-Image 2512Alibaba · 20.4B 90Excellent quality (A+) · 90 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 74.0 min Very slow 88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 4.1 min Very slowCPU offload
Qwen-Image 2.1 TurboAlibaba · 7.1B 89Excellent quality (A+) · 89 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 9.5 min Very slow 82Great quality (A) · 82 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ming-Image 0.1 DesignAnt Group (inclusionAI) · 6.2B 86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 13.4 min Very slow 84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen-ImageAlibaba · 20.4B 88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 74.0 min Very slow 86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 4.1 min Very slowCPU offload
ERNIE-Image TurboBaidu · 8.0B 85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 2.6 min Very slow 84Great quality (A) · 84 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 8.7 s Runs welltight fit
Z-Image TurboAlibaba (Tongyi) · 6.2B 84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 2.1 min Slow 83Great quality (A) · 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 6.8 s Runs great
FireRed-Image-Edit 1.1rednote (FireRed Team) · 20.4B 87Excellent quality (A+) · 87 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 65.8 min Very slow 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 4.7 min Very slowCPU offload
FLUX.2 [klein] 4BBlack Forest Labs · 3.9B 78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 49 s Slow 77Great quality (A) · 77 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 2.7 s Runs great
ERNIE-ImageBaidu · 8.0B 88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 18.1 min Very slow 81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 62 s Slowtight fit
Qwen-Image-Edit 2511Alibaba · 20.4B 86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 65.8 min Very slow 85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 4.7 min Very slowCPU offload
HiDream-I1 FullHiDream.ai · 17.1B 84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 37.0 min Very slow 78Great quality (A) · 78 of 100: how capable the model is at NF4, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 2.0 min Slowencoder in RAM
HiDream-I1 DevHiDream.ai · 17.1B 80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 14.0 min Very slow 74Great quality (A) · 74 of 100: how capable the model is at NF4, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 46 s Slowencoder in RAM
FLUX.1 Krea [dev]Black Forest Labs · 11.9B 82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 9.9 min Very slow 76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 34 s Runs welltight fit
FLUX.1 Kontext [dev]Black Forest Labs · 11.9B 82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 18.1 min Very slow 76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 62 s Slowtight fit
MOSS-Image SFTOpenMOSS · 5.1B 82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 11.5 min Very slow 82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 35 s Runs wellencoder in RAM
FLUX.1 [dev]Black Forest Labs · 11.9B 80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 9.9 min Very slow 74Great quality (A) · 74 of 100: how capable the model is at NF4, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 31 s Runs welltight fit
Z-ImageAlibaba (Tongyi) · 6.2B 82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 14.0 min Very slow 80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 36 s Runs well
HiDream-I1 FastHiDream.ai · 17.1B 76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 8.2 min Very slow 70Great quality (A) · 70 of 100: how capable the model is at NF4, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 27 s Runs wellencoder in RAM
Chroma1-HDLodestone Rock · 8.9B 78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 16.4 min Very slow 72Great quality (A) · 72 of 100: how capable the model is at NF4, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 52 s Slow
FLUX.1 [schnell]Black Forest Labs · 11.9B 70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 1.6 min Slow 65Good quality (B) · 65 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 5.7 s Runs welltight fit
FLUX.1 Fill [dev]Black Forest Labs · 11.9B 76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 10.3 min Very slow 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 28 s Runs welltight fit · encoder in RAM
Iris-3BSperidlabs · 3.0B 72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 13.2 min Very slow 72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 40 s Runs wellencoder in RAM
CogView4 6BZhipu AI · 6.4B 70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 14.8 min Very slow 68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 40 s Slowencoder in RAM
Stable Diffusion 3.5 Large TurboStability AI · 8.1B 62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 79 s Slow 57Good quality (B) · 57 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 4.5 s Runs great
SDXL TurboStability AI · 2.6B 40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 7.4 s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 0.4 s Runs great
Stable Diffusion 3.5 LargeStability AI · 8.1B 66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 9.0 min Very slow 61Good quality (B) · 61 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 31 s Runs well
SDXL-Lightning (4-step)ByteDance · 2.6B 54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 35 s Runs well 54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 1.7 s Runs great
DreamShaper XL v2 TurboLykon · 2.6B 58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 64 s Slow 58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 3.1 s Runs great
OmniGen2BAAI (VectorSpaceLab) · 4.0B 66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 13.2 min Very slow 64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 34 s Runs well
Juggernaut XL v9RunDiffusion · 2.6B 62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 3.7 min Very slow 62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 11 s Runs great
RealVisXL V5.0SG161222 · 2.6B 62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 3.7 min Very slow 62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 11 s Runs great
KolorsKuaishou · 2.6B 60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 3.3 min Very slow 58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 8.5 s Runs great
Lumina-Image 2.0Shanghai AI Lab · 2.6B 62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 5.8 min Very slow 60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 15 s Runs well
Animagine XL 4.0Cagliostro Lab · 2.6B 60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.5 min Very slow 60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 9.9 s Runs great
Illustrious XL v0.1OnomaAI · 2.6B 60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.5 min Very slow 60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 9.9 s Runs great
Pony Diffusion V6 XLPurpleSmartAI · 2.6B 58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.1 min Very slow 58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 9.0 s Runs great
Playground v2.5Playground AI · 2.6B 60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 6.2 min Very slow 58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 16 s Runs well
Stable Diffusion 3.5 MediumStability AI · 2.5B 58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 4.9 min Very slow 56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 13 s Runs great
Animagine XL 3.1Cagliostro Lab · 2.6B 56Good quality (B) · 56 of 100: how capable the model is at FP16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 3.5 min Very slow 56Good quality (B) · 56 of 100: how capable the model is at FP16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 9.9 s Runs great
AuraFlow v0.3fal · 6.9B 58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 11.5 min Very slow 56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 30 s Runs well
PixArt-Σ XL 1024PixArt · 0.6B 50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 74 s Slow 50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 3.5 s Runs great
Stable Diffusion XL 1.0Stability AI · 2.6B 55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 3.7 min Very slow 55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 11 s Runs great
Stable Diffusion 3 MediumStability AI · 2.0B 52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 2.5 min Very slow 50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 6.6 s Runs great
SD TurboStability AI · 0.9B 28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 3.9 s Runs great 28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 0.2 s Runs great
LCM DreamShaper v7Tsinghua (LCM) · 0.9B 30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 12 s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 0.6 s Runs great
Stable CascadeStability AI · 5.2B 52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 4.1 min Very slow 50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 11 s Runs great
Hunyuan-DiT v1.2Tencent · 1.5B 52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 4.9 min Very slow 52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 14 s Runs well
Realistic Vision V5.1SG161222 · 0.9B 42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 59 s Slow 42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 2.8 s Runs great
Kandinsky 3Sber AI · 3.0B 48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 7.4 min Very slow 48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 22 s Runs wellencoder in RAM
DreamShaper 8Lykon · 0.9B 40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 59 s Slow 40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 2.8 s Runs great
Stable Diffusion 1.5Stability AI / Runway · 0.9B 35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 59 s Slow 35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 2.8 s Runs great

Video models

43 of 43 video models run on the Intel Xeon Silver (4100 to 4300 series) NAS (512 GB). 471 GB usable for models.

17 of 43 video models run on the AMD Radeon RX 7600 XT. 16.0 GB usable for models.

Model QualityXeon Silver (4100 to 4300 series) NAS SpeedXeon Silver (4100 to 4300 series) NAS RunsXeon Silver (4100 to 4300 series) NAS QualityRX 7600 XT SpeedRX 7600 XT RunsRX 7600 XT
Cosmos3-Super Image2Video (64B)NVIDIA · 64.0B 88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 30.8 h Very slow 88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax H3 (33B, video + audio)MiniMax · 20.1B 90Excellent quality (A+) · 90 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 14.2 h Very slow 83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LTX-2.5 (22B, video + audio)Lightricks · 21.0B 88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 74.8 min Very slow 81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kandinsky 6.0 Pro (29B, video + audio)Sber AI · 30.1B 88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 40.3 h Very slow 88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LTX-2.3Lightricks · 18.9B 86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 5.5 h Very slow 79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.2 I2V A14BAlibaba · 14.0B 86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 21.9 h Very slow 79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.2 T2V A14BAlibaba · 14.0B 86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 21.9 h Very slow 79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LTX-2 (19B, video + audio)Lightricks · 18.9B 82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 5.5 h Very slow 76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Bernini-R A14B (video generation + editing)ByteDance · 14.3B 85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 9.9 h Very slow 83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 33.2 min SlowCPU offload
HunyuanVideo 1.5 (8.3B)Tencent · 8.3B 84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 24.7 h Very slow 82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
SCAIL-2 14B (character animation)Zhipu AI · 16.4B 80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 16.4 h Very slow 78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 55.3 min Very slowCPU offload
Wan 2.2 Animate 14BAlibaba · 17.3B 78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 26.0 h Very slow 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.1 VACE 14BAlibaba · 17.3B 78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 30.1 h Very slow 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
SkyReels V2 T2V 14B 720PSkywork · 14.0B 78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 32.9 h Very slow 76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.1 I2V 14B 720PAlibaba · 14.0B 78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 27.4 h Very slow 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.1 T2V 14BAlibaba · 14.0B 78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 27.4 h Very slow 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Step-Video T2V (30B)StepFun · 30.0B 76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 42.5 h Very slow 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan-Dancer 14B (music to dance)Alibaba · 17.3B 76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 60.3 h Very slow 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.2 S2V 14B (speech to video)Alibaba · 16.3B 76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 13.7 h Very slow 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 46.0 min Very slowCPU offload
HY-OmniWeaving (8.3B, unified video)Tencent · 8.3B 76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 7.3 h Very slow 76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 27.1 min SlowCPU offload
FramePack I2V (HunyuanVideo 13B)lllyasviel · 12.9B 74Great quality (A) · 74 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 5.5 h Very slow 72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 18.4 min SlowCPU offload
LTX-Video 13B 0.9.8 (distilled)Lightricks · 13.0B 68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 49.3 min Very slow 66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.1 I2V 14B 480PAlibaba · 14.0B 74Great quality (A) · 74 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 12.3 h Very slow 68Good quality (B) · 68 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 44.6 min Very slowtight fit · encoder in RAM
SkyReels V2 T2V 14B 540PSkywork · 14.0B 74Great quality (A) · 74 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 19.2 h Very slow 72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 64.5 min Very slowtight fit · CPU offload
HunyuanVideo 13BTencent · 13.0B 72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 32.9 h Very slow 66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
HunyuanVideo I2V 13BTencent · 13.0B 72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 32.9 h Very slow 66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Cosmos-Predict2 14B Video2WorldNVIDIA · 14.0B 70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 21.9 h Very slow 68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.2 TI2V 5BAlibaba · 5.0B 70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 7.1 h Very slow 65Good quality (B) · 65 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Open-Sora 2.0 (11B)HPC-AI Tech · 11.0B 70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 41.1 h Very slow 70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kandinsky 6.0 Video Lite (3B, video + audio)Sber AI · 3.2B 68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 4.2 h Very slow 68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 12.8 min Slowtight fit · encoder in RAM
Mochi 1 Preview (10B)Genmo · 10.0B 62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 15.1 h Very slow 60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 50.7 min Very slowtight fit · CPU offload
Cosmos 1.0 Diffusion 7B Text2WorldNVIDIA · 7.0B 60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 7.5 h Very slow 60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
CogVideoX 1.5 5BZhipu AI · 5.6B 60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 12.3 h Very slow 58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Cosmos-Predict2 2B Video2WorldNVIDIA · 2.0B 58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 3.4 h Very slow 56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LTX-Video 2BLightricks · 1.9B 50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 20.5 min Slow 49Fair quality (C) · 49 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 53 s Runs great
CogVideoX 5B I2VZhipu AI · 5.6B 55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 2.5 h Very slow 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 6.4 min Runs welltight fit
CogVideoX 5BZhipu AI · 5.0B 55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 2.5 h Very slow 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 6.4 min Runs welltight fit
Wan 2.1 VACE 1.3BAlibaba · 2.2B 55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 3.6 h Very slow 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 9.2 min Runs welltight fit
Wan 2.1 T2V 1.3BAlibaba · 1.3B 55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 3.3 h Very slow 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 8.5 min Runs well
SkyReels V2 Diffusion Forcing 1.3BSkywork · 1.4B 52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 4.1 h Very slow 50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 11.2 min Runs wellencoder in RAM
Allegro (2.8B)Rhymes AI · 2.8B 52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 16.4 h Very slow 52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
CogVideoX 2BZhipu AI · 1.7B 45Fair quality (C) · 45 of 100: how capable the model is at FP16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 49.3 min Very slow 44Fair quality (C) · 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 2.1 min Runs great
AnimateDiff v3 (SD 1.5)Shanghai AI Lab · 1.3B 35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 16.4 min Slow 35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 47 s Runs great

Decision models

70 of 70 decision models run on the Intel Xeon Silver (4100 to 4300 series) NAS (512 GB). 471 GB usable for models.

68 of 70 decision models run on the AMD Radeon RX 7600 XT. 16.0 GB usable for models.

Model QualityXeon Silver (4100 to 4300 series) NAS SpeedXeon Silver (4100 to 4300 series) NAS RunsXeon Silver (4100 to 4300 series) NAS QualityRX 7600 XT SpeedRX 7600 XT RunsRX 7600 XT
OpenAI Privacy FilterOpenAI · 1.4B (50M active) 78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 50 ms Runs great 78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 4.6 ms Runs great
Llama Prompt Guard 2 22MMeta · 71M 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 117 ms Runs great 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 5.6 ms Runs great
Jina Reranker v3.5Jina AI · 600M 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 899 ms Slow 86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 23 ms Runs great
GLiNER2.5 DecideFastino · 490M 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 394 ms Runs well 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 9.6 ms Runs great
Jina Reranker v3Jina AI · 600M 84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 899 ms Slow 84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 23 ms Runs great
Qwen3 Reranker 4BAlibaba · 4.0B 90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 6.0 s Very slow 90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. 104 ms Runs great
Qwen3 Reranker 8BAlibaba · 8.2B 90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 12.2 s Very slow 90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 207 ms Runs well
FRIDA-DecisionsSber AI · 820M 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 656 ms Slow 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 13 ms Runs great
GLiClass Base v3Knowledgator · 190M 66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 155 ms Runs well 66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 6.2 ms Runs great
GLiClass ModernBERT Large v3Knowledgator · 400M 74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 322 ms Runs well 74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 8.6 ms Runs great
zerank-2ZeroEntropy · 4.0B 88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 6.0 s Very slow 88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. 104 ms Runs great
Skywork Reward V2 Llama 3.1 8BSkywork · 7.5B 90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 23.9 s Very slow 90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 303 ms Runs well
Llama Prompt Guard 2 86MMeta · 280M 76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 449 ms Slow 76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 10 ms Runs great
DeBERTa v3 Base Zero-shot v2.0Moritz Laurer · 180M 64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 147 ms Runs well 64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 6.1 ms Runs great
GTE Reranker ModernBERT BaseAlibaba · 150M 70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 243 ms Runs well 70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 7.4 ms Runs great
Qwen3 Reranker 0.6BAlibaba · 600M 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 899 ms Slow 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 23 ms Runs great
Shieldstral 1.0 3BMistral AI · 3.9B 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 5.7 s Very slow 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 99 ms Runs great
DeBERTa v3 Large Zero-shot v2.0Moritz Laurer · 440M 72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 354 ms Runs well 72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 9.0 ms Runs great
Intern-Decision 4BShanghai AI Lab · 4.5B 84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 3.5 s Very slow 84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 75 ms Runs great
Granite Embedding Reranker English R2IBM · 150M 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 243 ms Runs well 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 7.4 ms Runs great
GLiNER2 Base v1Fastino · 210M 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 171 ms Runs well 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 6.4 ms Runs great
mxbai-rerank-large v2Mixedbread · 1.5B 82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 2.3 s Very slow 82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 54 ms Runs great
d1 3BLiquid AI · 3.1B 82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 2.4 s Very slow 82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 76 ms Runs great
ModernBERT Large Zero-shot v2.0Moritz Laurer · 400M 70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 322 ms Runs well 70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 8.6 ms Runs great
GLiDE 27BFastino · 26B 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 20.7 s Very slow 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
mxbai-rerank-base v2Mixedbread · 490M 76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 735 ms Slow 76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 20 ms Runs great
zerank-1 smallZeroEntropy · 1.7B 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 2.6 s Very slow 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 60 ms Runs great
Skywork Reward V2 Qwen3 8BSkywork · 7.6B 84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 24.1 s Very slow 84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 305 ms Runs well
GLiGuard 300MFastino · 210M 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 338 ms Runs well 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 8.8 ms Runs great
LayaConvai Innovations · 420M 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 338 ms Runs well 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 8.8 ms Runs great
Jina Reranker v2 Base MultilingualJina AI · 280M 70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 449 ms Slow 70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 10 ms Runs great
ProtectAI DeBERTa v3 Prompt Injection v2Protect AI · 180M 66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 290 ms Runs well 66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 8.1 ms Runs great
LemonDecider 4BAnt Group (inclusionAI) · 4.5B 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 3.6 s Very slow 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 56 ms Runs great
ms-marco MiniLM L6 v2Sentence Transformers · 23M 52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 46 ms Runs great 52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 4.5 ms Runs great
DuoGuard 0.5BDuoGuard · 490M 72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 783 ms Slow 72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 15 ms Runs great
BGE Reranker v2 GemmaBAAI · 2.5B 78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 3.7 s Very slow 78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 85 ms Runs great
BGE Reranker v2 M3BAAI · 570M 72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 910 ms Slow 72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 17 ms Runs great
Qwen3Guard Gen 8BAlibaba · 8.2B 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 13.4 s Very slow 85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 387 ms Runs well
Qwen3Guard Gen 4BAlibaba · 4.4B 84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 7.2 s Very slow 83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 210 ms Runs well
Laya MultilingualConvai Innovations · 320M 60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 258 ms Runs well 60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 7.7 ms Runs great
Llama 3.1 Nemotron Safety Guard 8B v3NVIDIA · 8.0B 84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 13.1 s Very slow 83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 379 ms Runs well
Granite Guardian 4.1 8BIBM · 8.4B 84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 13.7 s Very slow 83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 395 ms Runs well
HHEM 2.1 OpenVectara · 110M 62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 354 ms Runs well 62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 9.0 ms Runs great
Skywork Reward V2 Qwen3 0.6BSkywork · 600M 72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 1.9 s Very slow 72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 31 ms Runs great
GLiNER Multi v2.1Urchade Zaratiana · 290M 62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 235 ms Runs well 62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 7.3 ms Runs great
Qwen3Guard Gen 0.6BAlibaba · 750M 74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 1.2 s Slow 74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 81 ms Runs great
WildGuard 7BAi2 · 7.3B 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 11.9 s Very slow 79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 343 ms Runs well
Granite Guardian 3.3 8BIBM · 8.2B 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 13.4 s Very slow 79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 386 ms Runs well
NuExtract 2.0 4BNuMind · 3.8B 76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 5.3 s Very slow 73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 396 ms Runs well
BART Large MNLIMeta · 410M 56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 330 ms Runs well 56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 8.7 ms Runs great
CompassJudger-2 32BOpenCompass · 33B 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 236.5 s Very slow 81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 30.7 s Very slowoffloads to RAM
NuExtract 2.0 2BNuMind · 2.2B 70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 3.1 s Very slow 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 198 ms Runs well
gpt-oss-safeguard 120BOpenAI · 117B (5.1B active) 80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 19.1 s Very slow 80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LettuceDetect LargeKR Labs · 400M 66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 1.3 s Slow 66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 22 ms Runs great
DynaGuard 8BUniversity of Maryland · 8.2B 79Great quality (A) · 79 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 49.5 s Very slow 79Great quality (A) · 79 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 8.6 s Very slow
Nemotron Content Safety Reasoning 4BNVIDIA · 4.3B 78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 26.0 s Very slow 77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 3.3 s Very slow
Selene 1 Mini 8BAtla · 8.0B 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 58.0 s Very slow 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 8.1 s Very slow
gpt-oss-safeguard 20BOpenAI · 22B (3.6B active) 76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 13.7 s Very slow 76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 2.4 s Very slowtight fit
ShieldGemma 27BGoogle · 27B 66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 40.6 s Very slow 64Good quality (B) · 64 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 817 ms Slowoffloads to RAM
ShieldGemma 9BGoogle · 9.2B 64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 13.8 s Very slow 63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 217 ms Runs well
CompassJudger-2 7BOpenCompass · 7.6B 78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 55.0 s Very slow 78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 7.7 s Very slow
Llama Guard 3 8BMeta · 8.0B 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 13.1 s Very slow 67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 379 ms Runs well
ShieldGemma 2BGoogle · 2.6B 58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 3.9 s Very slow 58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 88 ms Runs great
DynaGuard 1.7BUniversity of Maryland · 2.0B 70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 12.3 s Very slow 67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 1.3 s Slow
Prometheus 2 8x7BPrometheus Eval · 47B (13B active) 74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 93.7 s Very slow 71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking. 20.9 s Very slowheavy RAM offload
Llama Guard 4 12BMeta · 12B 66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 19.6 s Very slow 65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 565 ms Slow
Prometheus 2 7BPrometheus Eval · 7.2B 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 52.3 s Very slow 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 7.3 s Very slow
Llama Guard 3 1BMeta · 1.5B 56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 2.5 s Very slow 56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 90 ms Runs great
GLIDERPatronus AI · 3.8B 66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 27.6 s Very slow 65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 2.9 s Very slow
Flow Judge v0.1Flow AI · 3.8B 64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 27.6 s Very slow 63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 2.9 s Very slow

Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.

Was this page helpful?