Intel Arc B580 vs Intel Arc A380 for local AI

Pick two to four systems or devices to see which runs more models, faster, for the kind of work you do.

Hardware to compare, two to four

Winner: Intel Arc B580

For general chat: Its ten best models average a score of 92 vs 87; it runs 44 more LLMs (477 vs 433). On the other hand, the Intel Arc A380 is about 1.2× faster on its own top models.

Side by side

Swap sides All models on the Arc B580 All models on the Arc A380

  Intel Arc B580 Limited EditionIntel Arc B580 Intel Arc A380Intel Arc A380
Memory 12.0 GB VRAM 6.0 GB VRAM
Usable for models 12.0 GB 6.0 GB
Memory bandwidth 456 GB/s 186 GB/s
FP16 compute 27.3 TFLOPS 8.2 TFLOPS
Launch price $249 $139
Released 2024 2022
LLMs that run 477 of 619 433 of 619
LLMs that run well 425 296
Image models that run 54 of 60 43 of 60
Video models that run 14 of 43 3 of 43
Decision models that run 68 of 70 63 of 70
Top-10 LLM average score 92 87
Top-10 LLM average speed 39.8 tok/s 47.8 tok/s
Best LLM K2 Horizon 7B Nanbeige4.2 3B

Highlighted values are the best of the two. Model results assume 8K of context.

LLMs

477 of 619 LLMs run on the Intel Arc B580. 12.0 GB usable for models.

433 of 619 LLMs run on the Intel Arc A380. 6.0 GB usable for models.

Model QualityArc B580 SpeedArc B580 RunsArc B580 QualityArc A380 SpeedArc A380 RunsArc A380
K2 Horizon 7BInstitute of Foundation Models · 9.0B 77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. 39.2 tok/s Runs great 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 31.8 tok/s Runs welltight fit
Gemma 4 12BGoogle · 12B 78Great quality (A) · 78 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. 35.8 tok/s Runs great 69Good quality (B) · 69 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 14.8 tok/s Runs welloffloads to RAM
SenseNova Loop 9B PreviewSenseTime · 9.7B 75Great quality (A) · 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 36.6 tok/s Runs great 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 29.7 tok/s Runs welltight fit
Ornith 1.5 9BDeepReinforce · 9.7B 74Great quality (A) · 74 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 36.6 tok/s Runs great 64Good quality (B) · 64 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 23.4 tok/s Runs welltight fit
Qwen3.5 9BAlibaba · 9.7B 73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 36.6 tok/s Runs great 63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 23.4 tok/s Runs welltight fit
TMax 9BAi2 · 9.0B 71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 39.5 tok/s Runs great 62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 25.2 tok/s Runs welltight fit
Ornith 1.0 9BDeepReinforce · 9.7B 72Great quality (A) · 72 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 36.6 tok/s Runs great 63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 23.4 tok/s Runs welltight fit
Nemotron 3.5 Bolt E4BNVIDIA · 8.0B 70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 48.4 tok/s Runs great 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 38.8 tok/s Runs welltight fit
Gemma 4 E4BGoogle · 8.0B 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 48.4 tok/s Runs great 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 38.8 tok/s Runs welltight fit
Granite 4.2 8BIBM · 8.8B 65Good quality (B) · 65 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 40.2 tok/s Runs great 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 32.6 tok/s Runs welltight fit
Nanbeige4.2 3BBOSS Zhipin (Nanbeige) · 4.2B 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 52.2 tok/s Runs great 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 41.8 tok/s Runs great
MiMo V2.6 Distill Qwen 9BXiaomi · 9.4B 75Great quality (A) · 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 37.5 tok/s Runs great 65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 24.0 tok/s Runs welltight fit
Gemma 4 26B-A4BGoogle · 26B (3.8B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 117 tok/s Runs welltight fit 87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 23.6 tok/s Slowheavy RAM offload
Apriel 1.5 15B ThinkerServiceNow · 15B 68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 28.8 tok/s Runs welltight fit 71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 4.5 tok/s Slowheavy RAM offload
InternVL3.5 8BShanghai AI Lab · 8.5B 60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 41.4 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 33.6 tok/s Runs welltight fit
Qwen3-VL 8B InstructAlibaba · 8.8B 60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 40.3 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 32.7 tok/s Runs welltight fit
Ling 3.0 Tiny (8B-A1.3B)Ant Group (inclusionAI) · 7.9B (1.3B active) 60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 102 tok/s Runs great 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 92.4 tok/s Runs welltight fit
Nemotron Labs Dory 1.0 4BNVIDIA · 4.0B 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 54.7 tok/s Runs great 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 43.8 tok/s Runs great
Granite 4.1 8BIBM · 8.8B 58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 40.2 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 32.6 tok/s Runs welltight fit
MiniCPM-V 4.5 8BOpenBMB · 8.7B 58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 32.9 tok/s Runs welltight fit
Qwen3 8BAlibaba · 8.2B 58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 35.0 tok/s Runs welltight fit
Spark-X2.6 4BSparkLLM · 4.1B 66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 53.0 tok/s Runs great 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 42.4 tok/s Runs great
Nemotron Nano 9B v2NVIDIA · 8.9B 58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 39.7 tok/s Runs great 50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 25.4 tok/s Runs welltight fit
Qwen3 14BAlibaba · 15B 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 28.9 tok/s Runs welltight fit 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 7.1 tok/s Slowheavy RAM offload
Gemma 4 E2BGoogle · 5.1B 62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 94.6 tok/s Runs great 60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 62.6 tok/s Runs great
GLM-4.1V 9B ThinkingZhipu AI · 10B 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 41.6 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 27.8 tok/s Runs welltight fit
Mellum2.1 Thinking (12B-A2.5B)JetBrains · 12B (2.5B active) 67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 94.5 tok/s Runs great 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 84.4 tok/s Runs welltight fit
MiMo-VL 7B RLXiaomi · 8.3B 56Good quality (B) · 56 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 42.5 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 34.5 tok/s Runs welltight fit
Qwen3.5 4BAlibaba · 4.7B 62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 46.7 tok/s Runs great 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 37.4 tok/s Runs great
UI-Mate 9BTencent · 9.4B 67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 37.5 tok/s Runs great 58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 24.0 tok/s Runs welltight fit
UI-Venus 2 9BAnt Group (inclusionAI) · 9.4B 67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 37.5 tok/s Runs great 58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 24.0 tok/s Runs welltight fit
Hermes 4 14BNous Research · 15B 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 29.0 tok/s Runs welltight fit 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 7.1 tok/s Slowheavy RAM offload
Phi-4 Reasoning Vision 15BMicrosoft · 15B 58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 36.6 tok/s Runs welltight fit 69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 4.4 tok/s Slowheavy RAM offload
Gemma 3 12B ITGoogle · 12B 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 35.0 tok/s Runs great 50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 14.3 tok/s Runs welloffloads to RAM
Kimi-VL A3B Thinking 2506Moonshot AI · 16B (3.0B active) 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 96.3 tok/s Runs welltight fit 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 59.9 tok/s Runs welloffloads to RAM
MiMo 7B RLXiaomi · 7.8B 55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great 47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 28.8 tok/s Runs welltight fit
Mage-VL 4.7BMicrosoft · 4.7B 56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 45.9 tok/s Runs great 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 36.8 tok/s Runs great
Ministral 3 14BMistral AI · 14B 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 30.7 tok/s Runs welltight fit 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 7.7 tok/s Slowheavy RAM offload
InternVL3 8BShanghai AI Lab · 7.9B 55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 37.4 tok/s Runs great 47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 28.4 tok/s Runs welltight fit
Ministral 3 8BMistral AI · 8.9B 54Fair quality (C) · 54 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great 39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 32.1 tok/s Runs welltight fit
Phi-4 (14B)Microsoft · 15B 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 29.1 tok/s Runs welltight fit 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 7.0 tok/s Slowheavy RAM offload
Agents-A1 4BInternScience · 4.5B 60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 47.9 tok/s Runs great 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great
TMax 4BAi2 · 4.2B 61Good quality (B) · 61 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 51.7 tok/s Runs great 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 41.4 tok/s Runs great
Llama 3.1 Nemotron Nano 8B v1NVIDIA · 8.0B 52Fair quality (C) · 52 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
DiffusionGemma 26B-A4B ITGoogle · 26B (4.0B active) 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 113 tok/s Runs welltight fit 74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 22.5 tok/s Slowheavy RAM offload
Qwen2.5 14B InstructAlibaba · 15B 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 29.1 tok/s Runs welltight fit 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 7.0 tok/s Slowheavy RAM offload
Fara 1.5 4BMicrosoft · 4.5B 58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 47.9 tok/s Runs great 55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great
LFM2.5 8B-A1BLiquid AI · 8.5B (1.5B active) 51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 115 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 101 tok/s Runs welltight fit
Hunyuan 7B InstructTencent · 7.5B 51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 30.1 tok/s Runs welltight fit
Qwen3-VL 4B InstructAlibaba · 4.4B 54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 49.0 tok/s Runs great 51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 39.3 tok/s Runs great
Qwen2.5 7B InstructAlibaba · 7.6B 51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 29.6 tok/s Runs welltight fit
MiniCPM-o 2.6 8BOpenBMB · 8.7B 50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 40.7 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 26.0 tok/s Runs welltight fit
Phi-4 Reasoning Plus (14B)Microsoft · 15B 68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 29.2 tok/s Runs welltight fit 71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 4.6 tok/s Slowheavy RAM offload
MiniCPM4 8BOpenBMB · 8.2B 52Fair quality (C) · 52 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great 45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 27.5 tok/s Runs welltight fit
InternVL3 14BShanghai AI Lab · 15B 53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 36.6 tok/s Runs welltight fit 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 6.8 tok/s Slowheavy RAM offload
Qwen2.5 14B Instruct 1MAlibaba · 15B 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 29.0 tok/s Runs welltight fit 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 7.0 tok/s Slowheavy RAM offload
LFM2.5 24B-A2BLiquid AI · 24B (2.5B active) 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 151 tok/s Runs welltight fit 70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 35.8 tok/s Slowheavy RAM offload
Olmo 3 7B InstructAi2 · 7.3B 49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 30.9 tok/s Runs welltight fit
GLM-4 9B 0414Zhipu AI · 9.4B 50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 37.6 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 24.0 tok/s Runs welltight fit
Falcon 3 10B InstructTII · 10B 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 41.5 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 17.0 tok/s Runs welloffloads to RAM
Qwen2.5-VL 7B InstructAlibaba · 8.3B 50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 42.6 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 27.2 tok/s Runs welltight fit
Qwen3 4B Instruct 2507Alibaba · 4.0B 54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 54.1 tok/s Runs great 51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 43.4 tok/s Runs great
Falcon-H1 7B InstructTII · 7.6B 49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 39.1 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 29.7 tok/s Runs well
Qwen2.5 7B Instruct 1MAlibaba · 7.6B 49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 29.6 tok/s Runs welltight fit
DeepSeek R1 0528 Qwen3 8BDeepSeek · 8.2B 60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 35.0 tok/s Runs welltight fit
Foundation-Sec 1.1 8B InstructCisco Foundation AI · 8.0B 48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
InternLM3 8B InstructShanghai AI Lab · 8.8B 48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 25.6 tok/s Runs welltight fit
InternVL2.5 8BShanghai AI Lab · 8.1B 48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 43.7 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 27.9 tok/s Runs welltight fit
Phi-4 Reasoning (14B)Microsoft · 15B 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 29.2 tok/s Runs welltight fit 69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 4.6 tok/s Slowheavy RAM offload
Gemma 2 9B ITGoogle · 9.2B 48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 38.2 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 19.4 tok/s Runs welloffloads to RAM
Mage-VL-Cowork 4.7BMicrosoft · 4.7B 60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 45.9 tok/s Runs great 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 36.8 tok/s Runs great
Mistral Nemo 12B InstructMistral AI · 12B 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 35.0 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 18.6 tok/s Runs welloffloads to RAM
Ling Mini 2.0 (16B-A1.4B)Ant Group (inclusionAI) · 16B (1.4B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 151 tok/s Runs welltight fit 36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 105 tok/s Runs welloffloads to RAM
NorthCUA Nanobeipo-ai · 9.4B 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 45.4 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 24.0 tok/s Runs welltight fit
Hermes 3 (Llama 3.1 8B)Nous Research · 8.0B 46Fair quality (C) · 46 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
Llama 3.1 8B InstructMeta · 8.0B 46Fair quality (C) · 46 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
Gemma 3n E4BGoogle · 7.9B 46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 54.4 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 43.6 tok/s Runs welltight fit
Groot 19B A3B 261023GrootTeam · 19B (3.0B active) 47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 115 tok/s Runs welltight fit 54Fair quality (C) · 54 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 31.3 tok/s Slowheavy RAM offload
GLiDE 9BFastino · 7.9B 48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 44.5 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 28.4 tok/s Runs well
Maple Preview (20B-A1B)DeepGrove · 20B (1.5B active) 58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 168 tok/s Runs welltight fit 69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 39.0 tok/s Slowheavy RAM offload
Falcon 3 7B InstructTII · 7.5B 45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 39.8 tok/s Runs great 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 30.2 tok/s Runs welltight fit
Kimi-VL A3B InstructMoonshot AI · 16B (3.0B active) 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 96.3 tok/s Runs welltight fit 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 59.9 tok/s Runs welloffloads to RAM
Ministral 3 14B ReasoningMistral AI · 14B 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 30.7 tok/s Runs welltight fit 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 7.7 tok/s Slowheavy RAM offload
Qwen3 4BAlibaba · 4.0B 50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 54.1 tok/s Runs great 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 43.4 tok/s Runs great
Seed-Coder 8B ReasoningByteDance · 8.3B 56Good quality (B) · 56 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 42.8 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 34.7 tok/s Runs welltight fit
Pixtral 12B (2409)Mistral AI · 12B 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 34.5 tok/s Runs great 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 18.1 tok/s Runs welloffloads to RAM
GLM-4 9B Chat 1MZhipu AI · 9.5B 45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 45.1 tok/s Runs great 33Basic quality (D) · 33 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 30.2 tok/s Runs welltight fit
MiniCPM-V 2.6 8BOpenBMB · 8.1B 46Fair quality (C) · 46 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 43.6 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 27.8 tok/s Runs welltight fit
MiniCPM5 2BOpenBMB · 2.5B 60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 86.4 tok/s Runs great 59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 48.0 tok/s Runs great
EXAONE 3.5 7.8BLG AI Research · 7.8B 45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 28.8 tok/s Runs welltight fit
DeepHermes 3 Llama 3 8B PreviewNous Research · 8.0B 44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
Dolphin 3.0 Llama 3.1 8BCognitive Computations · 8.0B 44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
Granite 3.2 8B InstructIBM · 8.2B 44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 43.2 tok/s Runs great 32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 35.1 tok/s Runs welltight fit
Granite 3.3 8BIBM · 8.2B 44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 43.2 tok/s Runs great 32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 35.1 tok/s Runs welltight fit
Ministral 8B (2410)Mistral AI · 8.0B 44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 35.7 tok/s Runs welltight fit
Tülu 3 8BAi2 · 8.0B 44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
Apertus v1.5 8BSwiss AI · 8.9B 44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 39.7 tok/s Runs great 32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 32.2 tok/s Runs welltight fit
AceReason Nemotron 14BNVIDIA · 15B 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 29.0 tok/s Runs welltight fit 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 7.0 tok/s Slowheavy RAM offload
OpenReasoning Nemotron 14BNVIDIA · 15B 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 29.0 tok/s Runs welltight fit 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 7.0 tok/s Slowheavy RAM offload
Qwen2.5 Coder 14B InstructAlibaba · 15B 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 29.0 tok/s Runs welltight fit 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 7.0 tok/s Slowheavy RAM offload
GLM-Z1 9B (0414)Zhipu AI · 9.4B 56Good quality (B) · 56 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 37.6 tok/s Runs great 48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 24.0 tok/s Runs welltight fit
Granite 4.0 H Tiny (7B-A1B)IBM · 6.9B (1.0B active) 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 119 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 95.8 tok/s Runs welltight fit
Trinity Mini (26B-A3B)Arcee AI · 26B (3.0B active) 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 135 tok/s Runs welltight fit 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 29.4 tok/s Slowheavy RAM offload
Falcon-H1R 7BTII · 7.6B 55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 39.1 tok/s Runs great 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 23.0 tok/s Runs welltight fit
Qwen2.5-Omni 7BAlibaba · 11B 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 39.9 tok/s Runs great 32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 26.7 tok/s Runs welltight fit
Phi-4 Multimodal InstructMicrosoft · 5.6B 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 39.1 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 40.5 tok/s Runs great
Phi-3 Small 8K InstructMicrosoft · 7.4B 43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 30.5 tok/s Runs welltight fit
EXAONE Deep 7.8BLG AI Research · 7.8B 55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great 47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 28.8 tok/s Runs welltight fit
GLM-4 9B Chat (2024)Zhipu AI · 9.4B 45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 45.5 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 24.0 tok/s Runs welltight fit
Granite 4.2 3BIBM · 3.7B 52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 59.5 tok/s Runs great 50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 39.4 tok/s Runs great
Seed-Coder 8B InstructByteDance · 8.3B 54Fair quality (C) · 54 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 42.8 tok/s Runs great 39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 34.7 tok/s Runs welltight fit
Qwen2 7B InstructAlibaba · 7.6B 43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 29.6 tok/s Runs welltight fit
LFM3 1.7BLiquid AI · 1.7B 56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 58.0 tok/s Runs great 56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 51.3 tok/s Runs great
Qwen2.5 Coder 7B InstructAlibaba · 7.6B 54Fair quality (C) · 54 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great 46Fair quality (C) · 46 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 29.6 tok/s Runs welltight fit
Dolphin 2.9.4 Llama 3.1 8BCognitive Computations · 8.0B 42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
Granite 3.1 8B InstructIBM · 8.2B 42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 43.2 tok/s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 35.1 tok/s Runs welltight fit
Hermes 2 Pro Llama 3 8BNous Research · 8.0B 42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
InternVL2 8BShanghai AI Lab · 8.1B 42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 43.7 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 27.9 tok/s Runs welltight fit
OpenChat 3.6 8BOpenChat · 8.0B 42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
ERNIE 4.5 21B-A3BBaidu · 22B (3.0B active) 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 135 tok/s Runs welltight fit 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 30.0 tok/s Slowheavy RAM offload
DeepCoder 14B PreviewAgentica · 15B 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 29.0 tok/s Runs welltight fit 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 7.0 tok/s Slowheavy RAM offload
Olmo 3 7B ThinkAi2 · 7.3B 53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs great 45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 30.9 tok/s Runs welltight fit
Aya Vision 8BCohere · 8.6B 42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 40.9 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 26.1 tok/s Runs welltight fit
Devstral Small 1.1 (24B)Mistral AI · 24B 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 29.8 tok/s Runs welltight fit 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 4.0 tok/s Slowheavy RAM offload
Magistral Small 2507Mistral AI · 24B 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 29.8 tok/s Runs welltight fit 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 4.0 tok/s Slowheavy RAM offload
OpenReasoning Nemotron 7BNVIDIA · 7.6B 53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great 45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 29.6 tok/s Runs welltight fit
Qwen2-VL 7B InstructAlibaba · 8.3B 42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 42.6 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 27.2 tok/s Runs welltight fit
Moonlight 16B-A3B InstructMoonshot AI · 16B (3.0B active) 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 96.3 tok/s Runs welltight fit 32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 60.8 tok/s Runs welloffloads to RAM
North Micro VisionCohere · 2.5B 44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 40.5 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 48.8 tok/s Runs great
Llama 3.1 Nemotron Nano 4B v1.1NVIDIA · 4.5B 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 48.3 tok/s Runs great 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 38.7 tok/s Runs great
Magistral Small 2506Mistral AI · 24B 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 29.8 tok/s Runs welltight fit 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 4.0 tok/s Slowheavy RAM offload
Apriel Nemotron 15B ThinkerServiceNow · 15B 55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 36.9 tok/s Runs welltight fit 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 6.8 tok/s Slowheavy RAM offload
Llama 3 8B InstructMeta · 8.0B 40Fair quality (C) · 40 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 35Basic quality (D) · 35 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
Mistral-NeMo-Minitron 8B InstructNVIDIA · 8.4B 40Fair quality (C) · 40 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great 29Basic quality (D) · 29 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 34.1 tok/s Runs welltight fit
DeepSeek R1 Distill Qwen 14BDeepSeek · 15B 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 29.0 tok/s Runs welltight fit 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 7.0 tok/s Slowheavy RAM offload
JanusCoder 14BShanghai AI Lab · 15B 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 29.0 tok/s Runs welltight fit 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 7.1 tok/s Slowheavy RAM offload
Raon-1 mini Preview (30B-A4B)KRAFTON · 29B (4.3B active) 73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 53.0 tok/s Runs welloffloads to RAM 83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 20.9 tok/s Slowheavy RAM offload
Command R7BCohere · 8.0B 42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
Nemotron 3 Nano 4BNVIDIA · 4.0B 50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 54.8 tok/s Runs great 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 43.9 tok/s Runs great
Qwen3 4B Thinking 2507Alibaba · 4.0B 56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 54.1 tok/s Runs great 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 43.4 tok/s Runs great
Phi-3.5 Mini InstructMicrosoft · 3.8B 40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 57.0 tok/s Runs great 33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 59.0 tok/s Runs welltight fit
Llama 3.2 11B Vision InstructMeta · 11B 39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 35Basic quality (D) · 35 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 16.0 tok/s Runs welloffloads to RAM
Foundation-Sec 8B ReasoningCisco Foundation AI · 8.0B 50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
InternLM2.5 7B ChatShanghai AI Lab · 7.7B 39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 38.3 tok/s Runs great 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 37.0 tok/s Runs great
Flint (40B-A2.5B)Recursive · 40B (2.5B active) 85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 43.2 tok/s Slowheavy RAM offload 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 53.5 tok/s Slowheavy RAM offload
Apertus 8B InstructSwiss AI · 8.1B 38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 43.9 tok/s Runs great 33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 28.0 tok/s Runs welltight fit
Aya Expanse 8BCohere · 8.0B 38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
LFM3-VL 2BLiquid AI · 1.9B 50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 52.3 tok/s Runs great 50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 46.2 tok/s Runs great
Command R7B ArabicCohere · 8.0B 40Fair quality (C) · 40 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 35Basic quality (D) · 35 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
Phi-3 Medium (14B)Microsoft · 14B 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great 32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 16.2 tok/s Runs welloffloads to RAM
Phi-3 Medium 128K InstructMicrosoft · 14B 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 7.5 tok/s Slowheavy RAM offload
Granite 3.0 8B InstructIBM · 8.2B 38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 43.2 tok/s Runs great 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 35.1 tok/s Runs great
Hunyuan 4B InstructTencent · 4.2B 42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 51.6 tok/s Runs great 39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 41.3 tok/s Runs great
Mistral Small 3 24B (2501)Mistral AI · 24B 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 29.8 tok/s Runs welltight fit 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 4.0 tok/s Slowheavy RAM offload
LFM2 8B-A1BLiquid AI · 8.3B (1.5B active) 38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 93.5 tok/s Runs great 31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 101 tok/s Runs great
Phi-3.5 Vision InstructMicrosoft · 4.2B 38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 52.5 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 69.0 tok/s Runs welltight fit
Nous Hermes 2 SOLAR 10.7BNous Research · 11B 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 39.9 tok/s Runs great 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 26.7 tok/s Runs welltight fit
SOLAR 10.7B InstructUpstage · 11B 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 39.9 tok/s Runs great 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 26.7 tok/s Runs welltight fit
Phi-4 Mini (3.8B)Microsoft · 3.8B 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 56.7 tok/s Runs great 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 45.4 tok/s Runs great
Mistral 7B Instruct v0.3Mistral AI · 7.3B 37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 40.9 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 39.5 tok/s Runs great
Zamba2 7B InstructZyphra · 7.4B 37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 40.3 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great
DeepSeek R1 Distill Llama 8BDeepSeek · 8.0B 48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
Qianfan-OCR 4BBaidu · 4.7B 50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 45.9 tok/s Runs great 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 36.8 tok/s Runs great
Gemma 3 4B ITGoogle · 4.3B 45Fair quality (C) · 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 50.6 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs great
LFM2 24B-A2BLiquid AI · 24B (2.3B active) 39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 156 tok/s Runs welltight fit 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Slowheavy RAM offload
Ministral 3 3BMistral AI · 3.9B 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 56.5 tok/s Runs great 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 45.3 tok/s Runs great
Dolphin 2.9 Llama 3 8BCognitive Computations · 8.0B 36Basic quality (D) · 36 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
gpt-oss 20BOpenAI · 21B (3.6B active) 70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 66.9 tok/s Runs welloffloads to RAM 70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Slowheavy RAM offload
Fara 7BMicrosoft · 8.3B 48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 42.6 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 27.2 tok/s Runs welltight fit
MedGemma 4B ITGoogle · 4.3B 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 50.6 tok/s Runs great 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs great
Solar Pro Preview 22BUpstage · 22B 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 31.7 tok/s Runs welltight fit 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 4.3 tok/s Slowheavy RAM offload
DeepSeek Coder V2 Lite (16B)DeepSeek · 16B (2.4B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 111 tok/s Runs welltight fit 36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 73.1 tok/s Runs welloffloads to RAM
Granite 4.1 3BIBM · 3.4B 46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 64.0 tok/s Runs great 44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 42.4 tok/s Runs great
Phi-3 Mini 128K InstructMicrosoft · 3.8B 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 57.0 tok/s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 59.0 tok/s Runs welltight fit
DeepSeek V2 Lite ChatDeepSeek · 16B (2.4B active) 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 111 tok/s Runs welltight fit 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 73.1 tok/s Runs welloffloads to RAM
ERNIE 4.5 21B-A3B ThinkingBaidu · 22B (3.0B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 135 tok/s Runs welltight fit 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 30.1 tok/s Slowheavy RAM offload
Hermes 2 Pro Mistral 7BNous Research · 7.2B 35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great
OLMo 2 7B InstructAi2 · 7.3B 35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs great 25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 39.2 tok/s Runs welltight fit
Starling LM 7B BetaNexusflow · 7.2B 35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great
AFM 4.5BArcee AI · 4.6B 40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 47.1 tok/s Runs great 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 37.8 tok/s Runs great
Dolphin 3.0 R1 Mistral 24BCognitive Computations · 24B 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 29.8 tok/s Runs welltight fit 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 4.0 tok/s Slowheavy RAM offload
Falcon3 Mamba 7B InstructTII · 7.3B 37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 40.8 tok/s Runs great 31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 31.0 tok/s Runs great
Yi 1.5 9B Chat01.AI · 8.8B 34Basic quality (D) · 34 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 40.0 tok/s Runs great 25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 32.4 tok/s Runs great
OLMo 2 13B InstructAi2 · 14B 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 40.3 tok/s Runs welltight fit 36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 8.9 tok/s Slowheavy RAM offload
EXAONE 3.0 7.8B InstructLG AI Research · 7.8B 36Basic quality (D) · 36 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 45.2 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 36.6 tok/s Runs great
OpenChat 3.5 (0106)OpenChat · 7.2B 34Basic quality (D) · 34 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 24Basic quality (D) · 24 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great
Phi-3 Vision 128K InstructMicrosoft · 4.2B 34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 52.5 tok/s Runs great 23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 69.0 tok/s Runs welltight fit
DeepSeek R1 Distill Qwen 7BDeepSeek · 7.6B 45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 29.6 tok/s Runs welltight fit
Chandra OCR 2Datalab · 5.3B 48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 41.1 tok/s Runs great 45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 32.9 tok/s Runs great
Mistral 7B Instruct v0.2Mistral AI · 7.2B 33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great
Nous Hermes 2 Mistral 7B DPONous Research · 7.2B 33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great
OpenChat 3.5 (1210)OpenChat · 7.2B 33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great
Voxtral Mini 3BMistral AI · 4.7B 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 46.5 tok/s Runs great 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 37.3 tok/s Runs great
Aya 23 8BCohere · 8.0B 32Basic quality (D) · 32 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 28.1 tok/s Runs welltight fit
Gemma 3n E2BGoogle · 5.4B 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 109 tok/s Runs great 34Basic quality (D) · 34 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 72.0 tok/s Runs great
Unlimited OCR 2 (3B-A0.6B)Baidu · 3.3B (0.6B active) 44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 96.6 tok/s Runs great 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 94.7 tok/s Runs great
WeVisOne-VL 4BTencent · 4.8B 45Fair quality (C) · 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 45.1 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 36.1 tok/s Runs great
Qwen3.6 35B-A3BAlibaba · 36B (3.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 82.6 tok/s Runs welloffloads to RAM 78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 36.5 tok/s Slowheavy RAM offload
Phi-3 Mini 4K InstructMicrosoft · 3.8B 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 57.0 tok/s Runs great 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 45.7 tok/s Runs great
MiniCPM-V 4.7 35B-A3BOpenBMB · 35B (3.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 84.6 tok/s Runs welloffloads to RAM 78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 36.6 tok/s Slowheavy RAM offload
Mistral Small 22B (2409)Mistral AI · 22B 37Basic quality (D) · 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 31.6 tok/s Runs welltight fit 45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 5.6 tok/s Slowheavy RAM offload
Qwen3.5 35B-A3BAlibaba · 36B (3.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 82.6 tok/s Runs welloffloads to RAM 78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 36.5 tok/s Slowheavy RAM offload
OpenHermes 2.5 Mistral 7BTeknium · 7.2B 31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 22Basic quality (D) · 22 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great
Starling LM 7B AlphaBerkeley · 7.2B 31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 22Basic quality (D) · 22 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great
Qwen3.5 2BAlibaba · 2.3B 40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 44.2 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 39.1 tok/s Runs great
Nex N2 Mini (35B-A3B)Nex-AGI · 35B (3.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 84.9 tok/s Runs welloffloads to RAM 77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 36.7 tok/s Slowheavy RAM offload
S5 nanoMaritaca AI · 4.7B 37Basic quality (D) · 37 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 46.7 tok/s Runs great 35Basic quality (D) · 35 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 37.4 tok/s Runs great
InternLM2.5 20B ChatShanghai AI Lab · 20B 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 35.4 tok/s Runs welltight fit 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 4.9 tok/s Slowheavy RAM offload
LFM2.5-VL 3BLiquid AI · 3.1B 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 69.8 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 38.8 tok/s Runs great
LightOnOCR-3 4BLightOn · 4.5B 48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 47.9 tok/s Runs great 45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great
Sage-A1.5 35B-A3BInternScience · 36B (3.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 82.6 tok/s Runs welloffloads to RAM 76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 36.5 tok/s Slowheavy RAM offload
SmolLM3 3BHugging Face · 3.1B 42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 70.7 tok/s Runs great 41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 39.3 tok/s Runs great
Jamba Reasoning 3BAI21 Labs · 3.2B 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 68.0 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 45.0 tok/s Runs great
CodeGeeX4 All 9BZhipu AI · 9.4B 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 45.5 tok/s Runs great 38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 24.0 tok/s Runs welltight fit
Reka Flash 3.1 (21B)Reka AI · 21B 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 33.6 tok/s Runs welltight fit 64Good quality (B) · 64 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload
Granite 4.0 Micro (3B)IBM · 3.4B 40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 64.0 tok/s Runs great 38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 42.4 tok/s Runs great
Qwen3-VL 2B InstructAlibaba · 2.1B 36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 47.1 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 41.7 tok/s Runs great
Mistral 7B Instruct v0.1Mistral AI · 7.2B 29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 21Basic quality (D) · 21 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great
Ornith 1.5 35B-A3BDeepReinforce · 36B (3.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 82.6 tok/s Runs welloffloads to RAM 75Great quality (A) · 75 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 36.5 tok/s Slowheavy RAM offload
RecurrentGemma 9B ITGoogle · 9.6B 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 44.4 tok/s Runs great 23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 29.7 tok/s Runs well
MiniCPM3 4BOpenBMB · 4.1B 38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 53.5 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 42.9 tok/s Runs great
Apodex 1.1 Mini (35B-A3B)Apodex · 36B (3.0B active) 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 82.6 tok/s Runs welloffloads to RAM 74Great quality (A) · 74 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 36.5 tok/s Slowheavy RAM offload
Ornith 1.0 35B-A3BDeepReinforce · 36B (3.0B active) 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 82.6 tok/s Runs welloffloads to RAM 74Great quality (A) · 74 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 36.5 tok/s Slowheavy RAM offload
Unlimited OCR (3B-A0.6B)Baidu · 3.3B (0.6B active) 40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 96.6 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 94.7 tok/s Runs great
Phi-4 Mini Reasoning (3.8B)Microsoft · 3.8B 46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 56.7 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 45.4 tok/s Runs great
LFM2.5 2.6BLiquid AI · 2.7B 44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 80.6 tok/s Runs great 43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 44.8 tok/s Runs great
Devstral Small 2505Mistral AI · 24B 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 29.8 tok/s Runs welltight fit 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 4.0 tok/s Slowheavy RAM offload
TMax 2BAi2 · 1.9B 40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 53.4 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 47.2 tok/s Runs great
GLM-4.7 Flash (30B-A3B)Zhipu AI · 31B (3.0B active) 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 95.4 tok/s Runs welloffloads to RAM 73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 37.0 tok/s Slowheavy RAM offload
Qwen2.5-Omni 3BAlibaba · 5.5B 30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 39.3 tok/s Runs great 25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 40.7 tok/s Runs great
Neural Chat 7B v3.3Intel · 7.2B 27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 19Basic quality (D) · 19 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great
DeepSeek Prover V2 7BDeepSeek · 6.9B 38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 51.1 tok/s Runs great 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking. 13.7 tok/s Slowheavy RAM offload
Yi Coder 9B Chat01.AI · 8.8B 38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 40.0 tok/s Runs great 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 32.4 tok/s Runs welltight fit
Agents-A1 35B-A3BInternScience · 35B (3.0B active) 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 84.9 tok/s Runs welloffloads to RAM 73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 36.7 tok/s Slowheavy RAM offload
Qwen AgentWorld 35B-A3BAlibaba · 35B (3.0B active) 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 86.2 tok/s Runs welloffloads to RAM 73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 36.7 tok/s Slowheavy RAM offload
Jamba2 3BAI21 Labs · 3.0B 42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 71.8 tok/s Runs great 41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 39.9 tok/s Runs great
LLaVA 1.6 Mistral 7BLLaVA · 7.6B 27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 39.2 tok/s Runs great 19Basic quality (D) · 19 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 37.8 tok/s Runs great
Falcon Mamba 7B InstructTII · 7.3B 29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 40.8 tok/s Runs great 25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 31.0 tok/s Runs great
Gemma 1.1 7B ITGoogle · 8.5B 26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 50.1 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking. 11.1 tok/s Slowheavy RAM offload
Codestral 22BMistral AI · 22B 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 31.6 tok/s Runs welltight fit 52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 5.6 tok/s Slowheavy RAM offload
OLMoE 1B-7B InstructAi2 · 6.9B (1.3B active) 26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 102 tok/s Runs great 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 92.4 tok/s Runs welltight fit
LLM4Decompile 9B V2LLM4Binary · 8.8B 36Basic quality (D) · 36 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 40.0 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 32.4 tok/s Runs welltight fit
Zephyr 7B BetaHugging Face · 7.2B 25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great
TMax 27BAi2 · 27B 75Great quality (A) · 75 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 10.1 tok/s Runs welloffloads to RAM 85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slowheavy RAM offload
MiniCPM-V 4.7 1BOpenBMB · 1.6B 40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 64.7 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 57.3 tok/s Runs great
Llama 3.2 3B InstructMeta · 3.2B 35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 67.8 tok/s Runs great 33Basic quality (D) · 33 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 44.9 tok/s Runs great
InternLM2 Chat 20BShanghai AI Lab · 20B 29Basic quality (D) · 29 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 35.4 tok/s Runs welltight fit 39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 4.9 tok/s Slowheavy RAM offload
EXAONE 3.5 2.4BLG AI Research · 2.4B 30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 41.6 tok/s Runs great 29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 50.2 tok/s Runs great
Gemma 7B ITGoogle · 8.5B 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 50.1 tok/s Runs great 24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking. 11.1 tok/s Slowheavy RAM offload
StableLM 2 12B ChatStability AI · 12B 26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 35.2 tok/s Runs great 19Basic quality (D) · 19 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 20.7 tok/s Runs welloffloads to RAM
Granite 4.0 H MicroIBM · 3.2B 38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 68.2 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 45.2 tok/s Runs great
Qwen2.5-VL 3B InstructAlibaba · 3.8B 34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 58.0 tok/s Runs great 32Basic quality (D) · 32 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great
Laguna XS 2.1 (33B-A3B)Poolside · 33B (3.0B active) 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 78.8 tok/s Runs welloffloads to RAM 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 45.2 tok/s Slowheavy RAM offload
Fara 1.5 27BMicrosoft · 27B 73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 10.1 tok/s Runs welloffloads to RAM 83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slowheavy RAM offload
WeVisOne VL 2BTencent · 2.4B 27Basic quality (D) · 27 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 41.1 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 49.6 tok/s Runs great
Qwen2.5-Math 7B InstructAlibaba · 7.6B 35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 29.6 tok/s Runs well
LLM4Decompile 22B V2LLM4Binary · 22B 40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 31.6 tok/s Runs welltight fit 48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking. 5.6 tok/s Slowheavy RAM offload
Qwen3.8 27BAlibaba · 28B 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 19.1 tok/s Runs welloffloads to RAM 90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slowheavy RAM offload
Falcon-H1 3B InstructTII · 3.2B 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 69.1 tok/s Runs great 35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great
Qwen3 1.7BAlibaba · 2.0B 30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 49.4 tok/s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 43.7 tok/s Runs great
Qwen1.5 7B ChatAlibaba · 7.7B 25Basic quality (D) · 25 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 45.8 tok/s Runs welltight fit 18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking. 20.2 tok/s Slowheavy RAM offload
North Mini Code 1.0 (30B-A3B)Cohere · 30B (3.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 64.6 tok/s Runs welloffloads to RAM 76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 28.6 tok/s Slowheavy RAM offload
Nemotron Cascade 2 30B-A3BNVIDIA · 32B (3.5B active) 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 88.9 tok/s Runs welloffloads to RAM 76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 24.9 tok/s Slowheavy RAM offload
Mathstral 7BMistral AI · 7.3B 33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 40.9 tok/s Runs great 23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 39.5 tok/s Runs great
UI-Mate 27BTencent · 27B 72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 10.1 tok/s Runs welloffloads to RAM 81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slowheavy RAM offload
Hermes 3 Llama 3.2 3BNous Research · 3.2B 32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 67.8 tok/s Runs great 31Basic quality (D) · 31 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 44.9 tok/s Runs great
Qwen3.6 27BAlibaba · 28B 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 19.1 tok/s Runs welloffloads to RAM 89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slowheavy RAM offload
Codestral Mamba 7BMistral AI · 7.3B 35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 40.7 tok/s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 30.9 tok/s Runs great
SmolVLM2 2.2B InstructHugging Face · 2.3B 24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 44.6 tok/s Runs great 24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 39.5 tok/s Runs great
Yi 1.5 6B Chat01.AI · 6.1B 27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 49.0 tok/s Runs great 23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 37.2 tok/s Runs great
DeepSeek MoE 16B ChatDeepSeek · 16B (2.8B active) 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 120 tok/s Runs great 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 44.6 tok/s Slowheavy RAM offload
Orca 2 7BMicrosoft · 6.7B 21Basic quality (D) · 21 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 15Basic quality (D) · 15 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 42.5 tok/s Runs welltight fit
Qwen3.5 27BAlibaba · 28B 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 19.1 tok/s Runs welloffloads to RAM 88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slowheavy RAM offload
Orca 2 13BMicrosoft · 13B 23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 42.5 tok/s Runs welltight fit 23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking. 9.5 tok/s Slowheavy RAM offload
Nanbeige4.3 33BBOSS Zhipin (Nanbeige) · 33B 90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slowheavy RAM offload 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 4.7 tok/s Slowheavy RAM offload
CodeGemma 7BGoogle · 8.5B 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 50.1 tok/s Runs great 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking. 11.1 tok/s Slowheavy RAM offload
DeepSeek Coder 6.7B InstructDeepSeek · 6.7B 32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 63.4 tok/s Runs great 23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking. 23.7 tok/s Slowheavy RAM offload
Qwen2.5 3B InstructAlibaba · 3.1B 34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 70.4 tok/s Runs great 33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 39.2 tok/s Runs great
Kimi Linear 48B-A3BMoonshot AI · 49B (3.0B active) 70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 34.0 tok/s Slowheavy RAM offload 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 44.6 tok/s Slowheavy RAM offload
Qwen1.5 MoE A2.7B ChatAlibaba · 14B (2.7B active) 22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 103 tok/s Runs welltight fit 22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 34.1 tok/s Slowheavy RAM offload
Nemotron 3 Nano 30B-A3BNVIDIA · 32B (3.5B active) 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 88.9 tok/s Runs welloffloads to RAM 73Great quality (A) · 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 24.9 tok/s Slowheavy RAM offload
Qwen3 30B-A3B Instruct 2507Alibaba · 31B (3.3B active) 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 87.5 tok/s Runs welloffloads to RAM 64Good quality (B) · 64 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 33.7 tok/s Slowheavy RAM offload
StarCoder2 15B InstructBigCode · 16B 33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 34.6 tok/s Runs great 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking. 6.8 tok/s Slowheavy RAM offload
Granite 3.1 3B-A800M InstructIBM · 3.3B (0.8B active) 20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 81.0 tok/s Runs great 20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 78.0 tok/s Runs great
Granite Code 8B Instruct (128K)IBM · 8.1B 31Basic quality (D) · 31 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 43.9 tok/s Runs great 22Basic quality (D) · 22 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 35.6 tok/s Runs welltight fit
SmolVLM 2.2B InstructHugging Face · 2.3B 22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 44.6 tok/s Runs great 21Basic quality (D) · 21 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 53.8 tok/s Runs great
Qwen3-VL 30B-A3B ThinkingAlibaba · 31B (3.3B active) 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 85.4 tok/s Runs welloffloads to RAM 63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 33.6 tok/s Slowheavy RAM offload
Llama 2 7B ChatMeta · 6.7B 19Basic quality (D) · 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 42.5 tok/s Runs welltight fit
InternLM2 Math Plus 7BShanghai AI Lab · 7.7B 31Basic quality (D) · 31 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 45.6 tok/s Runs great 22Basic quality (D) · 22 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 37.0 tok/s Runs great
Llama 2 13B ChatMeta · 13B 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 42.5 tok/s Runs welltight fit 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking. 9.5 tok/s Slowheavy RAM offload
Baichuan 2 7B ChatBaichuan · 7.5B 19Basic quality (D) · 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 39.5 tok/s Runs great 14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 37.4 tok/s Runs welloffloads to RAM
Falcon 3 3B InstructTII · 3.2B 30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 67.4 tok/s Runs great 29Basic quality (D) · 29 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 44.6 tok/s Runs great
Baichuan 2 13B ChatBaichuan · 14B 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 39.8 tok/s Runs welltight fit 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 40/100 (quality, fit, speed and memory use) sets the ranking. 8.8 tok/s Slowheavy RAM offload
LLM4Decompile 6.7B V2LLM4Binary · 6.7B 30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 63.4 tok/s Runs great 22Basic quality (D) · 22 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 40/100 (quality, fit, speed and memory use) sets the ranking. 23.7 tok/s Slowheavy RAM offload
KAT-Coder V2.5 Dev (35B-A3B)Kuaishou · 35B (3.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 86.2 tok/s Runs welloffloads to RAM 76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 36.7 tok/s Slowheavy RAM offload
Muse Glimmer 30BMeta · 30B 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 16.5 tok/s Runs welloffloads to RAM 87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 3.2 tok/s Slowheavy RAM offload
Qwen3 30B-A3BAlibaba · 31B (3.3B active) 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 87.6 tok/s Runs welloffloads to RAM 62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 33.7 tok/s Slowheavy RAM offload
Qwen3-VL 30B-A3B InstructAlibaba · 31B (3.3B active) 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 85.4 tok/s Runs welloffloads to RAM 62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 33.6 tok/s Slowheavy RAM offload
LFM2 2.6BLiquid AI · 2.6B 34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 84.7 tok/s Runs great 33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 47.1 tok/s Runs great
Granite 3.0 3B-A800M InstructIBM · 3.4B (0.8B active) 18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 81.0 tok/s Runs great 18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 78.0 tok/s Runs great
MiniCPM5 1BOpenBMB · 1.1B 36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 92.9 tok/s Runs great 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 82.2 tok/s Runs great
DeepSeek LLM 7B ChatDeepSeek · 6.9B 17Basic quality (D) · 17 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 42.9 tok/s Runs great 12Basic quality (D) · 12 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 41.5 tok/s Runs welltight fit
SQLCoder 7B v2Defog · 6.7B 28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 63.4 tok/s Runs great 21Basic quality (D) · 21 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking. 23.7 tok/s Slowheavy RAM offload
Nemotron 3.5 Lightning 30B-A3BNVIDIA · 32B (3.0B active) 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 99.1 tok/s Runs welloffloads to RAM 68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 28.6 tok/s Slowheavy RAM offload
Kolibri 1 (78B-A3.5B)Aleph Alpha · 78B (3.5B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 45.9 tok/s Slowheavy RAM offload 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
TeleOCRXingChen-AGI · 1.4B 40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 70.6 tok/s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 62.5 tok/s Runs great
LLaVA 1.5 13BLLaVA · 13B 18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 41.4 tok/s Runs welltight fit 18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking. 9.2 tok/s Slowheavy RAM offload
WizardLM 13B v1.2WizardLM · 13B 18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 42.5 tok/s Runs welltight fit 18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking. 9.5 tok/s Slowheavy RAM offload
Granite 3.2 2B InstructIBM · 2.5B 30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 86.0 tok/s Runs great 29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 47.8 tok/s Runs great
Granite 3.3 2BIBM · 2.5B 30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 86.0 tok/s Runs great 29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 47.8 tok/s Runs great
Devstral Small 2 (24B)Mistral AI · 24B 63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 12.6 tok/s Runs welloffloads to RAM 72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slowheavy RAM offload
LLaVA 1.5 7BLLaVA · 7.1B 15Basic quality (D) · 15 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great 11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs welltight fit
Qwen2-VL 2B InstructAlibaba · 2.2B 24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 45.4 tok/s Runs great 24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 40.2 tok/s Runs great
Qwen3-Omni 30B-A3B InstructAlibaba · 35B (3.3B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 73.5 tok/s Runs welloffloads to RAM 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 42.2 tok/s Slowheavy RAM offload
Sarvam 30BSarvam AI · 32B (2.4B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 111 tok/s Runs welloffloads to RAM 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 34.8 tok/s Slowheavy RAM offload
Nemotron Mini 4B InstructNVIDIA · 4.2B 22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 52.0 tok/s Runs great 20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 41.6 tok/s Runs great
Hunyuan Hy-Memory 30B-A3BTencent · 30B (3.0B active) 59Good quality (B) · 59 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 95.8 tok/s Runs welloffloads to RAM 71Great quality (A) · 71 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 36.8 tok/s Slowheavy RAM offload
EXAONE 4.0 1.2BLG AI Research · 1.3B 30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 78.4 tok/s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 69.4 tok/s Runs great
WizardMath 7B v1.1WizardLM · 7.2B 25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great
Vicuna 13B v1.5LMSYS · 13B 16Basic quality (D) · 16 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 42.5 tok/s Runs welltight fit 16Basic quality (D) · 16 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 38/100 (quality, fit, speed and memory use) sets the ranking. 9.5 tok/s Slowheavy RAM offload
MiniCPM-OCR 1.0 1.3BOpenBMB · 1.3B 42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 77.2 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 68.3 tok/s Runs great
LFM2.5 1.2B InstructLiquid AI · 1.2B 32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 85.7 tok/s Runs great 32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 75.9 tok/s Runs great
Gemma 4 31BGoogle · 31B 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 10.6 tok/s Runs welloffloads to RAM 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 5.0 tok/s Slowheavy RAM offload
Granite Vision 3.2 2BIBM · 3.0B 26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 73.0 tok/s Runs great 25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs great
Granite 3.1 2B InstructIBM · 2.5B 28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 86.0 tok/s Runs great 27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 47.8 tok/s Runs great
Yi 6B Chat01.AI · 6.1B 19Basic quality (D) · 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 49.0 tok/s Runs great 16Basic quality (D) · 16 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 37.2 tok/s Runs great
OLMo 7B InstructAi2 · 6.9B 13Basic quality (D) · 13 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great 9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 41.6 tok/s Runs great
Vicuna 7B v1.5LMSYS · 6.7B 13Basic quality (D) · 13 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 42.5 tok/s Runs welltight fit
Gemma 2 2BGoogle · 2.6B 28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 83.4 tok/s Runs great 27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 46.4 tok/s Runs great
StarCoder2 15B (base)BigCode · 16B 26Basic quality (D) · 26 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 34.6 tok/s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 38/100 (quality, fit, speed and memory use) sets the ranking. 6.8 tok/s Slowheavy RAM offload
SmolLM2 1.7B InstructHugging Face · 1.7B 20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 58.7 tok/s Runs great 20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 51.9 tok/s Runs great
InternLM2 Math Plus 20BShanghai AI Lab · 20B 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 35.4 tok/s Runs welltight fit 38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking. 4.9 tok/s Slowheavy RAM offload
Granite 4.0 H 1BIBM · 1.5B 28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 68.7 tok/s Runs great 28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 60.8 tok/s Runs great
Qwen2.5 Coder 3B InstructAlibaba · 3.1B 36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 70.4 tok/s Runs great 35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 39.2 tok/s Runs great
Granite Code 20B InstructIBM · 20B 25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 35.0 tok/s Runs great 34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking. 5.2 tok/s Slowheavy RAM offload
Falcon-H1 1.5B InstructTII · 1.6B 26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 64.7 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 57.3 tok/s Runs great
InternLM 7BShanghai AI Lab · 7.3B 11Basic quality (D) · 11 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 40.5 tok/s Runs great 8Basic quality (D) · 8 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 39.1 tok/s Runs great
StarCoder2 7BBigCode · 7.2B 23Basic quality (D) · 23 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 41.4 tok/s Runs great 16Basic quality (D) · 16 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 39.9 tok/s Runs great
Code Llama 7B InstructMeta · 6.7B 22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 63.4 tok/s Runs great 16Basic quality (D) · 16 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 38/100 (quality, fit, speed and memory use) sets the ranking. 23.7 tok/s Slowheavy RAM offload
PaddleOCR-VL 1.6 0.9BBaidu · 1.0B 42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 105 tok/s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 42.6 tok/s Runs great
Hunyuan 1.8B InstructTencent · 1.8B 22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 56.0 tok/s Runs great 22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 49.6 tok/s Runs great
Granite 3.0 2B InstructIBM · 2.6B 26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 82.8 tok/s Runs great 25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 46.0 tok/s Runs great
Qwen3 30B-A3B Thinking 2507Alibaba · 31B (3.3B active) 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 87.5 tok/s Runs welloffloads to RAM 65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 33.7 tok/s Slowheavy RAM offload
Qwen3 Coder 30B-A3BAlibaba · 31B (3.3B active) 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 87.5 tok/s Runs welloffloads to RAM 65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 33.7 tok/s Slowheavy RAM offload
Axion 1.0 35BAxion · 35B (3.0B active) 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 84.9 tok/s Runs welloffloads to RAM 52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 36.7 tok/s Slowheavy RAM offload
Qwen2.5 1.5B InstructAlibaba · 1.5B 24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 65.1 tok/s Runs great 24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 57.7 tok/s Runs great
Granite 4.0 H Small (32B-A9B)IBM · 32B (9.0B active) 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs welloffloads to RAM 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 10.1 tok/s Slowheavy RAM offload
LFM2-VL 1.6BLiquid AI · 1.6B 24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 63.5 tok/s Runs great 24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 56.2 tok/s Runs great
InternLM2.5 1.8B ChatShanghai AI Lab · 1.9B 18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 53.1 tok/s Runs great 18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 47.0 tok/s Runs great
Mistral Small 3.2 24BMistral AI · 24B 57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 12.6 tok/s Runs welloffloads to RAM 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slowheavy RAM offload
MedGemma 27B Text ITGoogle · 27B 58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 10.1 tok/s Runs welloffloads to RAM 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slowheavy RAM offload
StarCoder 15BBigCode · 16B 21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 34.9 tok/s Runs great 18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking. 14.5 tok/s Runs welloffloads to RAM
EXAONE 4.5 33BLG AI Research · 34B 77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slowheavy RAM offload 68Good quality (B) · 68 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 3.6 tok/s Slowheavy RAM offload
LFM2 1.2BLiquid AI · 1.2B 26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 85.7 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 75.9 tok/s Runs great
Nemotron 3.5 Bolt 74B-A9BNVIDIA · 75B (9.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 19.7 tok/s Slowheavy RAM offload 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
NorthCUA Minibeipo-ai · 35B (3.0B active) 41Fair quality (C) · 41 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 84.9 tok/s Runs welloffloads to RAM 49Fair quality (C) · 49 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 36.7 tok/s Slowheavy RAM offload
Granite Code 3B Instruct (128K)IBM · 3.5B 20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 62.6 tok/s Runs great 19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 50.1 tok/s Runs welltight fit
Mistral Small 3.1 24BMistral AI · 24B 55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 12.6 tok/s Runs welloffloads to RAM 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slowheavy RAM offload
Zamba2 2.7B InstructZyphra · 2.7B 22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 81.8 tok/s Runs great 21Basic quality (D) · 21 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 45.5 tok/s Runs great
Hemmingway 1.5Hemmingway · 27B 56Good quality (B) · 56 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 10.6 tok/s Runs welloffloads to RAM 63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 3.6 tok/s Slowheavy RAM offload
Gemma 3 27B ITGoogle · 27B 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 18.7 tok/s Runs welloffloads to RAM 68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slowheavy RAM offload
StableLM Zephyr 3BStability AI · 2.8B 16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 77.7 tok/s Runs great 15Basic quality (D) · 15 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 43.2 tok/s Runs great
Qwen3.5 0.8BAlibaba · 0.9B 26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 115 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 47.0 tok/s Runs great
Seed-OSS 36B InstructByteDance · 36B 76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slowheavy RAM offload 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Llama 3.2 1B InstructMeta · 1.2B 22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 80.9 tok/s Runs great 22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 71.6 tok/s Runs great
GLiDE 2BFastino · 1.9B 17Basic quality (D) · 17 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 53.4 tok/s Runs great 17Basic quality (D) · 17 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 47.2 tok/s Runs great
Stable Code Instruct 3BStability AI · 2.8B 20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 77.7 tok/s Runs great 19Basic quality (D) · 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 43.2 tok/s Runs welltight fit
Falcon 3 1B InstructTII · 1.7B 16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 60.1 tok/s Runs great 16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 53.2 tok/s Runs great
Hermes 4.3 36BNous Research · 36B 75Great quality (A) · 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.3 tok/s Slowheavy RAM offload 55Good quality (B) · 55 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Dolphin Mistral 24B Venice EditionCognitive Computations · 24B 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 27.5 tok/s Runs welloffloads to RAM 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slowheavy RAM offload
Granite 4.2 30BIBM · 29B 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 11.4 tok/s Runs welloffloads to RAM 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 5.3 tok/s Slowheavy RAM offload
Qwen3-VL 32B InstructAlibaba · 33B 74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 3.7 tok/s Slowheavy RAM offload 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Terminal RSI 27BTeriminal-RSI · 28B 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 19.1 tok/s Runs welloffloads to RAM 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slowheavy RAM offload
Voxtral Small 24BMistral AI · 24B 52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 12.2 tok/s Runs welloffloads to RAM 58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slowheavy RAM offload
Phi-2 2.7BMicrosoft · 2.8B 16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 78.3 tok/s Runs great 15Basic quality (D) · 15 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 43.5 tok/s Runs great
Falcon 7B InstructTII · 7.2B 5Basic quality (D) · 5 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 41.1 tok/s Runs great 4Basic quality (D) · 4 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 39.7 tok/s Runs great
GLM-4 32B 0414Zhipu AI · 33B 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 11.4 tok/s Runs welloffloads to RAM 62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload
Spark X2.6 1.7BSparkLLM · 1.7B 16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 58.7 tok/s Runs great 16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 51.9 tok/s Runs great
DeepScaleR 1.5B PreviewAgentica · 1.8B 26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 56.4 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 49.9 tok/s Runs great
Qwen3 32BAlibaba · 33B 72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
SmolLM 1.7B InstructHugging Face · 1.7B 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 58.7 tok/s Runs great 14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 51.9 tok/s Runs great
Qwen2 1.5B InstructAlibaba · 1.5B 16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 65.1 tok/s Runs great 16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 57.7 tok/s Runs great
Granite 4.1 30BIBM · 29B 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 11.8 tok/s Runs welloffloads to RAM 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 5.4 tok/s Slowheavy RAM offload
EXAONE 4.0 32BLG AI Research · 32B 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 11.5 tok/s Runs welloffloads to RAM 60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slowheavy RAM offload
Olmo 3.1 32B InstructAi2 · 32B 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 11.3 tok/s Runs welloffloads to RAM 60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload
Magistral Small 1.2 (24B)Mistral AI · 24B 60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 12.6 tok/s Runs welloffloads to RAM 68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slowheavy RAM offload
InternVL3.5 38BShanghai AI Lab · 38B 72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slowheavy RAM offload 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen2.5 Coder 1.5B InstructAlibaba · 1.5B 26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 65.1 tok/s Runs great 26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 57.7 tok/s Runs great
OLMo 2 1B InstructAi2 · 1.5B 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 67.8 tok/s Runs great 14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 60.0 tok/s Runs great
Gemma 3 1BGoogle · 1.0B 20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 100 tok/s Runs great 20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 40.9 tok/s Runs great
StableLM 2 1.6B ChatStability AI · 1.6B 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 61.2 tok/s Runs great 10Basic quality (D) · 10 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 54.1 tok/s Runs great
StableLM 2 Zephyr 1.6BStability AI · 1.6B 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 61.2 tok/s Runs great 10Basic quality (D) · 10 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 54.1 tok/s Runs great
DeepSeek R1 Distill Qwen 1.5BDeepSeek · 1.8B 22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 56.4 tok/s Runs great 22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 49.9 tok/s Runs great
d1 2BLiquid AI · 1.9B 21Basic quality (D) · 21 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 52.3 tok/s Runs great 21Basic quality (D) · 21 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 46.2 tok/s Runs great
Gemma 2 27B ITGoogle · 27B 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 15.1 tok/s Runs welloffloads to RAM 52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 4.5 tok/s Slowheavy RAM offload
Baichuan M2 32BBaichuan · 33B 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen2.5 32B InstructAlibaba · 33B 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LFM2 700MLiquid AI · 0.7B 18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 136 tok/s Runs great 18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 55.3 tok/s Runs great
Qwen3 0.6BAlibaba · 0.8B 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 134 tok/s Runs great 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 54.6 tok/s Runs great
Yi Coder 1.5B Chat01.AI · 1.5B 16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 67.8 tok/s Runs great 16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 60.0 tok/s Runs great
Gemma 2B ITGoogle · 2.5B 12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 86.7 tok/s Runs great 11Basic quality (D) · 11 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 48.2 tok/s Runs great
InternVL3 38BShanghai AI Lab · 38B 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slowheavy RAM offload 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Phi-3.5 MoE (42B-A6.6B)Microsoft · 42B (6.6B active) 53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 16.9 tok/s Slowheavy RAM offload 39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen2.5-VL 32B InstructAlibaba · 33B 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 3.7 tok/s Slowheavy RAM offload 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
SenseNova 6.8 Code2VideoSenseTime · 27B 55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 10.1 tok/s Runs welloffloads to RAM 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 3.5 tok/s Slowheavy RAM offload
Qwen2.5-Math 1.5B InstructAlibaba · 1.5B 20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 65.1 tok/s Runs great 20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 57.7 tok/s Runs great
Falcon-H1 34B InstructTII · 34B 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 9.4 tok/s Slowoffloads to RAM 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 4.7 tok/s Slowheavy RAM offload
Command R (08-2024) 32BCohere · 32B 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 10.2 tok/s Runs welloffloads to RAM 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 4.9 tok/s Slowheavy RAM offload
StarCoder2 3BBigCode · 3.0B 18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 71.8 tok/s Runs great 17Basic quality (D) · 17 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 39.9 tok/s Runs great
QwQ 32BAlibaba · 33B 74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Olmo 3.1 32B ThinkAi2 · 32B 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 11.3 tok/s Runs welloffloads to RAM 63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload
OLMo 2 32B InstructAi2 · 32B 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 10.7 tok/s Runs welloffloads to RAM 50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload
Aya Expanse 32BCohere · 32B 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 10.2 tok/s Runs welloffloads to RAM 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 4.9 tok/s Slowheavy RAM offload
GLM-Z1 32B (0414)Zhipu AI · 33B 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 11.4 tok/s Runs welloffloads to RAM 62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload
Aya Vision 32BCohere · 33B 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 9.6 tok/s Slowoffloads to RAM 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 4.8 tok/s Slowheavy RAM offload
OpenReasoning Nemotron 32BNVIDIA · 33B 72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Skywork OR1 32BSkywork · 33B 72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek Coder 1.3B InstructDeepSeek · 1.4B 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 74.3 tok/s Runs great 12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 65.8 tok/s Runs great
LLM4Decompile 1.3B V2LLM4Binary · 1.4B 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 74.3 tok/s Runs great 12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 65.8 tok/s Runs great
Falcon-H1 0.5B InstructTII · 0.5B 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 193 tok/s Runs great 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 78.7 tok/s Runs great
EXAONE Deep 32BLG AI Research · 32B 71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 3.1 tok/s Slowheavy RAM offload 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLiDE 0.8BFastino · 0.8B 11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 134 tok/s Runs great 11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 54.6 tok/s Runs great
LFM2.5 350MLiquid AI · 0.4B 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 287 tok/s Runs great 14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 117 tok/s Runs great
LFM2-VL 450MLiquid AI · 0.5B 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 223 tok/s Runs great 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 90.9 tok/s Runs great
Hunyuan 0.5B InstructTencent · 0.5B 8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 186 tok/s Runs great 8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 75.8 tok/s Runs great
DeepSeek R1 Distill Qwen 32BDeepSeek · 33B 68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
SmolVLM 500M InstructHugging Face · 0.5B 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 197 tok/s Runs great 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 80.2 tok/s Runs great
TinyLlama 1.1B ChatTinyLlama · 1.1B 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 91.2 tok/s Runs great 6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 80.7 tok/s Runs great
LFM2 350MLiquid AI · 0.4B 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 287 tok/s Runs great 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 117 tok/s Runs great
Granite 4.0 H 350MIBM · 0.3B 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 295 tok/s Runs great 12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 120 tok/s Runs great
CodeGemma 2BGoogle · 2.5B 14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 86.7 tok/s Runs great 13Basic quality (D) · 13 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 48.2 tok/s Runs great
Llama 3.3 Nemotron Super 49B v1.5NVIDIA · 50B 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 4.0 tok/s Slowheavy RAM offload 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen2.5 0.5B InstructAlibaba · 0.5B 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 205 tok/s Runs great 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 83.5 tok/s Runs great
OpenHands LM 32BAll Hands AI · 33B 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Nous Hermes 2 Mixtral 8x7B DPONous Research · 47B (12.9B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 8.5 tok/s Slowheavy RAM offload 36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Yi 1.5 34B Chat01.AI · 34B 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 9.2 tok/s Slowoffloads to RAM 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking. 4.6 tok/s Slowheavy RAM offload
SmolLM2 360M InstructHugging Face · 0.4B 8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 279 tok/s Runs great 8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 114 tok/s Runs great
CogVLM2 Llama 3 Chat 19BZhipu AI · 20B 26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs welloffloads to RAM 26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LFM2.5 230MLiquid AI · 0.2B 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 436 tok/s Runs great 10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 178 tok/s Runs great
Jamba 1.5 MiniAI21 Labs · 52B (12.0B active) 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 9.1 tok/s Slowheavy RAM offload 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Mixtral 8x7B InstructMistral AI · 47B (12.9B active) 47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 8.5 tok/s Slowheavy RAM offload 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen2.5 Coder 32B InstructAlibaba · 33B 63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen2 0.5B InstructAlibaba · 0.5B 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 205 tok/s Runs great 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 83.5 tok/s Runs great
Qwen1.5 14B ChatAlibaba · 14B 23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 22.8 tok/s Runs welloffloads to RAM 23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
surya ocr 2.5Datalab · 0.7B 15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 145 tok/s Runs great 15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 59.3 tok/s Runs great
QwQ 32B PreviewAlibaba · 33B 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
ERNIE 4.5 0.3BBaidu · 0.4B 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 279 tok/s Runs great 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 114 tok/s Runs great
SmolVLM 256M InstructHugging Face · 0.3B 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 386 tok/s Runs great 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 157 tok/s Runs great
Gemma 3 270MGoogle · 0.3B 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 372 tok/s Runs great 6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 152 tok/s Runs great
SmolLM 360M InstructHugging Face · 0.4B 5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 279 tok/s Runs great 5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 114 tok/s Runs great
Hermes 4 70BNous Research · 71B 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 2.5 tok/s Very slowheavy RAM offload 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Llama 3.3 70B InstructMeta · 71B 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 2.5 tok/s Very slowheavy RAM offload 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LLaVA 1.6 34BLLaVA · 35B 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 9.0 tok/s Slowoffloads to RAM 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking. 4.6 tok/s Slowheavy RAM offload
Qwen2 57B-A14B InstructAlibaba · 57B (14.0B active) 42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 10.4 tok/s Slowheavy RAM offload 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek R1 Distill Llama 70BDeepSeek · 71B 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 2.5 tok/s Very slowheavy RAM offload 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Tülu 3 70BAi2 · 71B 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 2.5 tok/s Very slowheavy RAM offload 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
SmolLM2 135M InstructHugging Face · 0.1B 4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 772 tok/s Runs great 4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 315 tok/s Runs great
Yi 34B Chat01.AI · 34B 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 9.2 tok/s Slowoffloads to RAM 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking. 4.6 tok/s Slowheavy RAM offload
Llama 3.1 70B InstructMeta · 71B 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 2.5 tok/s Very slowheavy RAM offload 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Falcon-H1 Tiny 90M InstructTII · 0.1B 3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 1,115 tok/s Runs great 3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 455 tok/s Runs great
Hermes 3 (Llama 3.1 70B)Nous Research · 71B 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 2.5 tok/s Very slowheavy RAM offload 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
SmolLM 135M InstructHugging Face · 0.1B 2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 772 tok/s Runs great 2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 315 tok/s Runs great
Llama 3.1 Nemotron 70B InstructNVIDIA · 71B 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 2.5 tok/s Very slowheavy RAM offload 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Apertus v1.5 70BSwiss AI · 72B 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 2.4 tok/s Very slowheavy RAM offload 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen2.5 Coder 0.5B InstructAlibaba · 0.5B 8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 205 tok/s Runs great 8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 83.5 tok/s Runs great
Qwen1.5 32B ChatAlibaba · 33B 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 3.9 tok/s Slowheavy RAM offload 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Code Llama 13B InstructMeta · 13B 21Basic quality (D) · 21 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 28.3 tok/s Runs welloffloads to RAM 21Basic quality (D) · 21 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 2/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Apertus 70B InstructSwiss AI · 71B 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking. 2.5 tok/s Very slowheavy RAM offload 42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Llama 3 70B InstructMeta · 71B 40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking. 2.5 tok/s Very slowheavy RAM offload 40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Granite Code 34B InstructIBM · 34B 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking. 10.6 tok/s Runs welloffloads to RAM 33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 38/100 (quality, fit, speed and memory use) sets the ranking. 3.7 tok/s Slowheavy RAM offload
DeepSeek Coder 33B InstructDeepSeek · 33B 41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen2.5-Math 72B InstructAlibaba · 73B 40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking. 2.5 tok/s Very slowheavy RAM offload 40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek LLM 67B ChatDeepSeek · 67B 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 40/100 (quality, fit, speed and memory use) sets the ranking. 2.8 tok/s Very slowheavy RAM offload 28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Code Llama 34B InstructMeta · 34B 36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload 26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 36/100 (quality, fit, speed and memory use) sets the ranking. 4.6 tok/s Slowheavy RAM offload
Llama 2 70B ChatMeta · 69B 25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking. 2.7 tok/s Very slowheavy RAM offload 25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Code Llama 70B InstructMeta · 69B 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 36/100 (quality, fit, speed and memory use) sets the ranking. 2.7 tok/s Very slowheavy RAM offload 30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Falcon 40B InstructTII · 42B 11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 35/100 (quality, fit, speed and memory use) sets the ranking. 6.6 tok/s Slowheavy RAM offload 11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 33/100 (quality, fit, speed and memory use) sets the ranking. 3.8 tok/s Slowheavy RAM offload
gpt-oss 120BOpenAI · 117B (5.1B active) 84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K3.1 Pro (2.8T)Moonshot AI · 2,780B (104.0B active) 69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V4.1 Flash 1017 (552B-A16B)DeepSeek · 763B (16.0B active) 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-5.3 (753B-A40B)Zhipu AI · 753B (40.0B active) 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3.8 Max (2.4T-A95B)Alibaba · 2,446B (95.0B active) 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K3 (2.8T)Moonshot AI · 2,780B (104.0B active) 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiMo V2.6 Pro (1T-A42B)Xiaomi · 1,024B (42.0B active) 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling 3.1 Flash 1019 (560B-A25B)Ant Group (inclusionAI) · 572B (25.0B active) 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax M3.1 Flash (428B-A23B)MiniMax · 429B (23.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Step 5 Preview (600B-A27B)StepFun · 604B (27.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-5.3 Flash (320B-A18B)Zhipu AI · 321B (18.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiMo V2.6 Flash (310B-A15B)Xiaomi · 311B (15.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V4.1 Flash (552B-A16B)DeepSeek · 763B (16.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Mistral Large 4 (1T-A52B)Mistral AI · 1,053B (52.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3.8 Flash Next (125B-A6B)Alibaba · 180B (6.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-5.2 (753B-A40B)Zhipu AI · 753B (40.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Hunyuan Hy4 Preview (780B)Tencent · 780B (55.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Muse Spark Open (816B-A42B)Meta · 822B (42.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V4 Pro (1.6T-A49B)DeepSeek · 1,651B (49.0B active) 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K2.6 (1T)Moonshot AI · 1,027B (32.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling 3.1 Flash (560B-A25B)Ant Group (inclusionAI) · 572B (25.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiMo V2.5 Pro (1T-A42B)Xiaomi · 1,023B (42.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Intern-S2 (397B-A17B)Shanghai AI Lab · 403B (17.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-5.1 (744B-A40B)Zhipu AI · 754B (40.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Nex N2 Pro (397B-A17B)Nex-AGI · 397B (17.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V4 Pro (April 2026)DeepSeek · 1,599B (49.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Naive N0.5 Flash (309B-A15.5B)Naive AI · 309B (15.5B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3.5 397B-A17BAlibaba · 403B (17.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax M3 (427B)MiniMax · 427B (25.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-5 (744B-A40B)Zhipu AI · 754B (40.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
K2-Horizon (375B-A23B)Institute of Foundation Models · 379B (23.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Hunyuan Hy3 (295B-A21B)Tencent · 299B (21.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Beam 1.0 (501B-A23B)Reflection AI · 501B (23.4B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K2.5 (1T)Moonshot AI · 1,027B (32.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K2.7 Code (1T)Moonshot AI · 1,027B (32.0B active) 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V4 Flash (284B-A13B)DeepSeek · 304B (13.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiMo V2.5 (310B-A15B)Xiaomi · 311B (15.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ornith 1.5 397B-A17BDeepReinforce · 403B (17.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Nemotron 3.5 Super VL 120B-A12BNVIDIA · 124B (12.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V3.2 (671B)DeepSeek · 685B (37.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Inkling Small (276B-A12B)Thinking Machines Lab · 266B (12.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-4.7 (355B-A32B)Zhipu AI · 358B (32.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Inkling (975B-A41B)Thinking Machines Lab · 952B (41.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax M2.7 (230B-A10B)MiniMax · 229B (10.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Step 3.7 Flash (198B-A11B)StepFun · 201B (11.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Nemotron 3 Ultra 550B-A55BNVIDIA · 561B (55.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek R1 0528 (671B)DeepSeek · 685B (37.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K2 Thinking (1T)Moonshot AI · 1,026B (32.0B active) 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ornith 1.5 122B-A10BDeepReinforce · 125B (10.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V3.2 ExpDeepSeek · 685B (37.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling 2.6 1TAnt Group (inclusionAI) · 1,025B (63.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax M2.5 (230B-A10B)MiniMax · 229B (10.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek R1 (671B)DeepSeek · 671B (37.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V4 Flash (April 2026)DeepSeek · 291B (13.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3.5 122B-A10BAlibaba · 125B (10.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Step 3.5 Flash (196B-A11B)StepFun · 199B (11.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V3.1 TerminusDeepSeek · 685B (37.0B active) 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V3.1 (671B)DeepSeek · 685B (37.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K2 Instruct 0905 (1T)Moonshot AI · 1,026B (32.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax M2.1 (230B-A10B)MiniMax · 229B (10.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3 235B-A22B Thinking 2507Alibaba · 235B (22.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3 Coder 480B-A35BAlibaba · 480B (35.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiMo V2 Flash (309B-A15B)Xiaomi · 310B (15.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling 1TAnt Group (inclusionAI) · 1,000B (50.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Mistral Large 3 (675B)Mistral AI · 675B (41.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3 235B-A22B Instruct 2507Alibaba · 235B (22.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Solar Open 2 250BUpstage · 250B (15.0B active) 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi K2 Instruct (July 2025)Moonshot AI · 1,026B (32.0B active) 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax M2 (230B-A10B)MiniMax · 229B (10.0B active) 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3-VL 235B-A22B InstructAlibaba · 236B (22.0B active) 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Command A+ (218B-A25B)Cohere · 219B (25.0B active) 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
K-EXAONE 236B-A23BLG AI Research · 237B (23.0B active) 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LongCat Flash Chat (560B)Meituan · 562B (27.0B active) 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Nemotron 3 Super 120B-A12BNVIDIA · 124B (12.0B active) 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3 235B-A22BAlibaba · 235B (22.0B active) 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Devstral 2 (123B)Mistral AI · 125B 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
InternVL3.5 241B-A28BShanghai AI Lab · 241B (28.0B active) 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Laguna S 2.1 (118B-A8B)Poolside · 118B (8.0B active) 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Llama 3.1 Nemotron Ultra 253B v1NVIDIA · 253B 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3 Coder Next (80B-A3B)Alibaba · 80B (3.0B active) 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3-Next 80B-A3B ThinkingAlibaba · 81B (3.0B active) 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Baichuan M3 235B-A22BBaichuan · 235B (22.0B active) 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-4.6 (355B-A32B)Zhipu AI · 357B (32.0B active) 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3 Next 80B-A3B InstructAlibaba · 81B (3.0B active) 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V3 0324 (671B)DeepSeek · 685B (37.0B active) 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-4.6VZhipu AI · 108B (12.0B active) 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax-M1 80KMiniMax · 456B (45.9B active) 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Command A Reasoning (111B)Cohere · 111B 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-4.5 (355B-A32B)Zhipu AI · 358B (32.0B active) 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling 3.0 Flash (127B-A5B)Ant Group (inclusionAI) · 127B (5.1B active) 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax-M1 40KMiniMax · 456B (45.9B active) 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Mistral Small 4 (119B-A6B)Mistral AI · 119B (6.0B active) 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Mistral Medium 3.5 (128B)Mistral AI · 128B 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Trinity Large Thinking (400B-A13B)Arcee AI · 399B (13.0B active) 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Command A (111B)Cohere · 111B 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
ERNIE 4.5 300B-A47BBaidu · 300B (47.0B active) 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-4.5VZhipu AI · 108B (12.0B active) 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Hermes 4 405BNous Research · 406B 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kimi Dev 72BMoonshot AI · 73B 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling 2.6 Flash (104B-A7.4B)Ant Group (inclusionAI) · 107B (7.4B active) 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Mistral Large 2.1 (2411)Mistral AI · 123B 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Solar Open 100BUpstage · 103B (12.0B active) 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLM-4.5 Air (106B-A12B)Zhipu AI · 110B (12.0B active) 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Mistral Large 2 (123B)Mistral AI · 123B 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
NorthCUA Probeipo-ai · 397B (17.0B active) 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Pixtral Large 124B (2411)Mistral AI · 124B 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ring Flash 2.0 (100B-A6B)Ant Group (inclusionAI) · 103B (6.1B active) 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
dots.llm1 Instructrednote · 143B (14.0B active) 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling Flash 2.0 (100B-A6B)Ant Group (inclusionAI) · 103B (6.1B active) 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax-Text-01MiniMax · 456B (45.9B active) 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Tülu 3 405BAi2 · 406B 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V3 (671B)DeepSeek · 671B (37.0B active) 52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Sarvam 105BSarvam AI · 106B (10.0B active) 52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek Coder V2 (236B)DeepSeek · 236B (21.0B active) 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Hunyuan A13B (80B-A13B)Tencent · 80B (13.0B active) 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
InternVL3 78BShanghai AI Lab · 78B 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Leanstral 119B-A6BMistral AI · 119B (6.5B active) 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen2.5 72B InstructAlibaba · 73B 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen2.5-VL 72B InstructAlibaba · 73B 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek Coder V2 Instruct (0614)DeepSeek · 236B (21.0B active) 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V2.5 1210DeepSeek · 236B (21.0B active) 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Llama 3.1 405B InstructMeta · 406B 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Llama 4 Maverick (17B x 128E)Meta · 402B (17.0B active) 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Llama 4 Scout 17B-16EMeta · 109B (17.0B active) 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Command R+ (08-2024) 104BCohere · 104B 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V2.5DeepSeek · 236B (21.0B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Groot 65B A6B 261023GrootTeam · 118B (6.0B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Jamba 1.5 LargeAI21 Labs · 399B (94.0B active) 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ling 3.0 flash SanteAnt Group (inclusionAI) · 127B (5.1B active) 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
QVQ 72B PreviewAlibaba · 73B 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Llama 3.2 90B Vision InstructMeta · 89B 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Command R+ 104B (04-2024)Cohere · 104B 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen2 72B InstructAlibaba · 73B 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen2-VL 72B InstructAlibaba · 73B 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
DeepSeek V2 ChatDeepSeek · 236B (21.0B active) 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Mixtral 8x22B InstructMistral AI · 141B (39.0B active) 38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Command R 35B (v01)Cohere · 35B 37Basic quality (D) · 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 37Basic quality (D) · 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen1.5 110B ChatAlibaba · 111B 36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Aya 23 35BCohere · 35B 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Grok-1 (314B)xAI · 314B (86.0B active) 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Snowflake Arctic InstructSnowflake · 479B (17.0B active) 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Falcon 180B ChatTII · 180B 23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking. — Won't run

Image models

54 of 60 image models run on the Intel Arc B580. 12.0 GB usable for models.

43 of 60 image models run on the Intel Arc A380. 6.0 GB usable for models.

Model QualityArc B580 SpeedArc B580 RunsArc B580 QualityArc A380 SpeedArc A380 RunsArc A380
FLUX.2 [klein] 4BBlack Forest Labs · 3.9B 76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 3.7 s Runs great 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 13 s Runs greatencoder in RAM
Z-Image TurboAlibaba (Tongyi) · 6.2B 78Great quality (A) · 78 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 12 s Runs great 82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 30 s Runs wellCPU offload
FLUX 3 Image KleinBlack Forest Labs · 5.6B 84Great quality (A) · 84 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 28 s Runs well 84Great quality (A) · 84 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 73 s Slowtight fit · encoder in RAM
ERNIE-Image TurboBaidu · 8.0B 79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 16 s Runs welltight fit 83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 38 s Runs welltight fit · CPU offload
FLUX.2 [klein] 9BBlack Forest Labs · 9.1B 78Great quality (A) · 78 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 9.4 s Runs greatencoder in RAM 82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 21 s Runs welltight fit · CPU offload
DreamShaper XL v2 TurboLykon · 2.6B 58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 5.4 s Runs great 56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 12 s Runs welltight fit
Z-ImageAlibaba (Tongyi) · 6.2B 76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 84 s Slow 80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 3.4 min Very slowCPU offload
SDXL-Lightning (4-step)ByteDance · 2.6B 54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 2.9 s Runs great 52Fair quality (C) · 52 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 6.4 s Runs welltight fit
FLUX.1 [schnell]Black Forest Labs · 11.9B 65Good quality (B) · 65 of 100: how capable the model is at NF4, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 9.5 s Runs greatencoder in RAM 65Good quality (B) · 65 of 100: how capable the model is at NF4, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 29 s Runs wellCPU offload
ERNIE-ImageBaidu · 8.0B 86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 86 s Slowtight fit · encoder in RAM 87Excellent quality (A+) · 87 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 5.6 min Very slowtight fit · CPU offload
Krea 2 TurboKrea · 12.8B 90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 21 s Runs wellCPU offload 90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Stable Diffusion 3 MediumStability AI · 2.0B 50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 12 s Runs great 50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 30 s Runs wellencoder in RAM
MOSS-Image SFTOpenMOSS · 5.1B 80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 55 s Slowencoder in RAM 76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 3.0 min Very slowtight fit · encoder in RAM
KolorsKuaishou · 2.6B 58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 15 s Runs welltight fit 58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 38 s Runs wellencoder in RAM
Lumina-Image 2.0Shanghai AI Lab · 2.6B 60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 26 s Runs well 60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 67 s Slowencoder in RAM
Playground v2.5Playground AI · 2.6B 60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 31 s Runs well 58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 69 s Slowtight fit
FLUX.1 Krea [dev]Black Forest Labs · 11.9B 76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 62 s Slowencoder in RAM 76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.2 min Very slowtight fit · CPU offload
Stable Diffusion 3.5 Large TurboStability AI · 8.1B 57Good quality (B) · 57 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 8.3 s Runs greatencoder in RAM 60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 19 s Runs welltight fit · CPU offload
FLUX.1 [dev]Black Forest Labs · 11.9B 74Great quality (A) · 74 of 100: how capable the model is at NF4, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 57 s Slowencoder in RAM 74Great quality (A) · 74 of 100: how capable the model is at NF4, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 2.9 min Very slowCPU offload
Juggernaut XL v9RunDiffusion · 2.6B 60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 17 s Runs well 60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 41 s Slowtight fit
RealVisXL V5.0SG161222 · 2.6B 60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 17 s Runs well 60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 41 s Slowtight fit
Animagine XL 4.0Cagliostro Lab · 2.6B 58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 16 s Runs well 58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 38 s Runs welltight fit
Illustrious XL v0.1OnomaAI · 2.6B 58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 16 s Runs well 58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 38 s Runs welltight fit
Stable CascadeStability AI · 5.2B 50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 19 s Runs well 50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 60 s SlowCPU offload
Pony Diffusion V6 XLPurpleSmartAI · 2.6B 56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 14 s Runs well 56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 35 s Runs welltight fit
Stable Diffusion 3.5 MediumStability AI · 2.5B 56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 22 s Runs welltight fit 56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 58 s Slowencoder in RAM
OmniGen2BAAI (VectorSpaceLab) · 4.0B 64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 59 s Slowtight fit 64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 3.2 min Very slowCPU offload
Fibo 2.0 DevBria AI · 8.1B 93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 2.0 min SlowCPU offload 93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Iris-3BSperidlabs · 3.0B 72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 69 s Slowencoder in RAM 72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.5 min Very slowCPU offload
Hunyuan-DiT v1.2Tencent · 1.5B 52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 25 s Runs well 52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 64 s Slowtight fit · encoder in RAM
Chroma1-HDLodestone Rock · 8.9B 76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 78 s Slowtight fit · encoder in RAM 76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 4.0 min Very slowtight fit · CPU offload
SDXL TurboStability AI · 2.6B 40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 0.6 s Runs great 39Basic quality (D) · 39 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 1.4 s Runs great
Animagine XL 3.1Cagliostro Lab · 2.6B 54Fair quality (C) · 54 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 16 s Runs well 54Fair quality (C) · 54 of 100: how capable the model is at FP8, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 38 s Runs welltight fit
Ideogram 4Ideogram · 18.6B 90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 2.0 min SlowCPU offload 85Excellent quality (A+) · 85 of 100: how capable the model is at NF4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
FLUX.1 Kontext [dev]Black Forest Labs · 11.9B 76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 1.9 min Slowtight fit · encoder in RAM 76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 5.8 min Very slowtight fit · CPU offload
Stable Diffusion XL 1.0Stability AI · 2.6B 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 17 s Runs well 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 41 s Slowtight fit
CogView4 6BZhipu AI · 6.4B 68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 70 s Slowencoder in RAM 68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 3.6 min Very slowCPU offload
AuraFlow v0.3fal · 6.9B 56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 52 s Slowtight fit 56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 2.8 min Very slowCPU offload
Stable Diffusion 3.5 LargeStability AI · 8.1B 64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 43 s Slowtight fit · encoder in RAM 64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 2.2 min Slowtight fit · CPU offload
FLUX.2 [dev]Black Forest Labs · 32.2B 85Excellent quality (A+) · 85 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 3.9 min Very slowCPU offload 85Excellent quality (A+) · 85 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
HiDream-I1 DevHiDream.ai · 17.1B 78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 82 s SlowCPU offload 74Great quality (A) · 74 of 100: how capable the model is at NF4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
HiDream-I1 FastHiDream.ai · 17.1B 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 48 s SlowCPU offload 70Great quality (A) · 70 of 100: how capable the model is at NF4, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
HiDream-I1 FullHiDream.ai · 17.1B 83Great quality (A) · 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 4.6 min Very slowCPU offload 78Great quality (A) · 78 of 100: how capable the model is at NF4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
FireRed-Image-Edit 1.1rednote (FireRed Team) · 20.4B 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 8.2 min Very slowtight fit · CPU offload 80Great quality (A) · 80 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen-Image-Edit 2511Alibaba · 20.4B 85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 8.2 min Very slowtight fit · CPU offload 79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
FLUX.1 Fill [dev]Black Forest Labs · 11.9B 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 60 s SlowCPU offload 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen-Image 2512Alibaba · 20.4B 83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 9.7 min Very slowtight fit · CPU offload 83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Realistic Vision V5.1SG161222 · 0.9B 42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 5.0 s Runs great 42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 12 s Runs great
Kandinsky 3Sber AI · 3.0B 48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 39 s Runs wellencoder in RAM 48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 2.0 min SlowCPU offload
PixArt-Σ XL 1024PixArt · 0.6B 50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 6.5 s Runs greatencoder in RAM 50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 16 s Runs wellencoder in RAM
DreamShaper 8Lykon · 0.9B 40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 5.0 s Runs great 40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 12 s Runs great
Stable Diffusion 1.5Stability AI / Runway · 0.9B 35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 5.0 s Runs great 35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 12 s Runs great
SD TurboStability AI · 0.9B 28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 0.3 s Runs great 28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 0.8 s Runs great
LCM DreamShaper v7Tsinghua (LCM) · 0.9B 30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 1.0 s Runs great 30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 2.5 s Runs great
Cosmos3-Super Text2Image (64B)NVIDIA · 64.0B 90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
HunyuanImage 3.0 (80B-A13B)Tencent · 83.0B 88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen-Image 2.1Alibaba · 7.1B 86Excellent quality (A+) · 86 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 86Excellent quality (A+) · 86 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen-ImageAlibaba · 20.4B 86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Ming-Image 0.1 DesignAnt Group (inclusionAI) · 6.2B 84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen-Image 2.1 TurboAlibaba · 7.1B 82Great quality (A) · 82 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 82Great quality (A) · 82 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run

Video models

14 of 43 video models run on the Intel Arc B580. 12.0 GB usable for models.

3 of 43 video models run on the Intel Arc A380. 6.0 GB usable for models.

Model QualityArc B580 SpeedArc B580 RunsArc B580 QualityArc A380 SpeedArc A380 RunsArc A380
LTX-Video 2BLightricks · 1.9B 49Fair quality (C) · 49 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 1.5 min Runs welltight fit 49Fair quality (C) · 49 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 5.0 min Runs wellCPU offload
CogVideoX 2BZhipu AI · 1.7B 44Fair quality (C) · 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 3.7 min Runs welltight fit 44Fair quality (C) · 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 11.9 min Runs wellCPU offload
CogVideoX 5BZhipu AI · 5.0B 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 11.7 min Runs wellencoder in RAM 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
CogVideoX 5B I2VZhipu AI · 5.6B 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 11.7 min Runs wellencoder in RAM 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.1 T2V 1.3BAlibaba · 1.3B 55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 17.3 min Slowencoder in RAM 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Bernini-R A14B (video generation + editing)ByteDance · 14.3B 83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 58.0 min Very slowtight fit · CPU offload 83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.1 VACE 1.3BAlibaba · 2.2B 55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 18.8 min Slowencoder in RAM 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
AnimateDiff v3 (SD 1.5)Shanghai AI Lab · 1.3B 35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 83 s Runs great 35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 3.4 min Runs great
HY-OmniWeaving (8.3B, unified video)Tencent · 8.3B 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 42.7 min Very slowCPU offload 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kandinsky 6.0 Video Lite (3B, video + audio)Sber AI · 3.2B 68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 27.7 min SlowCPU offload 68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
FramePack I2V (HunyuanVideo 13B)lllyasviel · 12.9B 72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 32.2 min Slowtight fit · CPU offload 72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
SkyReels V2 Diffusion Forcing 1.3BSkywork · 1.4B 50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 19.5 min Slowtight fit · encoder in RAM 50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.2 S2V 14B (speech to video)Alibaba · 16.3B 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 80.5 min Very slowtight fit · CPU offload 70Great quality (A) · 70 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.1 I2V 14B 480PAlibaba · 14.0B 72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 72.5 min Very slowtight fit · CPU offload 68Good quality (B) · 68 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Cosmos3-Super Image2Video (64B)NVIDIA · 64.0B 88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Kandinsky 6.0 Pro (29B, video + audio)Sber AI · 30.1B 88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
MiniMax H3 (33B, video + audio)MiniMax · 20.1B 83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
HunyuanVideo 1.5 (8.3B)Tencent · 8.3B 82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LTX-2.5 (22B, video + audio)Lightricks · 21.0B 81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LTX-2.3Lightricks · 18.9B 79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.2 I2V A14BAlibaba · 14.0B 79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.2 T2V A14BAlibaba · 14.0B 79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
SCAIL-2 14B (character animation)Zhipu AI · 16.4B 78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
SkyReels V2 T2V 14B 720PSkywork · 14.0B 76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LTX-2 (19B, video + audio)Lightricks · 18.9B 76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Step-Video T2V (30B)StepFun · 30.0B 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan-Dancer 14B (music to dance)Alibaba · 17.3B 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.1 I2V 14B 720PAlibaba · 14.0B 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.1 T2V 14BAlibaba · 14.0B 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.1 VACE 14BAlibaba · 17.3B 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.2 Animate 14BAlibaba · 17.3B 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
SkyReels V2 T2V 14B 540PSkywork · 14.0B 72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Open-Sora 2.0 (11B)HPC-AI Tech · 11.0B 70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Cosmos-Predict2 14B Video2WorldNVIDIA · 14.0B 68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
HunyuanVideo 13BTencent · 13.0B 66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
HunyuanVideo I2V 13BTencent · 13.0B 66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
LTX-Video 13B 0.9.8 (distilled)Lightricks · 13.0B 66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Wan 2.2 TI2V 5BAlibaba · 5.0B 65Good quality (B) · 65 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 65Good quality (B) · 65 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Mochi 1 Preview (10B)Genmo · 10.0B 60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Cosmos 1.0 Diffusion 7B Text2WorldNVIDIA · 7.0B 60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
CogVideoX 1.5 5BZhipu AI · 5.6B 58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Cosmos-Predict2 2B Video2WorldNVIDIA · 2.0B 56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Allegro (2.8B)Rhymes AI · 2.8B 52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. — Won't run

Decision models

68 of 70 decision models run on the Intel Arc B580. 12.0 GB usable for models.

63 of 70 decision models run on the Intel Arc A380. 6.0 GB usable for models.

Model QualityArc B580 SpeedArc B580 RunsArc B580 QualityArc A380 SpeedArc A380 RunsArc A380
Jina Reranker v3.5Jina AI · 600M 86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 30 ms Runs great 86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 87 ms Runs great
Jina Reranker v3Jina AI · 600M 84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 30 ms Runs great 84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 87 ms Runs great
Intern-Decision 4BShanghai AI Lab · 4.5B 84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 102 ms Runs great 83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 299 ms Runs well
Qwen3 Reranker 4BAlibaba · 4.0B 90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 160 ms Runs well 89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 494 ms Slow
d1 3BLiquid AI · 3.1B 82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 88 ms Runs great 82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 216 ms Runs well
mxbai-rerank-large v2Mixedbread · 1.5B 82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 72 ms Runs great 82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 196 ms Runs well
zerank-2ZeroEntropy · 4.0B 88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 160 ms Runs well 87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 494 ms Slow
FRIDA-DecisionsSber AI · 820M 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 21 ms Runs great 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 62 ms Runs great
GLiNER2.5 DecideFastino · 490M 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 14 ms Runs great 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 39 ms Runs great
LemonDecider 4BAnt Group (inclusionAI) · 4.5B 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 88 ms Runs great 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Qwen3 Reranker 0.6BAlibaba · 600M 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 30 ms Runs great 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 87 ms Runs great
zerank-1 smallZeroEntropy · 1.7B 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 80 ms Runs great 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 219 ms Runs well
Shieldstral 1.0 3BMistral AI · 3.9B 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 153 ms Runs well 85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 473 ms Slow
BGE Reranker v2 GemmaBAAI · 2.5B 78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 101 ms Runs great 78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 317 ms Runs well
Llama Prompt Guard 2 86MMeta · 280M 76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 16 ms Runs great 76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 44 ms Runs great
mxbai-rerank-base v2Mixedbread · 490M 76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 26 ms Runs great 76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 72 ms Runs great
Qwen3 Reranker 8BAlibaba · 8.2B 90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 321 ms Runs well 89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 1.3 s Slowoffloads to RAM
GLiClass ModernBERT Large v3Knowledgator · 400M 74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 13 ms Runs great 74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 32 ms Runs great
OpenAI Privacy FilterOpenAI · 1.4B (50M active) 78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 5.1 ms Runs great 78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 7.5 ms Runs great
BGE Reranker v2 M3BAAI · 570M 72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 28 ms Runs great 72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 85 ms Runs great
DeBERTa v3 Large Zero-shot v2.0Moritz Laurer · 440M 72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 13 ms Runs great 72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 35 ms Runs great
DuoGuard 0.5BDuoGuard · 490M 72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 25 ms Runs great 72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 74 ms Runs great
Skywork Reward V2 Qwen3 0.6BSkywork · 600M 72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 55 ms Runs great 72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 152 ms Runs well
GTE Reranker ModernBERT BaseAlibaba · 150M 70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 10 ms Runs great 70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 25 ms Runs great
Jina Reranker v2 Base MultilingualJina AI · 280M 70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 16 ms Runs great 70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 44 ms Runs great
ModernBERT Large Zero-shot v2.0Moritz Laurer · 400M 70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 13 ms Runs great 70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 32 ms Runs great
Skywork Reward V2 Llama 3.1 8BSkywork · 7.5B 90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 560 ms Slowtight fit 90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLiGuard 300MFastino · 210M 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 13 ms Runs great 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 34 ms Runs great
Granite Embedding Reranker English R2IBM · 150M 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 10 ms Runs great 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 25 ms Runs great
LayaConvai Innovations · 420M 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 13 ms Runs great 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 34 ms Runs great
Llama Prompt Guard 2 22MMeta · 71M 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 7.0 ms Runs great 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 14 ms Runs great
Qwen3Guard Gen 0.6BAlibaba · 750M 74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 74 ms Runs great 73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 126 ms Runs great
GLiClass Base v3Knowledgator · 190M 66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 8.0 ms Runs great 66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 17 ms Runs great
ProtectAI DeBERTa v3 Prompt Injection v2Protect AI · 180M 66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 12 ms Runs great 66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 30 ms Runs great
Qwen3Guard Gen 4BAlibaba · 4.4B 83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 244 ms Runs well 83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 723 ms Slow
DeBERTa v3 Base Zero-shot v2.0Moritz Laurer · 180M 64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 7.8 ms Runs great 64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 17 ms Runs great
GLiNER2 Base v1Fastino · 210M 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 8.5 ms Runs great 68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 19 ms Runs great
Skywork Reward V2 Qwen3 8BSkywork · 7.6B 84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 565 ms Slowtight fit 84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
HHEM 2.1 OpenVectara · 110M 62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 13 ms Runs great 62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 35 ms Runs great
LettuceDetect LargeKR Labs · 400M 66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 38 ms Runs great 66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 103 ms Runs great
Laya MultilingualConvai Innovations · 320M 60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 11 ms Runs great 60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 27 ms Runs great
Granite Guardian 4.1 8BIBM · 8.4B 83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 459 ms Slow 79Great quality (A) · 79 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 1.3 s Slowoffloads to RAM
Qwen3Guard Gen 8BAlibaba · 8.2B 81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 398 ms Runs well 81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 1.3 s Slowoffloads to RAM
GLiNER Multi v2.1Urchade Zaratiana · 290M 62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 10 ms Runs great 62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 25 ms Runs great
ShieldGemma 2BGoogle · 2.6B 58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 105 ms Runs great 58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 330 ms Runs well
Llama 3.1 Nemotron Safety Guard 8B v3NVIDIA · 8.0B 79Great quality (A) · 79 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 390 ms Runs well 79Great quality (A) · 79 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 1.2 s Slowoffloads to RAM
BART Large MNLIMeta · 410M 56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 13 ms Runs great 56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 33 ms Runs great
WildGuard 7BAi2 · 7.3B 79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 398 ms Runs well 76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 1.1 s Slowtight fit
NuExtract 2.0 4BNuMind · 3.8B 75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 371 ms Runs well 75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 961 ms Slow
NuExtract 2.0 2BNuMind · 2.2B 66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 166 ms Runs well 69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 568 ms Slow
Granite Guardian 3.3 8BIBM · 8.2B 76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 397 ms Runs well 76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 1.3 s Slowoffloads to RAM
ms-marco MiniLM L6 v2Sentence Transformers · 23M 52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 5.0 ms Runs great 52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 7.3 ms Runs great
Llama Guard 3 1BMeta · 1.5B 56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 96 ms Runs great 55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 249 ms Runs well
ShieldGemma 9BGoogle · 9.2B 63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 350 ms Runs well 60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 1.3 s Slowoffloads to RAM
Llama Guard 3 8BMeta · 8.0B 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 390 ms Runs well 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 1.2 s Slowoffloads to RAM
Nemotron Content Safety Reasoning 4BNVIDIA · 4.3B 77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 2.5 s Very slow 77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 6.2 s Very slow
DynaGuard 8BUniversity of Maryland · 8.2B 79Great quality (A) · 79 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 6.3 s Very slow 78Great quality (A) · 78 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 15.5 s Very slowoffloads to RAM
Selene 1 Mini 8BAtla · 8.0B 80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 6.1 s Very slow 79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 15.2 s Very slowoffloads to RAM
Llama Guard 4 12BMeta · 12B 64Good quality (B) · 64 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 617 ms Slow 66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 3.8 s Very slowheavy RAM offload
CompassJudger-2 7BOpenCompass · 7.6B 78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 5.8 s Very slow 74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 8.3 s Very slowtight fit
DynaGuard 1.7BUniversity of Maryland · 2.0B 69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 1.2 s Slow 69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 2.9 s Very slow
Prometheus 2 7BPrometheus Eval · 7.2B 68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 5.5 s Very slow 64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 7.9 s Very slowtight fit
GLIDERPatronus AI · 3.8B 65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 2.2 s Very slow 65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 5.6 s Very slow
Flow Judge v0.1Flow AI · 3.8B 63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 2.2 s Very slow 63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 5.6 s Very slow
gpt-oss-safeguard 20BOpenAI · 22B (3.6B active) 76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 2.6 s Very slowoffloads to RAM 76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 6.4 s Very slowheavy RAM offload
ShieldGemma 27BGoogle · 27B 65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 1.8 s Very slowheavy RAM offload 62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 6.7 s Very slowheavy RAM offload
CompassJudger-2 32BOpenCompass · 33B 85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 63.7 s Very slowheavy RAM offload 81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
Prometheus 2 8x7BPrometheus Eval · 47B (13B active) 70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking. 18.7 s Very slowheavy RAM offload 70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
GLiDE 27BFastino · 26B 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. — Won't run
gpt-oss-safeguard 120BOpenAI · 117B (5.1B active) 80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run 80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. — Won't run

Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.

Was this page helpful?