Apps · Desktop app

Unsloth Desktop

by Unsloth AI · Apache-2.0 · 78,556 GitHub stars

Chat with GGUF and MLX models and fine-tune them (LoRA or full) in the same app, from PDFs, CSV or JSON data.

macOSWindowsLinux

Get Unsloth Desktop

At a glance

Best for
Running a model and then training your own version of it without leaving the app.
Inference engine
  • llama.cpp
  • MLX
  • stable-diffusion.cpp
GPU support
  • NVIDIA CUDA
  • AMD ROCm
  • Intel oneAPI / SYCL
  • Apple Metal
  • Apple MLX
  • CPU only
Opens
  • GGUF
  • MLX
Features
  • Chat window
  • OpenAI-compatible API
  • GPU + RAM split for big models
  • Fine-tuning
  • Tool calling
Platforms
macOS, Windows and Linux (including WSL) on NVIDIA, AMD, Intel and Apple GPUs.

Sources: unsloth.ai · unsloth.ai

Install

macOS

Download from unsloth.ai

Windows

Download from unsloth.ai

Linux

Download from unsloth.ai

550 models on this site run in Unsloth Desktop

Based on the files published for each model: GGUF, MLX. Open a model to see the exact command or file for your quantization.

Model Made by Type Size Opens it as Quality tier
CompassJudger-2 32B OpenCompass Decision 33B GGUF 86
Qwen3Guard Gen 8B Alibaba Decision 8.2B GGUF 86
Granite Guardian 4.1 8B IBM Decision 8.4B GGUF 84
Llama 3.1 Nemotron Safety Guard 8B v3 NVIDIA Decision 8.0B GGUF 84
Qwen3Guard Gen 4B Alibaba Decision 4.4B GGUF 84
gpt-oss-safeguard 120B OpenAI Decision 117B (5.1B active) GGUF or MLX 80
Granite Guardian 3.3 8B IBM Decision 8.2B GGUF 80
Selene 1 Mini 8B Atla Decision 8.0B GGUF 80
DynaGuard 8B University of Maryland Decision 8.2B GGUF 79
CompassJudger-2 7B OpenCompass Decision 7.6B GGUF 78
Nemotron Content Safety Reasoning 4B NVIDIA Decision 4.3B GGUF 78
gpt-oss-safeguard 20B OpenAI Decision 22B (3.6B active) GGUF or MLX 76
NuExtract 2.0 4B NuMind Decision 3.8B GGUF 76
Prometheus 2 8x7B Prometheus Eval Decision 47B (13B active) GGUF 74
Qwen3Guard Gen 0.6B Alibaba Decision 750M GGUF or MLX 74
DynaGuard 1.7B University of Maryland Decision 2.0B GGUF 70
NuExtract 2.0 2B NuMind Decision 2.2B GGUF 70
Llama Guard 3 8B Meta Decision 8.0B GGUF 68
Prometheus 2 7B Prometheus Eval Decision 7.2B GGUF or MLX 68
GLIDER Patronus AI Decision 3.8B GGUF 66
ShieldGemma 27B Google Decision 27B GGUF 66
Flow Judge v0.1 Flow AI Decision 3.8B GGUF 64
ShieldGemma 9B Google Decision 9.2B GGUF 64
ShieldGemma 2B Google Decision 2.6B GGUF 58
Llama Guard 3 1B Meta Decision 1.5B GGUF 56
FLUX.2 [dev] Black Forest Labs Image 32.2B GGUF 92
Qwen-Image 2512 Alibaba Image 20.4B GGUF 90
Qwen-Image-Edit 2511 Alibaba Image 20.4B GGUF 86
FLUX.2 [klein] 9B Black Forest Labs Image 9.1B GGUF 84
HiDream-I1 Full HiDream.ai Image 17.1B GGUF 84
Z-Image Turbo Alibaba (Tongyi) Image 6.2B GGUF 84
FLUX.1 Kontext [dev] Black Forest Labs Image 11.9B GGUF 82
FLUX.1 Krea [dev] Black Forest Labs Image 11.9B GGUF 82
Z-Image Alibaba (Tongyi) Image 6.2B GGUF 82
FLUX.1 [dev] Black Forest Labs Image 11.9B GGUF 80
HiDream-I1 Dev HiDream.ai Image 17.1B GGUF 80
FLUX.2 [klein] 4B Black Forest Labs Image 3.9B GGUF 78
HiDream-I1 Fast HiDream.ai Image 17.1B GGUF 76
FLUX.1 [schnell] Black Forest Labs Image 11.9B GGUF 70
Stable Diffusion 3.5 Large Stability AI Image 8.1B GGUF 66
Stable Diffusion 3.5 Large Turbo Stability AI Image 8.1B GGUF 62
AuraFlow v0.3 fal Image 6.9B GGUF 58
Stable Diffusion 3.5 Medium Stability AI Image 2.5B GGUF 58
MiMo V2.6 Pro (1T-A42B) Xiaomi LLM 1,024B (42.0B active) GGUF or MLX 99
Qwen3.8 Max (2.4T-A95B) Alibaba LLM 2,446B (95.0B active) GGUF 98.3
GLM-5.3 (753B-A40B) Zhipu AI LLM 753B (40.0B active) GGUF or MLX 97.8
MiMo V2.6 Flash (310B-A15B) Xiaomi LLM 311B (15.0B active) GGUF or MLX 97.2
DeepSeek V4 Pro (1.6T-A49B) DeepSeek LLM 1,651B (49.0B active) GGUF 96.8
GLM-5.2 (753B-A40B) Zhipu AI LLM 753B (40.0B active) GGUF or MLX 96.3
Hunyuan Hy4 Preview (780B) Tencent LLM 780B (55.0B active) GGUF or MLX 96.3
Qwen3.8 Flash Next (125B-A6B) Alibaba LLM 180B (6.0B active) GGUF or MLX 96.5
DeepSeek V4 Pro (April 2026) DeepSeek LLM 1,599B (49.0B active) MLX 95
GLM-5.1 (744B-A40B) Zhipu AI LLM 754B (40.0B active) GGUF or MLX 95.2
GLM-5.3 Flash (320B-A18B) Zhipu AI LLM 321B (18.0B active) GGUF or MLX 95.4
Kimi K2.6 (1T) Moonshot AI LLM 1,027B (32.0B active) GGUF 95.7
Qwen3.5 397B-A17B Alibaba LLM 403B (17.0B active) GGUF or MLX 95
DeepSeek V4 Flash (284B-A13B) DeepSeek LLM 304B (13.0B active) GGUF 94
GLM-5 (744B-A40B) Zhipu AI LLM 754B (40.0B active) GGUF or MLX 94.7
Hunyuan Hy3 (295B-A21B) Tencent LLM 299B (21.0B active) GGUF 94.4
Kimi K2.5 (1T) Moonshot AI LLM 1,027B (32.0B active) GGUF or MLX 94.3
Kimi K2.7 Code (1T) Moonshot AI LLM 1,027B (32.0B active) GGUF or MLX 94.3
MiniMax M3 (427B) MiniMax LLM 427B (25.0B active) GGUF or MLX 94.8
Ornith 1.5 397B-A17B DeepReinforce LLM 403B (17.0B active) GGUF 94
Qwen3.6 27B Alibaba LLM 28B GGUF or MLX 94
DeepSeek R1 0528 (671B) DeepSeek LLM 685B (37.0B active) GGUF or MLX 93
DeepSeek V3.2 (671B) DeepSeek LLM 685B (37.0B active) GGUF or MLX 93.7
GLM-4.7 (355B-A32B) Zhipu AI LLM 358B (32.0B active) GGUF or MLX 93.5
Kimi K2 Thinking (1T) Moonshot AI LLM 1,026B (32.0B active) GGUF or MLX 93
MiniMax M2.7 (230B-A10B) MiniMax LLM 229B (10.0B active) GGUF or MLX 93.5
Nemotron 3 Ultra 550B-A55B NVIDIA LLM 561B (55.0B active) GGUF 93.2
DeepSeek R1 (671B) DeepSeek LLM 671B (37.0B active) GGUF or MLX 92
DeepSeek V4 Flash (April 2026) DeepSeek LLM 291B (13.0B active) GGUF or MLX 92
Falcon 180B Chat TII LLM 180B GGUF 92
Gemma 4 26B-A4B Google LLM 26B (3.8B active) GGUF or MLX 92.5
MiniMax M2.5 (230B-A10B) MiniMax LLM 229B (10.0B active) GGUF or MLX 92.5
Qwen3.5 122B-A10B Alibaba LLM 125B (10.0B active) GGUF or MLX 92
Step 3.5 Flash (196B-A11B) StepFun LLM 199B (11.0B active) GGUF or MLX 92
DeepSeek V3.1 (671B) DeepSeek LLM 685B (37.0B active) GGUF or MLX 91
DeepSeek V3.1 Terminus DeepSeek LLM 685B (37.0B active) GGUF or MLX 91.5
Gemma 4 31B Google LLM 31B GGUF or MLX 91
Kimi K2 Instruct 0905 (1T) Moonshot AI LLM 1,026B (32.0B active) GGUF or MLX 91
MiniMax M2.1 (230B-A10B) MiniMax LLM 229B (10.0B active) GGUF or MLX 91
Qwen3 235B-A22B Thinking 2507 Alibaba LLM 235B (22.0B active) GGUF or MLX 91
Qwen3 Coder 480B-A35B Alibaba LLM 480B (35.0B active) GGUF or MLX 91
Kimi K3 (2.8T) Moonshot AI LLM 2,780B (104.0B active) GGUF 90
Ling 1T Ant Group (inclusionAI) LLM 1,000B (50.0B active) GGUF 90
MiMo V2 Flash (309B-A15B) Xiaomi LLM 310B (15.0B active) GGUF or MLX 90.9
Mistral Large 3 (675B) Mistral AI LLM 675B (41.0B active) GGUF 90
Qwen3 235B-A22B Instruct 2507 Alibaba LLM 235B (22.0B active) GGUF or MLX 90
Apodex 1.1 Mini (35B-A3B) Apodex LLM 36B (3.0B active) GGUF 89
Kimi K2 Instruct (July 2025) Moonshot AI LLM 1,026B (32.0B active) GGUF or MLX 89
MiniMax M2 (230B-A10B) MiniMax LLM 229B (10.0B active) GGUF or MLX 89
Qwen3-VL 235B-A22B Instruct Alibaba LLM 236B (22.0B active) GGUF or MLX 89
Fara 1.5 27B Microsoft LLM 27B GGUF 88
GLM-4.7 Flash (30B-A3B) Zhipu AI LLM 31B (3.0B active) GGUF or MLX 88
K-EXAONE 236B-A23B LG AI Research LLM 237B (23.0B active) GGUF or MLX 88
LongCat Flash Chat (560B) Meituan LLM 562B (27.0B active) GGUF or MLX 88
Nemotron 3 Super 120B-A12B NVIDIA LLM 124B (12.0B active) GGUF 88
Qwen3 235B-A22B Alibaba LLM 235B (22.0B active) GGUF or MLX 88
Agents-A1 35B-A3B InternScience LLM 35B (3.0B active) GGUF or MLX 87
Qwen AgentWorld 35B-A3B Alibaba LLM 35B (3.0B active) GGUF or MLX 87
Devstral 2 (123B) Mistral AI LLM 125B GGUF or MLX 86
InternVL3.5 241B-A28B Shanghai AI Lab LLM 241B (28.0B active) GGUF 86
Laguna S 2.1 (118B-A8B) Poolside LLM 118B (8.0B active) GGUF or MLX 86
Llama 3.1 Nemotron Ultra 253B v1 NVIDIA LLM 253B GGUF 86
Qwen3 Coder Next (80B-A3B) Alibaba LLM 80B (3.0B active) GGUF or MLX 86
Qwen3-Next 80B-A3B Thinking Alibaba LLM 81B (3.0B active) GGUF or MLX 86
UI-Mate 27B Tencent LLM 27B GGUF 86
Baichuan M3 235B-A22B Baichuan LLM 235B (22.0B active) GGUF 84
GLM-4.6 (355B-A32B) Zhipu AI LLM 357B (32.0B active) GGUF or MLX 84
gpt-oss 120B OpenAI LLM 117B (5.1B active) GGUF or MLX 84
Qwen3 Next 80B-A3B Instruct Alibaba LLM 81B (3.0B active) GGUF or MLX 84
DeepSeek V3 0324 (671B) DeepSeek LLM 685B (37.0B active) GGUF or MLX 83
Gemma 4 12B Google LLM 12B GGUF or MLX 83
GLM-4.6V Zhipu AI LLM 108B (12.0B active) GGUF or MLX 83
Command A Reasoning (111B) Cohere LLM 111B GGUF or MLX 82
DeepSeek R1 Distill Llama 70B DeepSeek LLM 71B GGUF or MLX 82
EXAONE 4.5 33B LG AI Research LLM 34B GGUF 82
GLM-4.5 (355B-A32B) Zhipu AI LLM 358B (32.0B active) GGUF or MLX 82
Leanstral 119B-A6B Mistral AI LLM 119B (6.5B active) MLX 82
Ling 3.0 Flash (127B-A5B) Ant Group (inclusionAI) LLM 127B (5.1B active) GGUF 82
Llama 2 70B Chat Meta LLM 69B GGUF 82
Llama 3.3 70B Instruct Meta LLM 71B GGUF or MLX 82
Command A (111B) Cohere LLM 111B GGUF or MLX 80
ERNIE 4.5 300B-A47B Baidu LLM 300B (47.0B active) GGUF or MLX 80
GLM-4.5V Zhipu AI LLM 108B (12.0B active) GGUF or MLX 80
Hermes 4 405B Nous Research LLM 406B GGUF or MLX 80
Kimi Dev 72B Moonshot AI LLM 73B GGUF or MLX 80
Llama 3.3 Nemotron Super 49B v1.5 NVIDIA LLM 50B GGUF or MLX 80
Mistral Large 2.1 (2411) Mistral AI LLM 123B GGUF 80
Mistral Medium 3.5 (128B) Mistral AI LLM 128B GGUF or MLX 80.4
Mistral Small 4 (119B-A6B) Mistral AI LLM 119B (6.0B active) GGUF or MLX 80.6
North Mini Code 1.0 (30B-A3B) Cohere LLM 30B (3.0B active) GGUF or MLX 80
Seed-OSS 36B Instruct ByteDance LLM 36B GGUF or MLX 80
Solar Open 100B Upstage LLM 103B (12.0B active) GGUF or MLX 80
Trinity Large Thinking (400B-A13B) Arcee AI LLM 399B (13.0B active) GGUF or MLX 80.3
Hermes 4.3 36B Nous Research LLM 36B GGUF 79
DiffusionGemma 26B-A4B IT Google LLM 26B (4.0B active) GGUF or MLX 78
GLM-4.5 Air (106B-A12B) Zhipu AI LLM 110B (12.0B active) GGUF or MLX 78
Mistral Large 2 (123B) Mistral AI LLM 123B GGUF or MLX 78
Qwen3 30B-A3B Thinking 2507 Alibaba LLM 31B (3.3B active) GGUF or MLX 78
Qwen3 Coder 30B-A3B Alibaba LLM 31B (3.3B active) GGUF or MLX 78
Qwen3-VL 32B Instruct Alibaba LLM 33B GGUF or MLX 78
QwQ 32B Alibaba LLM 33B GGUF or MLX 78
Ring Flash 2.0 (100B-A6B) Ant Group (inclusionAI) LLM 103B (6.1B active) GGUF 78
Nemotron 3 Nano 30B-A3B NVIDIA LLM 32B (3.5B active) GGUF 77
Qwen3 30B-A3B Instruct 2507 Alibaba LLM 31B (3.3B active) GGUF or MLX 77
Devstral Small 2 (24B) Mistral AI LLM 24B GGUF or MLX 76
dots.llm1 Instruct rednote LLM 143B (14.0B active) GGUF 76
Granite 4.2 30B IBM LLM 29B GGUF or MLX 76
InternVL3.5 38B Shanghai AI Lab LLM 38B GGUF 76
Ling Flash 2.0 (100B-A6B) Ant Group (inclusionAI) LLM 103B (6.1B active) GGUF 76
Olmo 3.1 32B Think Ai2 LLM 32B GGUF or MLX 76
OpenReasoning Nemotron 32B NVIDIA LLM 33B GGUF or MLX 76
Qwen3 32B Alibaba LLM 33B GGUF or MLX 76
Qwen3-VL 30B-A3B Thinking Alibaba LLM 31B (3.3B active) GGUF or MLX 76
Qwen3.5 9B Alibaba LLM 9.7B GGUF or MLX 76
Skywork OR1 32B Skywork LLM 33B GGUF 76
Tülu 3 405B Ai2 LLM 406B GGUF 76
DeepSeek V3 (671B) DeepSeek LLM 671B (37.0B active) GGUF or MLX 75
Sarvam 105B Sarvam AI LLM 106B (10.0B active) GGUF 75
DeepSeek Coder V2 (236B) DeepSeek LLM 236B (21.0B active) GGUF 74
EXAONE Deep 32B LG AI Research LLM 32B GGUF or MLX 74
GLM-4 32B 0414 Zhipu AI LLM 33B GGUF or MLX 74
GLM-Z1 32B (0414) Zhipu AI LLM 33B GGUF or MLX 74
Hermes 4 70B Nous Research LLM 71B GGUF or MLX 74
Hunyuan A13B (80B-A13B) Tencent LLM 80B (13.0B active) GGUF or MLX 74
InternVL3 78B Shanghai AI Lab LLM 78B GGUF or MLX 74
Kimi Linear 48B-A3B Moonshot AI LLM 49B (3.0B active) GGUF or MLX 74
Qwen3 30B-A3B Alibaba LLM 31B (3.3B active) GGUF or MLX 74
Qwen3-VL 30B-A3B Instruct Alibaba LLM 31B (3.3B active) GGUF or MLX 74
Qwen2.5 72B Instruct Alibaba LLM 73B GGUF or MLX 73
Qwen2.5-VL 72B Instruct Alibaba LLM 73B GGUF or MLX 73
Apriel 1.5 15B Thinker ServiceNow LLM 15B GGUF or MLX 72
DeepSeek Coder V2 Instruct (0614) DeepSeek LLM 236B (21.0B active) GGUF 72
DeepSeek R1 Distill Qwen 32B DeepSeek LLM 33B GGUF or MLX 72
DeepSeek V2.5 1210 DeepSeek LLM 236B (21.0B active) GGUF or MLX 72
EXAONE 4.0 32B LG AI Research LLM 32B GGUF or MLX 72
Gemma 3 27B IT Google LLM 27B GGUF or MLX 72
Granite 4.1 30B IBM LLM 29B GGUF or MLX 72
Llama 3.1 405B Instruct Meta LLM 406B GGUF or MLX 72
Llama 4 Maverick (17B x 128E) Meta LLM 402B (17.0B active) GGUF or MLX 72
Llama 4 Scout 17B-16E Meta LLM 109B (17.0B active) GGUF or MLX 72
Magistral Small 1.2 (24B) Mistral AI LLM 24B GGUF or MLX 72
Nemotron 3.5 Lightning 30B-A3B NVIDIA LLM 32B (3.0B active) GGUF 72
Olmo 3.1 32B Instruct Ai2 LLM 32B GGUF or MLX 72
Phi-4 Reasoning Plus (14B) Microsoft LLM 15B GGUF or MLX 72
Qwen3.6 35B-A3B Alibaba LLM 36B (3.0B active) GGUF or MLX 72
Tülu 3 70B Ai2 LLM 71B GGUF or MLX 72
Nemotron Cascade 2 30B-A3B NVIDIA LLM 32B (3.5B active) GGUF or MLX 71
Ornith 1.0 35B-A3B DeepReinforce LLM 36B (3.0B active) GGUF or MLX 71
Ornith 1.5 35B-A3B DeepReinforce LLM 36B (3.0B active) GGUF or MLX 71
Qwen3.5 35B-A3B Alibaba LLM 36B (3.0B active) GGUF or MLX 71
Baichuan M2 32B Baichuan LLM 33B GGUF 70
Command R+ (08-2024) 104B Cohere LLM 104B GGUF or MLX 70
DeepSeek V2.5 DeepSeek LLM 236B (21.0B active) GGUF 70
Devstral Small 1.1 (24B) Mistral AI LLM 24B GGUF or MLX 70
ERNIE 4.5 21B-A3B Thinking Baidu LLM 22B (3.0B active) GGUF 70
gpt-oss 20B OpenAI LLM 21B (3.6B active) GGUF or MLX 70
InternVL3 38B Shanghai AI Lab LLM 38B GGUF or MLX 70
Laguna XS 2.1 (33B-A3B) Poolside LLM 33B (3.0B active) GGUF or MLX 70
Llama 3.1 70B Instruct Meta LLM 71B GGUF 70
Magistral Small 2507 Mistral AI LLM 24B GGUF or MLX 70
MedGemma 27B Text IT Google LLM 27B GGUF or MLX 70
OpenHands LM 32B All Hands AI LLM 33B GGUF or MLX 70
Phi-4 Reasoning (14B) Microsoft LLM 15B GGUF or MLX 70
Qwen2.5 32B Instruct Alibaba LLM 33B GGUF or MLX 70
Qwen3-Omni 30B-A3B Instruct Alibaba LLM 35B (3.3B active) GGUF or MLX 70
Qwen3.5 27B Alibaba LLM 28B GGUF or MLX 70
Qwen3.8 27B Alibaba LLM 28B GGUF or MLX 70
Sarvam 30B Sarvam AI LLM 32B (2.4B active) GGUF 70
TMax 27B Ai2 LLM 27B MLX 70
UI-Mate 9B Tencent LLM 9.4B GGUF 70
UI-Venus 2 9B Ant Group (inclusionAI) LLM 9.4B GGUF 70
Gemma 4 E4B Google LLM 8.0B GGUF or MLX 68
Granite 4.2 8B IBM LLM 8.8B GGUF or MLX 68
Hermes 3 (Llama 3.1 70B) Nous Research LLM 71B GGUF or MLX 68
Magistral Small 2506 Mistral AI LLM 24B GGUF or MLX 68
Mistral Small 3.2 24B Mistral AI LLM 24B GGUF or MLX 68
QVQ 72B Preview Alibaba LLM 73B GGUF or MLX 68
Qwen2.5-VL 32B Instruct Alibaba LLM 33B GGUF or MLX 68
Qwen3 14B Alibaba LLM 15B GGUF or MLX 68
Qwen2.5 Coder 32B Instruct Alibaba LLM 33B GGUF or MLX 67
AceReason Nemotron 14B NVIDIA LLM 15B GGUF or MLX 66
Apriel Nemotron 15B Thinker ServiceNow LLM 15B GGUF 66
Devstral Small 2505 Mistral AI LLM 24B GGUF or MLX 66
Hermes 4 14B Nous Research LLM 15B GGUF or MLX 66
Llama 3.1 Nemotron 70B Instruct NVIDIA LLM 71B GGUF or MLX 66
Llama 3.2 90B Vision Instruct Meta LLM 89B MLX 66
Ministral 3 14B Reasoning Mistral AI LLM 14B GGUF or MLX 66
Mistral Small 3.1 24B Mistral AI LLM 24B GGUF or MLX 66
OpenReasoning Nemotron 14B NVIDIA LLM 15B GGUF 66
Qwen2.5 Coder 14B Instruct Alibaba LLM 15B GGUF or MLX 66
QwQ 32B Preview Alibaba LLM 33B GGUF or MLX 66
Reka Flash 3.1 (21B) Reka AI LLM 21B GGUF or MLX 66
Trinity Mini (26B-A3B) Arcee AI LLM 26B (3.0B active) GGUF or MLX 66
Command R+ 104B (04-2024) Cohere LLM 104B GGUF or MLX 64
DeepCoder 14B Preview Agentica LLM 15B GGUF or MLX 64
ERNIE 4.5 21B-A3B Baidu LLM 22B (3.0B active) GGUF or MLX 64
Falcon-H1 34B Instruct TII LLM 34B GGUF 64
InternVL3 14B Shanghai AI Lab LLM 15B GGUF or MLX 64
Ministral 3 14B Mistral AI LLM 14B GGUF or MLX 64
Mistral Small 3 24B (2501) Mistral AI LLM 24B GGUF or MLX 64
Phi-4 (14B) Microsoft LLM 15B GGUF or MLX 64
Qwen2 72B Instruct Alibaba LLM 73B GGUF or MLX 64
Qwen2-VL 72B Instruct Alibaba LLM 73B GGUF or MLX 64
Codestral 22B Mistral AI LLM 22B GGUF or MLX 62
Command R (08-2024) 32B Cohere LLM 32B GGUF or MLX 62
DeepSeek R1 0528 Qwen3 8B DeepSeek LLM 8.2B GGUF or MLX 62
DeepSeek R1 Distill Qwen 14B DeepSeek LLM 15B GGUF or MLX 62
DeepSeek V2 Chat DeepSeek LLM 236B (21.0B active) GGUF 62
Dolphin Mistral 24B Venice Edition Cognitive Computations LLM 24B GGUF or MLX 62
Gemma 2 27B IT Google LLM 27B GGUF or MLX 62
Gemma 4 E2B Google LLM 5.1B GGUF or MLX 62
Granite 4.0 H Small (32B-A9B) IBM LLM 32B (9.0B active) GGUF or MLX 62
InternVL3.5 8B Shanghai AI Lab LLM 8.5B GGUF 62
JanusCoder 14B Shanghai AI Lab LLM 15B GGUF 62
Qwen2.5 14B Instruct Alibaba LLM 15B GGUF or MLX 62
Qwen3-VL 8B Instruct Alibaba LLM 8.8B GGUF or MLX 62
Qwen3.5 4B Alibaba LLM 4.7B GGUF or MLX 62
Voxtral Small 24B Mistral AI LLM 24B GGUF 62
Agents-A1 4B InternScience LLM 4.5B GGUF 60
Apertus 70B Instruct Swiss AI LLM 71B GGUF 60
Aya Expanse 32B Cohere LLM 32B GGUF or MLX 60
Aya Vision 32B Cohere LLM 33B MLX 60
Dolphin 3.0 R1 Mistral 24B Cognitive Computations LLM 24B GGUF or MLX 60
Gemma 3 12B IT Google LLM 12B GGUF or MLX 60
GLM-4.1V 9B Thinking Zhipu AI LLM 10B GGUF or MLX 60
Granite 4.1 8B IBM LLM 8.8B GGUF or MLX 60
Kimi-VL A3B Thinking 2506 Moonshot AI LLM 16B (3.0B active) GGUF or MLX 60
Ling 3.0 Tiny (8B-A1.3B) Ant Group (inclusionAI) LLM 7.9B (1.3B active) GGUF 60
MiniCPM-V 4.5 8B OpenBMB LLM 8.7B GGUF 60
Nemotron Nano 9B v2 NVIDIA LLM 8.9B GGUF or MLX 60
OLMo 2 32B Instruct Ai2 LLM 32B GGUF or MLX 60
Qwen2.5 14B Instruct 1M Alibaba LLM 15B GGUF or MLX 60
Qwen3 8B Alibaba LLM 8.2B GGUF or MLX 60
Solar Pro Preview 22B Upstage LLM 22B GGUF 60
Fara 1.5 4B Microsoft LLM 4.5B GGUF or MLX 58
GLM-Z1 9B (0414) Zhipu AI LLM 9.4B GGUF or MLX 58
Llama 3 70B Instruct Meta LLM 71B GGUF or MLX 58
MiMo-VL 7B RL Xiaomi LLM 8.3B GGUF or MLX 58
Qwen2.5-Math 72B Instruct Alibaba LLM 73B GGUF or MLX 58
Seed-Coder 8B Reasoning ByteDance LLM 8.3B GGUF or MLX 58
EXAONE Deep 7.8B LG AI Research LLM 7.8B GGUF or MLX 56
Falcon-H1R 7B TII LLM 7.6B GGUF or MLX 56
InternVL3 8B Shanghai AI Lab LLM 7.9B GGUF or MLX 56
LFM2 24B-A2B Liquid AI LLM 24B (2.3B active) GGUF or MLX 56
Mage-VL 4.7B Microsoft LLM 4.7B MLX 56
Ministral 3 8B Mistral AI LLM 8.9B GGUF or MLX 56
Phi-3.5 MoE (42B-A6.6B) Microsoft LLM 42B (6.6B active) GGUF or MLX 56
Qwen3 4B Thinking 2507 Alibaba LLM 4.0B GGUF or MLX 56
Seed-Coder 8B Instruct ByteDance LLM 8.3B GGUF or MLX 56
Mixtral 8x22B Instruct Mistral AI LLM 141B (39.0B active) GGUF or MLX 55
Qwen2.5 Coder 7B Instruct Alibaba LLM 7.6B GGUF or MLX 55
Command R 35B (v01) Cohere LLM 35B GGUF or MLX 54
Llama 3.1 Nemotron Nano 8B v1 NVIDIA LLM 8.0B GGUF 54
MiniCPM4 8B OpenBMB LLM 8.2B GGUF or MLX 54
Mistral Small 22B (2409) Mistral AI LLM 22B GGUF or MLX 54
Olmo 3 7B Think Ai2 LLM 7.3B GGUF or MLX 54
OpenReasoning Nemotron 7B NVIDIA LLM 7.6B GGUF 54
Qwen3 4B Instruct 2507 Alibaba LLM 4.0B GGUF or MLX 54
Qwen3-VL 4B Instruct Alibaba LLM 4.4B GGUF or MLX 54
DeepSeek Coder V2 Lite (16B) DeepSeek LLM 16B (2.4B active) GGUF or MLX 52
Falcon 3 10B Instruct TII LLM 10B GGUF or MLX 52
GLM-4 9B 0414 Zhipu AI LLM 9.4B GGUF or MLX 52
Granite 4.2 3B IBM LLM 3.7B GGUF or MLX 52
Hunyuan 7B Instruct Tencent LLM 7.5B GGUF or MLX 52
LFM2.5 8B-A1B Liquid AI LLM 8.5B (1.5B active) GGUF or MLX 52
Ling Mini 2.0 (16B-A1.4B) Ant Group (inclusionAI) LLM 16B (1.4B active) GGUF or MLX 52
MiniCPM-o 2.6 8B OpenBMB LLM 8.7B GGUF 52
Mistral Nemo 12B Instruct Mistral AI LLM 12B GGUF or MLX 52
Nous Hermes 2 Mixtral 8x7B DPO Nous Research LLM 47B (12.9B active) GGUF or MLX 52
Qwen1.5 110B Chat Alibaba LLM 111B GGUF or MLX 52
Qwen2.5 7B Instruct Alibaba LLM 7.6B GGUF or MLX 52
Qwen2.5-VL 7B Instruct Alibaba LLM 8.3B GGUF or MLX 52
Aya 23 35B Cohere LLM 35B GGUF or MLX 50
DeepSeek R1 Distill Llama 8B DeepSeek LLM 8.0B GGUF or MLX 50
Falcon-H1 7B Instruct TII LLM 7.6B GGUF or MLX 50
Fara 7B Microsoft LLM 8.3B GGUF or MLX 50
Gemma 2 9B IT Google LLM 9.2B GGUF or MLX 50
Grok-1 (314B) xAI LLM 314B (86.0B active) GGUF 50
InternLM2.5 20B Chat Shanghai AI Lab LLM 20B GGUF 50
InternLM3 8B Instruct Shanghai AI Lab LLM 8.8B GGUF or MLX 50
Jamba 1.5 Mini AI21 Labs LLM 52B (12.0B active) GGUF 50
Kimi-VL A3B Instruct Moonshot AI LLM 16B (3.0B active) GGUF 50
Mixtral 8x7B Instruct Mistral AI LLM 47B (12.9B active) GGUF or MLX 50
Nemotron 3 Nano 4B NVIDIA LLM 4.0B GGUF 50
Olmo 3 7B Instruct Ai2 LLM 7.3B GGUF or MLX 50
Qwen2 57B-A14B Instruct Alibaba LLM 57B (14.0B active) GGUF or MLX 50
Qwen2.5 7B Instruct 1M Alibaba LLM 7.6B GGUF or MLX 50
Qwen3 4B Alibaba LLM 4.0B GGUF or MLX 50
Yi 1.5 34B Chat 01.AI LLM 34B GGUF or MLX 50
GLM-4 9B Chat (2024) Zhipu AI LLM 9.4B GGUF 48
GLM-4 9B Chat 1M Zhipu AI LLM 9.5B GGUF or MLX 48
Hermes 3 (Llama 3.1 8B) Nous Research LLM 8.0B GGUF or MLX 48
Llama 3.1 8B Instruct Meta LLM 8.0B GGUF or MLX 48
MiniCPM-V 2.6 8B OpenBMB LLM 8.1B GGUF 48
CodeGeeX4 All 9B Zhipu AI LLM 9.4B GGUF 46
DeepHermes 3 Llama 3 8B Preview Nous Research LLM 8.0B GGUF or MLX 46
DeepSeek R1 Distill Qwen 7B DeepSeek LLM 7.6B GGUF or MLX 46
Dolphin 3.0 Llama 3.1 8B Cognitive Computations LLM 8.0B GGUF or MLX 46
EXAONE 3.5 7.8B LG AI Research LLM 7.8B GGUF or MLX 46
Falcon 3 7B Instruct TII LLM 7.5B GGUF or MLX 46
Gemma 3n E4B Google LLM 7.9B GGUF or MLX 46
Granite 3.2 8B Instruct IBM LLM 8.2B GGUF 46
Granite 3.3 8B IBM LLM 8.2B GGUF or MLX 46
Granite 4.1 3B IBM LLM 3.4B GGUF or MLX 46
Ministral 8B (2410) Mistral AI LLM 8.0B GGUF or MLX 46
Phi-3 Medium (14B) Microsoft LLM 14B GGUF or MLX 46
Phi-3 Medium 128K Instruct Microsoft LLM 14B GGUF or MLX 46
Phi-4 Mini Reasoning (3.8B) Microsoft LLM 3.8B GGUF or MLX 46
Qwen2.5-Omni 7B Alibaba LLM 11B GGUF 46
Tülu 3 8B Ai2 LLM 8.0B GGUF or MLX 46
Gemma 3 4B IT Google LLM 4.3B GGUF or MLX 45
Ornith 1.0 9B DeepReinforce LLM 9.7B GGUF or MLX 45
Ornith 1.5 9B DeepReinforce LLM 9.7B GGUF or MLX 45
Aya Vision 8B Cohere LLM 8.6B MLX 44
Code Llama 70B Instruct Meta LLM 69B GGUF or MLX 44
Command R7B Cohere LLM 8.0B GGUF or MLX 44
DeepSeek Coder 33B Instruct DeepSeek LLM 33B GGUF 44
Dolphin 2.9.4 Llama 3.1 8B Cognitive Computations LLM 8.0B GGUF or MLX 44
Granite 3.1 8B Instruct IBM LLM 8.2B GGUF 44
Granite 4.0 H Tiny (7B-A1B) IBM LLM 6.9B (1.0B active) GGUF or MLX 44
Hermes 2 Pro Llama 3 8B Nous Research LLM 8.0B GGUF or MLX 44
Jamba Reasoning 3B AI21 Labs LLM 3.2B GGUF or MLX 44
LFM2.5 2.6B Liquid AI LLM 2.7B GGUF or MLX 44
LFM2.5-VL 3B Liquid AI LLM 3.1B GGUF or MLX 44
Llama 3.1 Nemotron Nano 4B v1.1 NVIDIA LLM 4.5B GGUF or MLX 44
MedGemma 4B IT Google LLM 4.3B GGUF or MLX 44
Ministral 3 3B Mistral AI LLM 3.9B GGUF or MLX 44
North Micro Vision Cohere LLM 2.5B MLX 44
OLMo 2 13B Instruct Ai2 LLM 14B GGUF or MLX 44
OpenChat 3.6 8B OpenChat LLM 8.0B GGUF or MLX 44
Phi-4 Mini (3.8B) Microsoft LLM 3.8B GGUF or MLX 44
Qwen1.5 32B Chat Alibaba LLM 33B GGUF or MLX 44
Qwen2 7B Instruct Alibaba LLM 7.6B GGUF or MLX 44
Qwen2-VL 7B Instruct Alibaba LLM 8.3B GGUF or MLX 44
TMax 9B Ai2 LLM 9.0B MLX 44
MiMo 7B RL Xiaomi LLM 7.8B MLX 43
Hunyuan 4B Instruct Tencent LLM 4.2B GGUF or MLX 42
InternLM2 Chat 20B Shanghai AI Lab LLM 20B GGUF 42
Jamba2 3B AI21 Labs LLM 3.0B GGUF 42
Llama 3 8B Instruct Meta LLM 8.0B GGUF or MLX 42
Llama 3.2 11B Vision Instruct Meta LLM 11B MLX 42
Mistral-NeMo-Minitron 8B Instruct NVIDIA LLM 8.4B GGUF or MLX 42
SmolLM3 3B Hugging Face LLM 3.1B GGUF or MLX 42
AFM 4.5B Arcee AI LLM 4.6B GGUF 40
Apertus 8B Instruct Swiss AI LLM 8.1B GGUF or MLX 40
Aya Expanse 8B Cohere LLM 8.0B GGUF or MLX 40
DeepSeek LLM 67B Chat DeepSeek LLM 67B GGUF 40
DeepSeek Prover V2 7B DeepSeek LLM 6.9B GGUF or MLX 40
DeepSeek V2 Lite Chat DeepSeek LLM 16B (2.4B active) GGUF or MLX 40
Granite 3.0 8B Instruct IBM LLM 8.2B GGUF 40
Granite 4.0 Micro (3B) IBM LLM 3.4B GGUF or MLX 40
Granite Code 34B Instruct IBM LLM 34B GGUF 40
InternLM2 Math Plus 20B Shanghai AI Lab LLM 20B GGUF 40
InternLM2.5 7B Chat Shanghai AI Lab LLM 7.7B GGUF or MLX 40
Nous Hermes 2 SOLAR 10.7B Nous Research LLM 11B GGUF or MLX 40
Phi-3.5 Mini Instruct Microsoft LLM 3.8B GGUF or MLX 40
Qwen3.5 2B Alibaba LLM 2.3B GGUF or MLX 40
SOLAR 10.7B Instruct Upstage LLM 11B GGUF or MLX 40
StarCoder2 15B Instruct BigCode LLM 16B GGUF or MLX 40
Unlimited OCR (3B-A0.6B) Baidu LLM 3.3B (0.6B active) MLX 40
Yi 34B Chat 01.AI LLM 34B GGUF 40
Yi Coder 9B Chat 01.AI LLM 8.8B GGUF or MLX 40
Code Llama 34B Instruct Meta LLM 34B GGUF or MLX 38
Dolphin 2.9 Llama 3 8B Cognitive Computations LLM 8.0B GGUF or MLX 38
EXAONE 3.0 7.8B Instruct LG AI Research LLM 7.8B GGUF 38
Falcon3 Mamba 7B Instruct TII LLM 7.3B GGUF or MLX 38
Granite 4.0 H Micro IBM LLM 3.2B GGUF or MLX 38
LFM2 8B-A1B Liquid AI LLM 8.3B (1.5B active) GGUF or MLX 38
MiniCPM3 4B OpenBMB LLM 4.1B GGUF or MLX 38
Mistral 7B Instruct v0.3 Mistral AI LLM 7.3B GGUF or MLX 38
Phi-3.5 Vision Instruct Microsoft LLM 4.2B MLX 38
Codestral Mamba 7B Mistral AI LLM 7.3B MLX 36
Falcon-H1 3B Instruct TII LLM 3.2B GGUF or MLX 36
Gemma 3n E2B Google LLM 5.4B GGUF or MLX 36
Granite Code 20B Instruct IBM LLM 20B GGUF 36
Hermes 2 Pro Mistral 7B Nous Research LLM 7.2B GGUF or MLX 36
OLMo 2 7B Instruct Ai2 LLM 7.3B GGUF or MLX 36
Phi-3 Mini 128K Instruct Microsoft LLM 3.8B GGUF or MLX 36
Phi-3 Mini 4K Instruct Microsoft LLM 3.8B GGUF or MLX 36
Qwen2.5 Coder 3B Instruct Alibaba LLM 3.1B GGUF or MLX 36
Qwen2.5-Math 7B Instruct Alibaba LLM 7.6B GGUF or MLX 36
Qwen3-VL 2B Instruct Alibaba LLM 2.1B GGUF or MLX 36
Starling LM 7B Beta Nexusflow LLM 7.2B GGUF or MLX 36
Voxtral Mini 3B Mistral AI LLM 4.7B GGUF or MLX 36
Yi 1.5 9B Chat 01.AI LLM 8.8B GGUF or MLX 36
Llama 3.2 3B Instruct Meta LLM 3.2B GGUF or MLX 35
OpenChat 3.5 (0106) OpenChat LLM 7.2B GGUF or MLX 35
Aya 23 8B Cohere LLM 8.0B GGUF or MLX 34
CodeGemma 7B Google LLM 8.5B GGUF or MLX 34
DeepSeek Coder 6.7B Instruct DeepSeek LLM 6.7B GGUF 34
LFM2 2.6B Liquid AI LLM 2.6B GGUF or MLX 34
Mathstral 7B Mistral AI LLM 7.3B GGUF or MLX 34
Mistral 7B Instruct v0.2 Mistral AI LLM 7.2B GGUF or MLX 34
Nous Hermes 2 Mistral 7B DPO Nous Research LLM 7.2B GGUF or MLX 34
OpenChat 3.5 (1210) OpenChat LLM 7.2B GGUF 34
Phi-3 Vision 128K Instruct Microsoft LLM 4.2B MLX 34
Qwen1.5 14B Chat Alibaba LLM 14B GGUF or MLX 34
Qwen2.5 3B Instruct Alibaba LLM 3.1B GGUF or MLX 34
Qwen2.5-VL 3B Instruct Alibaba LLM 3.8B GGUF or MLX 34
Granite Code 8B Instruct (128K) IBM LLM 8.1B GGUF 32
Hermes 3 Llama 3.2 3B Nous Research LLM 3.2B GGUF or MLX 32
InternLM2 Math Plus 7B Shanghai AI Lab LLM 7.7B GGUF 32
LFM2.5 1.2B Instruct Liquid AI LLM 1.2B GGUF or MLX 32
OpenHermes 2.5 Mistral 7B Teknium LLM 7.2B GGUF or MLX 32
StarCoder2 15B (base) BigCode LLM 16B GGUF or MLX 32
Starling LM 7B Alpha Berkeley LLM 7.2B GGUF 32
Code Llama 13B Instruct Meta LLM 13B GGUF or MLX 30
EXAONE 3.5 2.4B LG AI Research LLM 2.4B GGUF 30
EXAONE 4.0 1.2B LG AI Research LLM 1.3B GGUF or MLX 30
Falcon 3 3B Instruct TII LLM 3.2B GGUF or MLX 30
Falcon Mamba 7B Instruct TII LLM 7.3B GGUF 30
Granite 3.2 2B Instruct IBM LLM 2.5B GGUF 30
Granite 3.3 2B IBM LLM 2.5B GGUF or MLX 30
Mistral 7B Instruct v0.1 Mistral AI LLM 7.2B GGUF or MLX 30
Qwen2.5-Omni 3B Alibaba LLM 5.5B GGUF 30
Qwen3 1.7B Alibaba LLM 2.0B GGUF or MLX 30
SQLCoder 7B v2 Defog LLM 6.7B GGUF 30
Gemma 1.1 7B IT Google LLM 8.5B GGUF or MLX 28
Gemma 2 2B Google LLM 2.6B GGUF or MLX 28
Granite 3.1 2B Instruct IBM LLM 2.5B GGUF 28
Granite 4.0 H 1B IBM LLM 1.5B GGUF or MLX 28
Neural Chat 7B v3.3 Intel LLM 7.2B GGUF 28
Orca 2 13B Microsoft LLM 13B GGUF 28
StableLM 2 12B Chat Stability AI LLM 12B GGUF or MLX 28
Yi 1.5 6B Chat 01.AI LLM 6.1B GGUF or MLX 28
Baichuan 2 13B Chat Baichuan LLM 14B GGUF 26
DeepScaleR 1.5B Preview Agentica LLM 1.8B GGUF or MLX 26
DeepSeek MoE 16B Chat DeepSeek LLM 16B (2.8B active) GGUF 26
Falcon-H1 1.5B Instruct TII LLM 1.6B GGUF or MLX 26
Gemma 7B IT Google LLM 8.5B GGUF 26
Granite 3.0 2B Instruct IBM LLM 2.6B GGUF 26
Granite Vision 3.2 2B IBM LLM 3.0B GGUF or MLX 26
LFM2 1.2B Liquid AI LLM 1.2B GGUF or MLX 26
Llama 2 13B Chat Meta LLM 13B GGUF 26
OLMoE 1B-7B Instruct Ai2 LLM 6.9B (1.3B active) GGUF or MLX 26
Qwen1.5 7B Chat Alibaba LLM 7.7B GGUF or MLX 26
Qwen2.5 Coder 1.5B Instruct Alibaba LLM 1.5B GGUF or MLX 26
Qwen3.5 0.8B Alibaba LLM 0.9B GGUF or MLX 26
StarCoder 15B BigCode LLM 16B GGUF 26
StarCoder2 3B BigCode LLM 3.0B GGUF or MLX 26
WizardMath 7B v1.1 WizardLM LLM 7.2B GGUF 26
Zephyr 7B Beta Hugging Face LLM 7.2B GGUF or MLX 26
Code Llama 7B Instruct Meta LLM 6.7B GGUF or MLX 24
LFM2-VL 1.6B Liquid AI LLM 1.6B GGUF or MLX 24
Qwen1.5 MoE A2.7B Chat Alibaba LLM 14B (2.7B active) MLX 24
Qwen2-VL 2B Instruct Alibaba LLM 2.2B GGUF or MLX 24
Qwen2.5 1.5B Instruct Alibaba LLM 1.5B GGUF or MLX 24
SmolVLM2 2.2B Instruct Hugging Face LLM 2.3B GGUF or MLX 24
StarCoder2 7B BigCode LLM 7.2B GGUF or MLX 24
DeepSeek R1 Distill Qwen 1.5B DeepSeek LLM 1.8B GGUF or MLX 22
Hunyuan 1.8B Instruct Tencent LLM 1.8B GGUF or MLX 22
Llama 3.2 1B Instruct Meta LLM 1.2B GGUF or MLX 22
Nemotron Mini 4B Instruct NVIDIA LLM 4.2B GGUF or MLX 22
Orca 2 7B Microsoft LLM 6.7B GGUF 22
SmolVLM 2.2B Instruct Hugging Face LLM 2.3B GGUF or MLX 22
WizardLM 13B v1.2 WizardLM LLM 13B GGUF 22
Baichuan 2 7B Chat Baichuan LLM 7.5B GGUF 20
Gemma 3 1B Google LLM 1.0B GGUF or MLX 20
Granite 3.1 3B-A800M Instruct IBM LLM 3.3B (0.8B active) GGUF 20
Granite Code 3B Instruct (128K) IBM LLM 3.5B GGUF 20
Llama 2 7B Chat Meta LLM 6.7B GGUF 20
Qwen2.5-Math 1.5B Instruct Alibaba LLM 1.5B GGUF or MLX 20
SmolLM2 1.7B Instruct Hugging Face LLM 1.7B GGUF or MLX 20
Stable Code Instruct 3B Stability AI LLM 2.8B GGUF or MLX 20
Vicuna 13B v1.5 LMSYS LLM 13B GGUF 20
Yi 6B Chat 01.AI LLM 6.1B GGUF or MLX 20
DeepSeek LLM 7B Chat DeepSeek LLM 6.9B GGUF 18
Granite 3.0 3B-A800M Instruct IBM LLM 3.4B (0.8B active) GGUF 18
InternLM2.5 1.8B Chat Shanghai AI Lab LLM 1.9B GGUF 18
LFM2 700M Liquid AI LLM 0.7B GGUF or MLX 18
Falcon 3 1B Instruct TII LLM 1.7B GGUF or MLX 16
Falcon 40B Instruct TII LLM 42B GGUF 16
LLaVA 1.5 7B LLaVA LLM 7.1B GGUF 16
Phi-2 2.7B Microsoft LLM 2.8B GGUF or MLX 16
Qwen2 1.5B Instruct Alibaba LLM 1.5B GGUF or MLX 16
StableLM Zephyr 3B Stability AI LLM 2.8B GGUF 16
Yi Coder 1.5B Chat 01.AI LLM 1.5B GGUF 16
CodeGemma 2B Google LLM 2.5B GGUF or MLX 14
Falcon-H1 0.5B Instruct TII LLM 0.5B GGUF or MLX 14
LFM2.5 350M Liquid AI LLM 0.4B GGUF or MLX 14
OLMo 2 1B Instruct Ai2 LLM 1.5B GGUF 14
Qwen3 0.6B Alibaba LLM 0.8B GGUF or MLX 14
SmolLM 1.7B Instruct Hugging Face LLM 1.7B GGUF or MLX 14
Vicuna 7B v1.5 LMSYS LLM 6.7B GGUF 14
DeepSeek Coder 1.3B Instruct DeepSeek LLM 1.4B GGUF or MLX 12
Gemma 2B IT Google LLM 2.5B GGUF 12
Granite 4.0 H 350M IBM LLM 0.3B GGUF or MLX 12
LFM2 350M Liquid AI LLM 0.4B GGUF or MLX 12
LFM2-VL 450M Liquid AI LLM 0.5B GGUF or MLX 12
LFM2.5 230M Liquid AI LLM 0.2B GGUF or MLX 10
Qwen2.5 0.5B Instruct Alibaba LLM 0.5B GGUF or MLX 10
SmolVLM 500M Instruct Hugging Face LLM 0.5B GGUF or MLX 10
StableLM 2 1.6B Chat Stability AI LLM 1.6B GGUF or MLX 10
StableLM 2 Zephyr 1.6B Stability AI LLM 1.6B GGUF or MLX 10
Hunyuan 0.5B Instruct Tencent LLM 0.5B GGUF or MLX 8
Qwen2.5 Coder 0.5B Instruct Alibaba LLM 0.5B GGUF or MLX 8
SmolLM2 360M Instruct Hugging Face LLM 0.4B GGUF or MLX 8
ERNIE 4.5 0.3B Baidu LLM 0.4B GGUF or MLX 6
Falcon 7B Instruct TII LLM 7.2B GGUF 6
Gemma 3 270M Google LLM 0.3B GGUF or MLX 6
Qwen2 0.5B Instruct Alibaba LLM 0.5B GGUF or MLX 6
SmolVLM 256M Instruct Hugging Face LLM 0.3B GGUF or MLX 6
TinyLlama 1.1B Chat TinyLlama LLM 1.1B GGUF or MLX 6
SmolLM 360M Instruct Hugging Face LLM 0.4B GGUF or MLX 5
SmolLM2 135M Instruct Hugging Face LLM 0.1B GGUF or MLX 4
Falcon-H1 Tiny 90M Instruct TII LLM 0.1B GGUF or MLX 3
SmolLM 135M Instruct Hugging Face LLM 0.1B GGUF or MLX 2

Alternatives

LM Studio Ollama ComfyUI Draw Things llama.cpp vLLM Jan Text Embeddings Inference MLX LM SGLang GLiNER Lemonade Infinity Sentence Transformers Docker Model Runner DiffusionBee LocalAI Atomic Chat JoyFusion

Which app should I use?

Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.