Apps for running AI locally

Every app that runs these models locally, plus the coding agents and assistants that use them: what each opens, which systems and GPUs it supports, and our pick for your setup.

Run LLMs on your Desktop

LM Studio Ollama llama.cpp Jan MLX LM macOS only vLLM SGLang Docker Model Runner Lemonade Unsloth Desktop Atomic Chat

Make images and video

Unsloth Desktop ComfyUI Draw Things macOS only DiffusionBee macOS only JoyFusion macOS only

System One · decision models

Apps that run decision models: safety classifiers, rerankers, judges and zero-shot classifiers that answer with a label, score or probability. Which one to use.

LM Studio Ollama llama.cpp Jan MLX LM macOS only vLLM SGLang Docker Model Runner Lemonade Unsloth Desktop Atomic Chat Text Embeddings Inference Infinity LocalAI Sentence Transformers GLiNER

Coding agents

Agents do the work: they read and edit your code and run commands. They do not run the AI model themselves. Each one talks to a model server such as Ollama, LM Studio or llama.cpp, so pointing one at a local model means no API bill and your code stays on your machine. Pick a model with tool calling.

OpenCode Codex CLI Pi OpenHands Zed Open Interpreter Cline Goose

Aider Tabby Crush Kilo Code Qwen Code

Personal assistants

Assistants for notes, documents, chat and everyday tasks beyond code. Most connect to a local model server such as Ollama; AnythingLLM brings its own engine.

OpenClaw Hermes Agent Open WebUI AnythingLLM Khoj

Letta

Which app should you use?

Our first pick for common setups. The same rules run on every model page with your exact hardware. These picks are model-running apps; to edit code or run tasks with the model, add one of the coding agents or assistants on top.

Step 1

Tell us your computer first

The table below shows examples for common setups, not your own. To see which app we would pick for you, choose your computer.

Detect my hardware Pick my GPU or Mac

Common setups

Example computer	Small model that fits8B	Bigger than GPU memory70B	Serving a team32B
Mac with Apple silicon	LM Studio Runs the MLX build, typically 20–35% faster than GGUF for a model with 8B active parameters on Apple silicon.	LM Studio Splits the model between GPU memory and system RAM, which this setup needs.	LM Studio Runs the MLX build, typically up to 10% faster than GGUF for a model with 32B active parameters on Apple silicon.
Windows PC, NVIDIA GeForce	Ollama One command with Ollama’s own build of this model.	LM Studio Splits the model between GPU memory and system RAM, which this setup needs.	Ollama One command with Ollama’s own build of this model.
Windows PC, AMD Radeon	Lemonade AMD’s own app: llama.cpp tuned for Radeon on Vulkan and ROCm.	llama.cpp Splits the model between GPU memory and system RAM, which this setup needs.	Lemonade AMD’s own app: llama.cpp tuned for Radeon on Vulkan and ROCm.
Windows PC, Intel Arc	llama.cpp Supports Intel Arc through SYCL and Vulkan.	llama.cpp Splits the model between GPU memory and system RAM, which this setup needs.	llama.cpp Supports Intel Arc through SYCL and Vulkan.
Ryzen AI mini PC or laptop	Lemonade Built for Ryzen AI and Strix Halo, including NPU + iGPU modes on Windows.	Lemonade Built for Ryzen AI and Strix Halo, including NPU + iGPU modes on Windows.	Lemonade Built for Ryzen AI and Strix Halo, including NPU + iGPU modes on Windows.
Linux, NVIDIA GeForce	Ollama One command with Ollama’s own build of this model.	LM Studio Splits the model between GPU memory and system RAM, which this setup needs.	Ollama One command with Ollama’s own build of this model.
Linux server, datacenter GPUs	vLLM The full model fits in GPU memory at 8-bit, so it can batch many requests: best for serving a team or an app.	vLLM The full model fits in GPU memory at 8-bit, so it can batch many requests: best for serving a team or an app.	vLLM The full model fits in GPU memory at 8-bit, so it can batch many requests: best for serving a team or an app.

Which app for decision models?

Scorers that read the input once (rerankers, classifiers, reward models) belong in a scoring server or a Python library; guard and judge models that write a short answer run in the same apps as LLMs. Agents are not listed here: they call a model server and do not run decision models themselves.

Step 1

Tell us your computer first

The table below shows examples for common setups, not your own. To see which app we would pick for your decision models, choose your computer.

Detect my hardware Pick my GPU or Mac

Common setups

Example computer	Safety checkQwen3Guard 4B	RerankerBGE Reranker v2 M3	Zero-shot classifierDeBERTa	Reward judgeSkywork 8B	Label decisionsGLiNER2.5 Decide
Mac with Apple silicon	LM Studio Point-and-click: search, download and chat in one window.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Infinity Local API for rerankers and classifiers, including models TEI does not support.	Sentence Transformers A few lines of Python with Transformers or Sentence Transformers; easiest for scripts and notebooks.	GLiNER Made for this model family: every label is scored in one pass, fast even on a CPU.
Windows PC, NVIDIA GeForce	LM Studio Point-and-click: search, download and chat in one window.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Sentence Transformers A few lines of Python with Transformers or Sentence Transformers; easiest for scripts and notebooks.	Sentence Transformers A few lines of Python with Transformers or Sentence Transformers; easiest for scripts and notebooks.	GLiNER Made for this model family: every label is scored in one pass, fast even on a CPU.
Windows PC, AMD Radeon	Lemonade AMD’s own app: llama.cpp tuned for Radeon on Vulkan and ROCm.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Sentence Transformers A few lines of Python with Transformers or Sentence Transformers; easiest for scripts and notebooks.	Sentence Transformers A few lines of Python with Transformers or Sentence Transformers; easiest for scripts and notebooks.	GLiNER Can only use the CPU on this hardware, so it will be slow.
Windows PC, Intel Arc	llama.cpp Supports Intel Arc through SYCL and Vulkan.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Sentence Transformers Can only use the CPU on this hardware, so it will be slow.	vLLM Serves the scoring head directly (rerank, classify or reward) with batching.	GLiNER Can only use the CPU on this hardware, so it will be slow.
Linux, NVIDIA GeForce	LM Studio Point-and-click: search, download and chat in one window.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Infinity Local API for rerankers and classifiers, including models TEI does not support.	vLLM Serves the scoring head directly (rerank, classify or reward) with batching.	GLiNER Made for this model family: every label is scored in one pass, fast even on a CPU.
Linux server, datacenter GPUs	vLLM The full model fits in GPU memory at 8-bit, so it can batch many requests: best for serving a team or an app.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Infinity Local API for rerankers and classifiers, including models TEI does not support.	vLLM Serves the scoring head directly (rerank, classify or reward) with batching.	GLiNER Made for this model family: every label is scored in one pass, fast even on a CPU.

System One models answer with a label, a score or a probability that code can use directly, instead of writing text. They are the fast, focused judgments behind safety filters, search ranking, answer grading and routing. Chat LLMs do the opposite job: slower, open-ended reasoning and writing.

Precision: How often it decides correctly, from the benchmark on each model page (F1, nDCG@10, RewardBench). Shown as a 0 to 100 tier.
Speed: Time for one decision on your hardware: reading the input, plus any words it writes. Encoders and rerankers take milliseconds; judges that write feedback take seconds.
Clarity: How easy the answer is to use in code. A probability or score is best, then labeled spans, fixed labels, a reasoned verdict, and written feedback before a score.
Usage: The task, what it checks, how much text it reads, its languages and license.

Prefer not to host one? TypeSafe’s Jev is a hosted System One model that returns typed answers and probabilities through an API; everything listed here runs on your own computer.

Compare every app

39 apps: 20 that run models and 19 agents and assistants that use them. Filter by type, operating system, GPU or file format from the column headers. Agents show “Via server” for GPUs because they use whatever your model server runs on.

Step 1

Tell us your computer first

The table below shows examples for common setups, not your own. To see which of these apps work on your computer, choose it.

Detect my hardware Pick my GPU or Mac

App	Type	Runs	Systems	GPUs	Formats	Engine	Models	Stars	License
LM StudioElement Labs	Desktop app	LLMs, Decisions	macOSWindowsLinux	Apple, MLX, NVIDIA, Vulkan, AMD, CPU	GGUF, MLX	llama.cpp, MLX	532	5,383	Free, closed
OllamaOllama	CLI + API	LLMs, Decisions	macOSWindowsLinux	Apple, MLX, NVIDIA, AMD, Vulkan, CPU	GGUF	llama.cpp, MLX	521	184,745	Open source
llama.cppggml.org	CLI + API	LLMs, Decisions	macOSWindowsLinux	Apple, NVIDIA, AMD, Vulkan, Intel, CPU	GGUF	llama.cpp	524	132,342	Open source
JanMenlo Research	Desktop app	LLMs, Decisions	macOSWindowsLinux	Apple, NVIDIA, Vulkan, CPU, MLX	GGUF	llama.cpp, MLX	532	45,082	Open source
MLX LMApple (ml-explore)	CLI + API	LLMs, Decisions	macOS	MLX	MLX	MLX	395	7,475	Open source
vLLMvLLM project	Server	LLMs, Decisions	WSLWindowsLinux	NVIDIA, AMD, Intel	Safetensors, AWQ, GPTQ, FP8	vLLM	579	95,479	Open source
SGLangLMSYS / SGLang project	Server	LLMs, Decisions	WSLWindowsLinux	NVIDIA, AMD	Safetensors, AWQ, GPTQ, FP8	SGLang	569	38,156	Open source
Docker Model RunnerDocker	CLI + API	LLMs, Decisions	macOSWindowsLinux	Apple, NVIDIA, AMD, Vulkan, CPU	GGUF, Safetensors	llama.cpp, vLLM	562	658	Open source
LemonadeAMD (lemonade-sdk)	Desktop app	LLMs, Decisions	macOSWindowsLinux	Vulkan, AMD, NPU, Apple, CPU	GGUF, ONNX	llama.cpp, ONNX Runtime GenAI	524	5,994	Open source
Unsloth DesktopUnsloth AI	Desktop app	LLMs, Images, Decisions	macOSWindowsLinux	NVIDIA, AMD, Intel, Apple, MLX, CPU	GGUF, MLX	llama.cpp, MLX, stable-diffusion.cpp	550	78,556	Open source
Atomic ChatAtomicBot	Desktop app	LLMs, Decisions	macOSWindowsLinux	Apple, MLX, NVIDIA, Vulkan, CPU	GGUF, MLX	llama.cpp, MLX-VLM	532	1,845	Open source
PiEarendil	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	190	117,771	Open source
Hermes AgentNous Research	Assistant	LLMs	macOSWSLWindowsLinux	Via server	GGUF, MLX	Your model server	190	257,038	Open source
OpenClawOpenClaw	Assistant	LLMs	macOSWSLWindowsLinux	Via server	GGUF, MLX	Your model server	190	393,046	Open source
OpenCodeAnomaly	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	190	222,100	Open source
Codex CLIOpenAI	Coding agent	LLMs	macOSWSLWindowsLinux	Via server	GGUF, MLX	Your model server	190	119,631	Open source
OpenHandsOpenHands	Coding agent	LLMs	macOSWSLWindowsLinux	Via server	GGUF, MLX	Your model server	190	92,000	Open source
ZedZed Industries	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	190	88,500	Open source
ClineCline	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	190	67,600	Open source
Open InterpreterOpen Interpreter	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	190	68,700	Open source
GooseAgentic AI Foundation	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	190	55,800	Open source
AiderAider	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	190	49,700	Open source
TabbyTabbyML	Coding agent	LLMs	macOSWSLWindowsLinux	Own engine	GGUF, MLX	runs its own models	—	33,900	Open source
CrushCharm	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	190	27,900	Source-available
Qwen CodeQwen	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	190	27,300	Open source
Kilo CodeKilo Code	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	190	27,800	Open source
Open WebUIOpen WebUI	Assistant	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	190	156,400	Source-available
AnythingLLMMintplex Labs	Assistant	LLMs	macOSWindowsLinux	Own engine	GGUF, MLX	built-in engine	—	67,500	Open source
KhojKhoj	Assistant	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	190	37,500	Open source
LettaLetta	Assistant	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	190	25,200	Open source
ComfyUIComfy Org	Desktop app	Images, Video	macOSWindowsLinux	NVIDIA, AMD, Intel, Apple, CPU	Safetensors, GGUF	PyTorch, GGUF via ComfyUI-GGUF	77	142,909	Open source
Draw ThingsDraw Things	Desktop app	Images, Video	macOS	Apple	Safetensors, Draw Things format	Draw Things	46	603	Free, closed
DiffusionBeeDivam Gupta	Desktop app	Images	macOS	Apple	Safetensors	DiffusionBee	14	13,596	Open source
JoyFusionJoyFusion	Desktop app	Images	macOS	Core ML	Core ML	Core ML	4	—	Free, closed
Text Embeddings InferenceHugging Face	Server	Decisions	macOSWSLWindowsLinux	NVIDIA, Apple, CPU	Safetensors	TEI	6	5,087	Open source
InfinityMichael Feil	Server	Decisions	macOSWSLWindowsLinux	NVIDIA, AMD, Apple, CPU	Safetensors, ONNX	PyTorch, ONNX, CTranslate2	12	2,968	Open source
LocalAIEttore Di Giacinto (mudler)	Server	Decisions	macOSWSLWindowsLinux	NVIDIA, AMD, Intel, Vulkan, Apple, CPU	GGUF, Safetensors	llama.cpp, rerankers, Transformers	30	49,957	Open source
Sentence TransformersHugging Face	Library	Decisions	macOSWindowsLinux	NVIDIA, AMD, Apple, CPU	Safetensors, ONNX	PyTorch	57	19,219	Open source
GLiNERUrchade Zaratiana, Fastino and Knowledgator	Library	Decisions	macOSWindowsLinux	NVIDIA, Apple, CPU	Safetensors, ONNX	PyTorch, ONNX	6	4,159	Open source

How we choose an app

The files that exist for the model. GGUF opens in llama.cpp, Ollama, LM Studio, Jan, Lemonade and Docker; MLX opens in LM Studio, MLX LM and others on Apple silicon; the original safetensors weights need vLLM or SGLang. We check Hugging Face and Ollama for every model, the same signals Hugging Face uses for its “Use this model” menu.
Your operating system and GPU. MLX is Mac-only; vLLM and SGLang are Linux (Windows through WSL2); Lemonade is built for AMD Radeon and Ryzen AI; llama.cpp has the broadest GPU support (CUDA, ROCm, Vulkan, SYCL, Metal).
Whether the model fits. If part of it has to sit in system RAM, only llama.cpp-based apps can split it; vLLM, SGLang and MLX need it all in GPU or unified memory.
Published speed tests. On an M4 Max, MLX generated 21–87% faster than llama.cpp for 4-bit models up to 8B and tied around 27B; llama.cpp still prefills long prompts faster. On an RTX 4090, vLLM (54 tok/s) and llama.cpp (50 tok/s) were close for one user, Ollama’s draft decoding reached 76 tok/s, and with spare memory vLLM handled 32 users far better.
Decision models. Guard and judge models that write a short answer follow the LLM rules above. Rerankers, classifiers and reward models only score their input, so they go to a scoring server (Text Embeddings Inference for BERT, XLM-RoBERTa and ModernBERT models; Infinity for DeBERTa; vLLM or SGLang with big GPUs), to llama.cpp when a GGUF build exists, or to a few lines of Python. GLiNER-family models need the GLiNER libraries. Ollama has no rerank endpoint, so it only runs the guard and judge models.
Agents and assistants. Coding agents, personal assistants and chat front ends are not ranked against the apps above, because they do not run a model: they send requests to one. We list them by GitHub stars and show which need a model server and which, like Tabby and AnythingLLM, bring their own. For a local model to work as an agent it must support tool calling.
Ease and popularity. Point-and-click apps rank above command-line tools for one person; GitHub stars and maintenance break ties.

Sources: Hugging Face local-apps rules · MLX vs llama.cpp on Apple silicon · Ollama vs vLLM vs llama.cpp on RTX 4090, 5090 and L40S. Speeds vary with versions, drivers and settings.

Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.