Add the computers, Macs, mini PCs and boards you own, and how they’re connected. We work out which models they can run together, how fast, and which free tool to use.
230 GBusable memory together (the largest device has 115 GB)
llama.cpp RPCbest estimate for 367 of the 401 models the cluster adds
Mixed devices you already own, a Mac, a gaming PC, a mini PC or a Jetson, when one GGUF model won't fit on any of them alone.
ggml-rpc-server --host 0.0.0.0 --port 50052 -c # on each worker
llama-server -hf <model-GGUF> -ngl 99 --rpc 192.168.1.10:50052,192.168.1.11:50052 # on the main machine
Also works. The number is how many of the added models each one runs:
Watch it run: one device, the cluster, every device
The same answer on your devices, at our estimated speeds. First the model reads your prompt (the devices light up blue), then it types. In a pipeline split the devices take turns on every word; in a tensor split they all work on each word together.
576 LLMs your devices can run
Models the cluster adds come first, best match first, then the ones a single device already runs as well. Filter vs one device to pick. Speeds are estimates for one person chatting.
Quality
Model
Quant
Memory
Speed
Runs
Devices
Tool
vs one device
A+91Excellent quality (A+) · 91 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.
A+92Excellent quality (A+) · 92 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.
A+90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.
A+87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.
A+86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.
A80Great quality (A) · 80 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
A79Great quality (A) · 79 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
A79Great quality (A) · 79 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
A78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
A79Great quality (A) · 79 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
A79Great quality (A) · 79 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
A78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
B67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
B67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
B66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
B65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
A76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
A70Great quality (A) · 70 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
B68Good quality (B) · 68 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
B60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
B63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
B58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
B67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
B63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
B60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
D35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
A+95Excellent quality (A+) · 95 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 58.5 tok/s
A+90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 34.9 tok/s
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 34.1 tok/s
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 34.9 tok/s
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 47.0 tok/s
A+87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 47.0 tok/s
A+86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 47.0 tok/s
A+91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 30.1 tok/s
A+91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 30.1 tok/s
A84Great quality (A) · 84 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 47.0 tok/s
A+91Excellent quality (A+) · 91 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 41.5 tok/s
A+90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 25.5 tok/s
A+85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 32.9 tok/s
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 23.2 tok/s
A83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 22.3 tok/s
A+91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 33.1 tok/s
A83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 20.6 tok/s
A+87Excellent quality (A+) · 87 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 23.2 tok/s
A81Great quality (A) · 81 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 39.2 tok/s
A+86Excellent quality (A+) · 86 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 23.2 tok/s
A+85Excellent quality (A+) · 85 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 25.5 tok/s
A+85Excellent quality (A+) · 85 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 23.2 tok/s
A+93Excellent quality (A+) · 93 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A83Great quality (A) · 83 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 36.7 tok/s
A+93Excellent quality (A+) · 93 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 39.7 tok/s
A+93Excellent quality (A+) · 93 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A+92Excellent quality (A+) · 92 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A+91Excellent quality (A+) · 91 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A+91Excellent quality (A+) · 91 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A83Great quality (A) · 83 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 18.5 tok/s
A81Great quality (A) · 81 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 23.2 tok/s
A84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 56.9 tok/s
A79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 39.3 tok/s
A+92Excellent quality (A+) · 92 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 38.3 tok/s
A+90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 39.7 tok/s
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 39.2 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 39.2 tok/s
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 45.2 tok/s
A73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 37.1 tok/s
A75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 39.2 tok/s
A78Great quality (A) · 78 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 25.5 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 24.2 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 24.2 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 24.2 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 25.5 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.2 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 30.1 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 25.5 tok/s
A80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 41.1 tok/s
A74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 25.5 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 24.2 tok/s
A+94Excellent quality (A+) · 94 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 10.1 tok/s
A+90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 12.1 tok/s
A+85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.5 tok/s
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 12.1 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 41.1 tok/s
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 12.1 tok/s
A78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 48.1 tok/s
A82Great quality (A) · 82 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 28.0 tok/s
A+87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 11.3 tok/s
A+86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.7 tok/s
A+85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.1 tok/s
A79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
IOK2 Horizon 7BInstitute of Foundation Models · 9.0B
Better on the clusterAlone: Runs great on Spark / Jetson 1, 37.2 tok/s
A79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.4 tok/s
A83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.1 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 24.2 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 23.7 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 34.7 tok/s
A79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.5 tok/s
A81Great quality (A) · 81 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.0 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 35.6 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 34.7 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 18.5 tok/s
A81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.1 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.5 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.5 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Spark / Jetson 1, 1.8 tok/s
A75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 34.7 tok/s
A79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.0 tok/s
A74Great quality (A) · 74 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 34.7 tok/s
A73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 37.4 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 12.6 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 51.7 tok/s
B69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 35.6 tok/s
B69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 35.6 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.9 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 51.7 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.6 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 12.0 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 11.0 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 40.9 tok/s
B67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 38.1 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 14.0 tok/s
A75Great quality (A) · 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 12.0 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.9 tok/s
A74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.2 tok/s
A74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.0 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.2 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 33.1 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 11.4 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 41.5 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.4 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.6 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.2 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.2 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.2 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 11.3 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 14.0 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 24.0 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.5 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.7 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 12.2 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.7 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.7 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.7 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.3 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.3 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.4 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 14.2 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 14.2 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 11.6 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 43.4 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 38.2 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 39.3 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 49.9 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.5 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 40.9 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.4 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 24.0 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.8 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 12.4 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.2 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.7 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.2 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 40.5 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 14.2 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 49.1 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 14.0 tok/s
B59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 38.1 tok/s
B59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 38.5 tok/s
B59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 40.9 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 51.2 tok/s
B59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 37.7 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.7 tok/s
B59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 32.6 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.2 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.2 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.2 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 16.0 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 12.1 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.8 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.7 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 27.5 tok/s
B67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.2 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 11.3 tok/s
B63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 12.5 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 14.2 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 14.0 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 35.7 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 40.3 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 40.6 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 51.2 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 12.2 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.0 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.7 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 10.7 tok/s
B63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.2 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 14.2 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 49.1 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 37.6 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 40.6 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 42.8 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 42.9 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 42.2 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 44.2 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.2 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 44.0 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 37.9 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 15.1 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 25.5 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 44.0 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 14.0 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 13.8 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 52.4 tok/s
C53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 12.3 tok/s
C53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 40.9 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.5 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.3 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.3 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 12.9 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 23.9 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.2 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.2 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 14.2 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 15.1 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 44.7 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 44.0 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.4 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 35.7 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 35.6 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.3 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.3 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 38.7 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 40.4 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 22.2 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 32.5 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 23.9 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 15.1 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 27.5 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 37.9 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 42.2 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 44.0 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 36.3 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.3 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.4 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 49.1 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.5 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 38.1 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 40.4 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.1 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.4 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 44.2 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.3 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.2 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.0 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 27.0 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 35.4 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 35.7 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 43.9 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 51.2 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.4 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 15.1 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.3 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 44.9 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.2 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 44.0 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 12.4 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 37.7 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 35.7 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.0 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.0 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.8 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 42.9 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 31.2 tok/s
C53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.2 tok/s
C53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.2 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 48.1 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 24.0 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 16.9 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 24.0 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 44.0 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.0 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 51.6 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.5 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 38.8 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 40.4 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 45.3 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 24.4 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 12.4 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 39.8 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.3 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 31.4 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 40.1 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.2 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.3 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 38.0 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.6 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 43.3 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.0 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 12.6 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 50.3 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.2 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 31.2 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 31.2 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.2 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 45.5 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.1 tok/s
D37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 38.0 tok/s
D37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.1 tok/s
D37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 42.9 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 21.0 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 16.9 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 45.9 tok/s
D37Basic quality (D) · 37 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 49.9 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.3 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.3 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.2 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.0 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 44.0 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 38.0 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 49.7 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.3 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.1 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 16.9 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 39.2 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.3 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.2 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.3 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.3 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.3 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.1 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 23.6 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 12.5 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 34.8 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 17.2 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.7 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.1 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.3 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.3 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 3.1 tok/s
D31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.6 tok/s
D31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 43.3 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.1 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 16.7 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.3 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 33.3 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 12.6 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.1 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on Spark / Jetson 1, 42.0 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 12.8 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 21.0 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 12.8 tok/s
D27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 39.2 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.3 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 45.7 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 4.9 tok/s
D27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 44.3 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 33.3 tok/s
D27Basic quality (D) · 27 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 35.7 tok/s
D25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 39.2 tok/s
D25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 43.4 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.3 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.3 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 39.6 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 33.3 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.1 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 5.0 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 46.7 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Spark / Jetson 1, 27.4 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.1 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.2 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 33.3 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Spark / Jetson 1, 4.9 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 41.1 tok/s
D19Basic quality (D) · 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 44.6 tok/s
D19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 42.3 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 45.7 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 40.1 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on Spark / Jetson 1, 47.5 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Spark / Jetson 1, 10.4 tok/s
A81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 54.2 tok/s
A79Great quality (A) · 79 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A82Great quality (A) · 82 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A82Great quality (A) · 82 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A80Great quality (A) · 80 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
A74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 43.2 tok/s
A74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 43.2 tok/s
A73Great quality (A) · 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 43.2 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 43.2 tok/s
A73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 55.0 tok/s
B69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 56.0 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 42.9 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 43.2 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 43.2 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 43.2 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 74.6 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
B69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 54.2 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 42.9 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 57.8 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 47.8 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 74.2 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
B59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 67.7 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 45.1 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 45.4 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 56.0 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 42.6 tok/s
C53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 42.4 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 42.4 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 41.0 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.6 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.8 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 42.4 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 43.0 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 58.5 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 83.3 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 50.2 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 42.6 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 44.4 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 68.8 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 56.0 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 63.2 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 83.3 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 44.4 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 44.3 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 54.7 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 53.3 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 60.5 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 81.9 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 42.6 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 39.7 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 55.4 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 55.4 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 39.7 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 41.8 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 50.7 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 56.3 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 75.2 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 44.7 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 40.4 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 50.2 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 85.3 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 41.1 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 53.5 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 80.1 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 72.8 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 44.7 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 41.9 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 44.7 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 54.2 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 55.2 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 66.4 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 53.1 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 45.5 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 41.1 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 55.2 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 67.2 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 53.1 tok/s
D29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 67.4 tok/s
D29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 67.4 tok/s
D29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 70.8 tok/s
D29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 84.0 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 61.4 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 52.8 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 49.5 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 47.8 tok/s
D27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 67.4 tok/s
D27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 65.4 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 53.9 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 69.9 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 50.9 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 44.2 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 51.1 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 50.7 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 90.4 tok/s
D25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 64.9 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 67.2 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 57.2 tok/s
D23Basic quality (D) · 23 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 75.8 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 51.1 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 49.8 tok/s
D23Basic quality (D) · 23 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 77.2 tok/s
D21Basic quality (D) · 21 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 75.8 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 43.9 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 44.2 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 63.4 tok/s
D21Basic quality (D) · 21 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 64.1 tok/s
D21Basic quality (D) · 21 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 41.0 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 68.7 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.0 tok/s
D19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 49.0 tok/s
D19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 60.9 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 51.1 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 40.7 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 78.6 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 68.7 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 41.6 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 106 tok/s
D17Basic quality (D) · 17 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 56.3 tok/s
D17Basic quality (D) · 17 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 41.8 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 53.1 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 47.1 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.0 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 51.1 tok/s
D15Basic quality (D) · 15 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 60.9 tok/s
D15Basic quality (D) · 15 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 61.4 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 41.1 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 40.2 tok/s
D15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 114 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 105 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.0 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 53.1 tok/s
D13Basic quality (D) · 13 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 68.0 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 151 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 225 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 45.8 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 58.2 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 58.2 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 68.0 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 175 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 225 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 231 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 105 tok/s
D10Basic quality (D) · 10 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 47.9 tok/s
D10Basic quality (D) · 10 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 47.9 tok/s
D10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 154 tok/s
D10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 160 tok/s
D10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 342 tok/s
D8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 146 tok/s
D8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 160 tok/s
D8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 218 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 46.4 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 71.5 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 160 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 218 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 302 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 291 tok/s
D5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 218 tok/s
D4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 605 tok/s
D3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 874 tok/s
D2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Spark / Jetson 1, 605 tok/s
Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.
NVIDIA GB10 (128 GB) 
