Four Raspberry Pi 5s (16 GB): what it runs together
Four 16 GB Raspberry Pi 5 boards on a gigabit switch.
A cheap way to learn how clustering works. Distributed Llama splits every layer across all four boards, so small models type more than twice as fast as on one Pi (about 14 tokens per second for Llama 3.2 3B). Pipeline tools only add memory: big models fit, but at well under a token per second.
llama.cpp RPCbest estimate for 182 of the 240 models the cluster adds
Mixed devices you already own, a Mac, a gaming PC, a mini PC or a Jetson, when one GGUF model won't fit on any of them alone.
ggml-rpc-server --host 0.0.0.0 --port 50052 -c # on each worker
llama-server -hf <model-GGUF> -ngl 99 --rpc 192.168.1.10:50052,192.168.1.11:50052 # on the main machine
Also works. The number is how many of the added models each one runs:
Watch it run: one device, the cluster, every device
The same answer on this cluster, at our estimated speeds. First the model reads your prompt (the devices light up blue), then it types. In a pipeline split the devices take turns on every word; in a tensor split they all work on each word together.
467 LLMs this cluster can run
Models the cluster adds come first, best match first, then the ones a single device already runs as well. Filter vs one device to pick. Speeds are estimates for one person chatting.
Quality
Model
Quant
Memory
Speed
Runs
Devices
Tool
vs one device
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
A+92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
A+91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
A+90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
A+87Excellent quality (A+) · 87 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
A+95Excellent quality (A+) · 95 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
A+86Excellent quality (A+) · 86 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
A+86Excellent quality (A+) · 86 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
A+86Excellent quality (A+) · 86 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
A82Great quality (A) · 82 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
A+90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
A81Great quality (A) · 81 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
A76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
A72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
A75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
A82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
A72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
A73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
A70Great quality (A) · 70 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
A73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
A77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
A78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
A78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
A77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
A76Great quality (A) · 76 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
B63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
A76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
A76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
A76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
A75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
B67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
A75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
A73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
A74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
A73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
A73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
A76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
A72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
A71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
A71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
A71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
B64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
B61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
B67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
B68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
A70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
A70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
B67Good quality (B) · 67 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
B69Good quality (B) · 69 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
B69Good quality (B) · 69 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
B66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
B58Good quality (B) · 58 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
B62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
B64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
B64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
B64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
B61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
B58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
C54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
B62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
B59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
B59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
C53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
B56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
B56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
B60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
B60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
C54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
C52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
B57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
C49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
C53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
C51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
B56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
B55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
B55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
B55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
C50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
C47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
C48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
C49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
C49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
C44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
D37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
D36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
D15Basic quality (D) · 15 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.4 tok/s
A+87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.9 tok/s
A+95Excellent quality (A+) · 95 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.0 tok/s
A+94Excellent quality (A+) · 94 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.0 tok/s
A+91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 0.8 tok/s
A84Great quality (A) · 84 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.9 tok/s
A+93Excellent quality (A+) · 93 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.0 tok/s
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
A79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.9 tok/s
A+86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
A79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 6.8 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.2 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.2 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.2 tok/s
A83Great quality (A) · 83 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 6.8 tok/s
A75Great quality (A) · 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.0 tok/s
A73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.2 tok/s
A71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.9 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Pi / ARM board 1, 14.5 tok/s
B69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.4 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 9.3 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.4 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.0 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B67Good quality (B) · 67 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
B65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.0 tok/s
B59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 4.7 tok/s
B65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.0 tok/s
B63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.4 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 0.9 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.0 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 9.8 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 5.7 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.0 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.1 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.3 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 2.7 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.5 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.0 tok/s
C53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.0 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
C53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.2 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.0 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.3 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.8 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.4 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.8 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.5 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.0 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.8 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Pi / ARM board 1, 12.7 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.6 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 4.5 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.6 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
D31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
D29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.9 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.8 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.5 tok/s
D25Basic quality (D) · 25 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 7.9 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.4 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Pi / ARM board 1, 26.0 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Pi / ARM board 1, 10.6 tok/s
D25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.7 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 5.4 tok/s
D25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 5.7 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.7 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 5.7 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.8 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Pi / ARM board 1, 11.1 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 4.7 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Pi / ARM board 1, 18.2 tok/s
D21Basic quality (D) · 21 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.7 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Pi / ARM board 1, 11.4 tok/s
D19Basic quality (D) · 19 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.7 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs well on Pi / ARM board 1, 11.4 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Slow on Pi / ARM board 1, 3.2 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.7 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on Pi / ARM board 1, 1.7 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Pi / ARM board 1, 10.2 tok/s
D19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Pi / ARM board 1, 14.7 tok/s
D15Basic quality (D) · 15 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on Pi / ARM board 1, 18.7 tok/s
D19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on Pi / ARM board 1, 3.4 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
A74Great quality (A) · 74 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 4.3 tok/s
B67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.2 tok/s
B65Good quality (B) · 65 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.2 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.3 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 12.9 tok/s
B65Good quality (B) · 65 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.5 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 12.0 tok/s
B59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.0 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.2 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.5 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.1 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 12.0 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Very slow on Pi / ARM board 1, 1.2 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
IOK2 Horizon 7BInstitute of Foundation Models · 9.0B
One device is enoughAlone: Slow on Pi / ARM board 1, 3.2 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.2 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 16.5 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.5 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.2 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.3 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.0 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.3 tok/s
C53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 10.1 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.2 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 4.4 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Very slow on Pi / ARM board 1, 1.1 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.0 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.5 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.4 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 4.6 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.5 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.3 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 12.5 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.3 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.3 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.2 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.3 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 4.6 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.5 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.5 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.9 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 4.6 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 5.1 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 4.6 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.5 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.2 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.4 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 18.2 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.7 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.0 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.4 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 4.1 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 4.1 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.8 tok/s
D37Basic quality (D) · 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.4 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 4.1 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 13.5 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.8 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 5.6 tok/s
D25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.4 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 5.7 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 12.9 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 5.5 tok/s
D37Basic quality (D) · 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.8 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.8 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D37Basic quality (D) · 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.5 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.3 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.4 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 4.6 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 5.1 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Very slow on Pi / ARM board 1, 1.4 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.3 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.0 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 5.8 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.5 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.5 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.3 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.5 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 16.2 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.3 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.7 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.8 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.5 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.5 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 4.3 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.8 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D29Basic quality (D) · 29 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.4 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.4 tok/s
D19Basic quality (D) · 19 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.4 tok/s
D29Basic quality (D) · 29 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 5.5 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.8 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 5.6 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.4 tok/s
D31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.7 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.3 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 15.0 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 5.4 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 13.7 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.5 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 10.6 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.3 tok/s
D21Basic quality (D) · 21 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Very slow on Pi / ARM board 1, 1.3 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 5.4 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 32.3 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.7 tok/s
D23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.3 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.6 tok/s
D19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 5.0 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.4 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 32.3 tok/s
D21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 12.0 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.7 tok/s
D13Basic quality (D) · 13 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 41.3 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 15.0 tok/s
D25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.1 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.0 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 10.1 tok/s
D9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 38.3 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 38.8 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 5.9 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.3 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 43.5 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 41.3 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 42.5 tok/s
D23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.7 tok/s
D23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.2 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 38.9 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 10.1 tok/s
D19Basic quality (D) · 19 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.8 tok/s
D10Basic quality (D) · 10 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 41.6 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 10.3 tok/s
D21Basic quality (D) · 21 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.9 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 13.2 tok/s
D21Basic quality (D) · 21 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.4 tok/s
D21Basic quality (D) · 21 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.4 tok/s
D15Basic quality (D) · 15 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.3 tok/s
D21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 12.7 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.2 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.1 tok/s
D9Basic quality (D) · 9 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 52.8 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.8 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 41.9 tok/s
D13Basic quality (D) · 13 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.2 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 28.7 tok/s
D8Basic quality (D) · 8 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 44.3 tok/s
D19Basic quality (D) · 19 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.4 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 4.2 tok/s
D7Basic quality (D) · 7 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 40.1 tok/s
D8Basic quality (D) · 8 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 46.2 tok/s
D19Basic quality (D) · 19 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.0 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 19.4 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 12.6 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.4 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 12.7 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 46.2 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 10.9 tok/s
D8Basic quality (D) · 8 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 21.3 tok/s
D8Basic quality (D) · 8 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 21.3 tok/s
D5Basic quality (D) · 5 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 46.7 tok/s
D5Basic quality (D) · 5 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 40.1 tok/s
D5Basic quality (D) · 5 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 45.0 tok/s
D15Basic quality (D) · 15 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 12.0 tok/s
D5Basic quality (D) · 5 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 46.2 tok/s
D17Basic quality (D) · 17 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.8 tok/s
D4Basic quality (D) · 4 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 40.1 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 3.7 tok/s
D4Basic quality (D) · 4 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 68.5 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 17.2 tok/s
D17Basic quality (D) · 17 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Slow on Pi / ARM board 1, 5.8 tok/s
D3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 45.6 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 10.3 tok/s
D9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 19.4 tok/s
D2Basic quality (D) · 2 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on Pi / ARM board 1, 68.5 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 16.8 tok/s
D9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 16.8 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 10.3 tok/s
D7Basic quality (D) · 7 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 17.5 tok/s
D7Basic quality (D) · 7 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 17.5 tok/s
D4Basic quality (D) · 4 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 26.1 tok/s
D9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.4 tok/s
D8Basic quality (D) · 8 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on Pi / ARM board 1, 11.4 tok/s
D4Basic quality (D) · 4 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking.
Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.
Raspberry Pi 5 (16 GB) 
