Two gaming PCs (RTX 3090 + RTX 4090): what it runs together
One PC with an RTX 3090 and one with an RTX 4090, 64 GB of RAM each, on 2.5 Gb Ethernet.
48 GB of fast graphics memory between them: a 70B model runs at about 18 tokens per second at 3-bit across the two cards, several times faster than one card spilling into system RAM, and gpt-oss-120b fits with help from system RAM.
llama.cpp RPCbest estimate for 138 of the 143 models the cluster adds
Mixed devices you already own, a Mac, a gaming PC, a mini PC or a Jetson, when one GGUF model won't fit on any of them alone.
ggml-rpc-server --host 0.0.0.0 --port 50052 -c # on each worker
llama-server -hf <model-GGUF> -ngl 99 --rpc 192.168.1.10:50052,192.168.1.11:50052 # on the main machine
Also works. The number is how many of the added models each one runs:
Watch it run: one device, the cluster, every device
The same answer on this cluster, at our estimated speeds. First the model reads your prompt (the devices light up blue), then it types. In a pipeline split the devices take turns on every word; in a tensor split they all work on each word together.
550 LLMs this cluster can run
Models the cluster adds come first, best match first, then the ones a single device already runs as well. Filter vs one device to pick. Speeds are estimates for one person chatting.
Quality
Model
Quant
Memory
Speed
Runs
Devices
Tool
vs one device
A+91Excellent quality (A+) · 91 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
A84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
A+87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
A78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
A77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
A76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
A74Great quality (A) · 74 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
A76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
A75Great quality (A) · 75 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
A74Great quality (A) · 74 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
A73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
A73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
A73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
B65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
B67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
B65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
A72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
A70Great quality (A) · 70 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
B65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
B64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
B66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
B63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
B62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
B60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
B60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
B58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
B56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
C52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
B60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
D32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 35/100 (quality, fit, speed and memory use) sets the ranking.
A+92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A+92Excellent quality (A+) · 92 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 124 tok/s
A+92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A+92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A+91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A+91Excellent quality (A+) · 91 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A+90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 208 tok/s
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A+90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 39.6 tok/s
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on NVIDIA PC 1, 146 tok/s
A+87Excellent quality (A+) · 87 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A+86Excellent quality (A+) · 86 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A+85Excellent quality (A+) · 85 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 155 tok/s
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on NVIDIA PC 1, 45.2 tok/s
A82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 161 tok/s
A80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 155 tok/s
A78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 121 tok/s
A78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 166 tok/s
A78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 166 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 41.9 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 161 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 166 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 166 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 83.9 tok/s
A74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 166 tok/s
A74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 166 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
A74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on NVIDIA PC 1, 170 tok/s
A71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 217 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 40.8 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 40.3 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 189 tok/s
B69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on NVIDIA PC 1, 189 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 40.3 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 141 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 31.2 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on NVIDIA PC 1, 141 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 44.2 tok/s
A75Great quality (A) · 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 44.2 tok/s
A74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on NVIDIA PC 1, 48.0 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 41.7 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 94.0 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 10.2 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on NVIDIA PC 1, 141 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 173 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 41.7 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 136 tok/s
B67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 5.0 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 13.6 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on NVIDIA PC 1, 174 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs great on NVIDIA PC 1, 47.8 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 18.6 tok/s
C53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 18.6 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 16.5 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 106 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 3.2 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 3.2 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 4.1 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 5.0 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 4.8 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 111 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 106 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 65.6 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 3.2 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 3.2 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 3.2 tok/s
C51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 16.4 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 3.2 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 4.8 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 3.0 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 3.2 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Runs well on NVIDIA PC 1, 16.4 tok/s
B61Good quality (B) · 61 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Faster on the clusterAlone: Runs great on NVIDIA PC 1, 69.0 tok/s
C53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 5.0 tok/s
C53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 4.8 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 5.6 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 3.2 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 15.1 tok/s
A78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 12.8 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 15.1 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 3.2 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on NVIDIA PC 1, 1.6 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 18.7 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 3.0 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 28.0 tok/s
B67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 25.0 tok/s
A73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 12.8 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on NVIDIA PC 1, 1.6 tok/s
B69Good quality (B) · 69 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 13.3 tok/s
B67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 13.3 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 6.6 tok/s
B65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 13.1 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 7.0 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 25.1 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 28.2 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 6.6 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 9.4 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on NVIDIA PC 1, 1.5 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on NVIDIA PC 1, 1.7 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on NVIDIA PC 1, 1.5 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on NVIDIA PC 1, 1.6 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on NVIDIA PC 1, 1.2 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on NVIDIA PC 1, 1.2 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Slow on NVIDIA PC 1, 5.1 tok/s
C43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
Better on the clusterAlone: Very slow on NVIDIA PC 1, 1.4 tok/s
A+90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 44.5 tok/s
A+89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 44.5 tok/s
A+88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 44.5 tok/s
A+90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 37.4 tok/s
A+87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 41.6 tok/s
A83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.2 tok/s
A83Great quality (A) · 83 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 52.7 tok/s
A81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.2 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 36.0 tok/s
A73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
A80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.
IOK2 Horizon 7BInstitute of Foundation Models · 9.0B
One device is enoughAlone: Runs great on NVIDIA PC 1, 70.0 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.4 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 43.0 tok/s
A74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 37.8 tok/s
A78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 66.9 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 64.5 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 43.0 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 41.7 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 42.3 tok/s
A77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 65.3 tok/s
B69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 38.4 tok/s
B69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 187 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 37.8 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 37.8 tok/s
A72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 37.7 tok/s
A76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 65.3 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.2 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 129 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 64.5 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
A75Great quality (A) · 75 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 65.3 tok/s
B67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 43.4 tok/s
B67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 43.4 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 38.7 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 38.0 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 38.0 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.8 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 164 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 42.9 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.1 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.2 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
B65Good quality (B) · 65 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 43.4 tok/s
B65Good quality (B) · 65 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 38.7 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 38.4 tok/s
B65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 41.1 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 37.7 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 44.5 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 43.4 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 41.7 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.2 tok/s
B64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.9 tok/s
A74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 70.4 tok/s
B63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 46.0 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 37.8 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 37.8 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 37.7 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.2 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 66.9 tok/s
A70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 66.9 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 43.4 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.9 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 71.7 tok/s
B63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 37.7 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.9 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.1 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 117 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 113 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.6 tok/s
A78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 65.2 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 51.6 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 37.8 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 69.6 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.5 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 43.4 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 46.2 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 36.8 tok/s
B68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 72.9 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 141 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.9 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 38.3 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 71.8 tok/s
B66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 70.6 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 126 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 61.2 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 73.8 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 38.4 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 38.3 tok/s
B57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 37.3 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 76.9 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 61.2 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 71.7 tok/s
B62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 62.3 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 72.4 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.9 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 131 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 105 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 166 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 51.6 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 76.9 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 70.9 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 67.0 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 63.9 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 117 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 50.8 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 75.8 tok/s
C49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 43.2 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 76.4 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 61.2 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 70.6 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 61.1 tok/s
B58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 63.9 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 76.4 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 80.4 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 80.6 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 72.2 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 65.4 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 72.6 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 117 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.1 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.1 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 79.3 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 68.2 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 67.0 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 83.0 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 66.9 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 72.2 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 86.3 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 128 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.9 tok/s
B55Good quality (B) · 55 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 82.7 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 76.9 tok/s
C47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 36.0 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 61.2 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 72.7 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 82.7 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 76.0 tok/s
B60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 115 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 58.7 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 84.0 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 54.8 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 43.2 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 66.4 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 71.6 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.0 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 82.7 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 52.1 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 67.0 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 76.0 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 72.2 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 86.3 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 84.6 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 131 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 39.5 tok/s
D39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 43.2 tok/s
C52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 79.3 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 59.0 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 70.8 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 82.7 tok/s
C41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 38.1 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 60.1 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 67.0 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 83.0 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 63.9 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 105 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.7 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 77.1 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 77.1 tok/s
C48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 77.8 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 79.3 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 73.1 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.5 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 48.0 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 46.0 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 58.7 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 58.7 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 80.6 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 52.4 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 64.4 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.8 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.8 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 77.1 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 42.8 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 84.4 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 75.6 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.0 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.8 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 82.7 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 74.9 tok/s
C45Fair quality (C) · 45 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 67.5 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 36.7 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 76.0 tok/s
B56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 168 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 36.0 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 86.8 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.7 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.8 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.8 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.8 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 76.0 tok/s
D33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 60.6 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 85.2 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 67.5 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 73.0 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 75.6 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 70.0 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 68.8 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 75.4 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 82.7 tok/s
C46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 85.4 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs well on NVIDIA PC 1, 36.7 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 71.3 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 86.8 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.8 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.8 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 73.8 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 39.5 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 62.8 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.2 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 77.1 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 81.4 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 71.3 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 145 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.8 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 86.8 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 155 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 76.0 tok/s
D29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 65.9 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 70.0 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
C50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 151 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 62.0 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 90.7 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 93.0 tok/s
D37Basic quality (D) · 37 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 62.3 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.8 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 48.4 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 51.9 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 86.3 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 71.3 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 80.6 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 117 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 94.3 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 71.3 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 85.4 tok/s
C44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 108 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 76.0 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 45.3 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 121 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 48.4 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 113 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.8 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 80.8 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 39.8 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 73.8 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 155 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 95.8 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 85.6 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 86.6 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.4 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 47.9 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 52.4 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 82.7 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 86.8 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 124 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 81.6 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 73.8 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 78.2 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 81.6 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 81.4 tok/s
D38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 91.0 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 86.4 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 47.2 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 86.8 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 48.4 tok/s
D35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 90.4 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 77.4 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 92.2 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 128 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 93.9 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 86.8 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 48.4 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 154 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 204 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 93.9 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 223 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 90.4 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 84.6 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 83.2 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 136 tok/s
C40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 187 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 86.6 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 47.9 tok/s
C42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 302 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 82.8 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 89.9 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 113 tok/s
D34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 65.4 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 84.9 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 86.8 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 115 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 115 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 120 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 83.9 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 69.3 tok/s
D36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 269 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 48.6 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 143 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 83.4 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 115 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 111 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 97.4 tok/s
D27Basic quality (D) · 27 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 119 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 110 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 104 tok/s
D32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 248 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 129 tok/s
D30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 227 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 2, 81.0 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 140 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 129 tok/s
D28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 199 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 109 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 163 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 131 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 189 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 187 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 140 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 95.8 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 248 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 189 tok/s
D24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 184 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 86.1 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 104 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 162 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 170 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 163 tok/s
D26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 334 tok/s
D21Basic quality (D) · 21 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 151 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 104 tok/s
D22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 234 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 154 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 189 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 196 tok/s
D17Basic quality (D) · 17 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 154 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 116 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 174 tok/s
D20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 290 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 170 tok/s
D16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 189 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 116 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 170 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 196 tok/s
D18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 392 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 215 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 215 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 387 tok/s
D10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 177 tok/s
D10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 177 tok/s
D15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 421 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 558 tok/s
D14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 829 tok/s
D11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 387 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 645 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 829 tok/s
D12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 854 tok/s
D10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 569 tok/s
D10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 592 tok/s
D8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 538 tok/s
D10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 1,262 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 264 tok/s
D8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 592 tok/s
D8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 806 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 592 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 806 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 1,117 tok/s
D6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 1,075 tok/s
D5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 806 tok/s
D4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 2,233 tok/s
D3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking.
One device is enoughAlone: Runs great on NVIDIA PC 1, 3,226 tok/s
D2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking.
Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.
NVIDIA GeForce RTX 4090 
NVIDIA GeForce RTX 3090 
