Qwen2.5 14B Instruct
Q4_K - Medium
14.8Bparams
COMPARE ACCELERATORS
137 accelerators tested
Select Accelerators
NVIDIA RTX PRO 6000 Blackwell Workstation Edition
95GB
NVIDIA GeForce RTX 4090 D
47GB
NVIDIA RTX 6000 Ada Generation
47GB
NVIDIA GeForce RTX 5090
31GB
NVIDIA H100 PCIe
79GB
Qwen2.5 14B Instruct - Q4_K - Medium
LEADERBOARD
PROMPT
5126
tokens/s
GENERATION
81.1
tokens/s
TTFT
245
ms
LOCALSCORE
1193
GPU / 47GB
PROMPT
3552
tokens/s
GENERATION
70.0
tokens/s
TTFT
369
ms
LOCALSCORE
877
GPU / 16GB
PROMPT
2454
tokens/s
GENERATION
50.4
tokens/s
TTFT
535
ms
LOCALSCORE
614
GPU / 48GB
PROMPT
2936
tokens/s
GENERATION
28.1
tokens/s
TTFT
471
ms
LOCALSCORE
560
GPU / 16GB
PROMPT
2424
tokens/s
GENERATION
29.9
tokens/s
TTFT
555
ms
LOCALSCORE
507
GPU / 12GB
PROMPT
1701
tokens/s
GENERATION
24.9
tokens/s
TTFT
786
ms
LOCALSCORE
377
GPU / 20GB
PROMPT
1408
tokens/s
GENERATION
31.4
tokens/s
TTFT
952
ms
LOCALSCORE
359
PROMPT
1337
tokens/s
GENERATION
32.8
tokens/s
TTFT
968
ms
LOCALSCORE
356
GPU / 11GB
PROMPT
1168
tokens/s
GENERATION
39.7
tokens/s
TTFT
1.08
sec
LOCALSCORE
351
GPU / 16GB
PROMPT
1246
tokens/s
GENERATION
26.3
tokens/s
TTFT
1.10
sec
LOCALSCORE
310
GPU / 20GB
PROMPT
1037
tokens/s
GENERATION
24.9
tokens/s
TTFT
1.34
sec
LOCALSCORE
268
GPU / 16GB
PROMPT
821
tokens/s
GENERATION
28.4
tokens/s
TTFT
1.55
sec
LOCALSCORE
247
GPU / 16GB
PROMPT
786
tokens/s
GENERATION
27.9
tokens/s
TTFT
1.55
sec
LOCALSCORE
242
GPU / 512GB
PROMPT
574
tokens/s
GENERATION
35.9
tokens/s
TTFT
2.06
sec
LOCALSCORE
215
GPU / 16GB
PROMPT
740
tokens/s
GENERATION
20.7
tokens/s
TTFT
1.81
sec
LOCALSCORE
204
GPU / 96GB
PROMPT
445
tokens/s
GENERATION
34.4
tokens/s
TTFT
2.67
sec
LOCALSCORE
179
GPU / 128GB
PROMPT
290
tokens/s
GENERATION
27.8
tokens/s
TTFT
4.04
sec
LOCALSCORE
126