Qwen2.5 14B Instruct
Q4_K - Medium
14.8Bparams
COMPARE ACCELERATORS
111 accelerators tested
Select Accelerators
NVIDIA RTX 6000 Ada Generation
47GB
NVIDIA GeForce RTX 5090
31GB
NVIDIA H100 PCIe
79GB
NVIDIA GeForce RTX 4090
24GB
NVIDIA GeForce RTX 4080
16GB
Qwen2.5 14B Instruct - Q4_K - Medium
LEADERBOARD
GPU / 47GB
PROMPT
3552
tokens/s
GENERATION
70.0
tokens/s
TTFT
369
ms
LOCALSCORE
877
GPU / 16GB
PROMPT
2454
tokens/s
GENERATION
50.4
tokens/s
TTFT
535
ms
LOCALSCORE
614
GPU / 48GB
PROMPT
2936
tokens/s
GENERATION
28.1
tokens/s
TTFT
471
ms
LOCALSCORE
560
GPU / 16GB
PROMPT
2424
tokens/s
GENERATION
29.9
tokens/s
TTFT
555
ms
LOCALSCORE
507
GPU / 12GB
PROMPT
1701
tokens/s
GENERATION
24.9
tokens/s
TTFT
786
ms
LOCALSCORE
377
GPU / 20GB
PROMPT
1408
tokens/s
GENERATION
31.4
tokens/s
TTFT
952
ms
LOCALSCORE
359
PROMPT
1337
tokens/s
GENERATION
32.8
tokens/s
TTFT
968
ms
LOCALSCORE
356
GPU / 11GB
PROMPT
1168
tokens/s
GENERATION
39.7
tokens/s
TTFT
1.08
sec
LOCALSCORE
351
GPU / 16GB
PROMPT
1246
tokens/s
GENERATION
26.3
tokens/s
TTFT
1.10
sec
LOCALSCORE
310
GPU / 20GB
PROMPT
1037
tokens/s
GENERATION
24.9
tokens/s
TTFT
1.34
sec
LOCALSCORE
268
GPU / 512GB
PROMPT
579
tokens/s
GENERATION
35.9
tokens/s
TTFT
2.03
sec
LOCALSCORE
217
GPU / 16GB
PROMPT
740
tokens/s
GENERATION
20.7
tokens/s
TTFT
1.81
sec
LOCALSCORE
204
GPU / 96GB
PROMPT
445
tokens/s
GENERATION
34.4
tokens/s
TTFT
2.67
sec
LOCALSCORE
179
GPU / 128GB
PROMPT
290
tokens/s
GENERATION
27.8
tokens/s
TTFT
4.04
sec
LOCALSCORE
126