logo
HomeLatest ResultsDownloadAboutBlog
model icon

Llama 3.2 3B Instruct

Q4_K - Medium

3.6Bparams
COMPARE ACCELERATORS

11 accelerators tested

Select Accelerators

NVIDIA GeForce RTX 5060 Ti

16GB

NVIDIA GeForce RTX 4050 Laptop GPU

6GB

AMD Radeon RX 6900 XT

16GB

NVIDIA GeForce GTX 1060 6GB

6GB

AMD Ryzen 9 5950X 16-Core Processor (znver3)

125.7GB

Llama 3.2 3B Instruct - Q4_K - Medium

LEADERBOARD
NVIDIA GeForce RTX 5060 Ti
GPU / 16GB
PROMPT
2980
tokens/s
GENERATION
67.8
tokens/s
TTFT
426
ms
LOCALSCORE
780
NVIDIA GeForce RTX 4050 Laptop GPU
GPU / 6GB
PROMPT
2518
tokens/s
GENERATION
60.0
tokens/s
TTFT
568
ms
LOCALSCORE
643
AMD Radeon RX 6900 XT
GPU / 16GB
PROMPT
2159
tokens/s
GENERATION
55.3
tokens/s
TTFT
705
ms
LOCALSCORE
554
NVIDIA GeForce GTX 1060 6GB
GPU / 6GB
PROMPT
723
tokens/s
GENERATION
34.0
tokens/s
TTFT
1.83
sec
LOCALSCORE
238
NVIDIA GeForce GTX 1660 SUPER
GPU / 6GB
PROMPT
293
tokens/s
GENERATION
14.4
tokens/s
TTFT
4.99
sec
LOCALSCORE
95
AMD Radeon 780M
GPU / 27GB
PROMPT
288
tokens/s
GENERATION
6.4
tokens/s
TTFT
4.58
sec
LOCALSCORE
74