logo
HomeLatest ResultsDownloadAboutBlog
model icon

Llama 3.2 1B Instruct

Q6_K

1.5Bparams
COMPARE ACCELERATORS

7 accelerators tested

Select Accelerators

NVIDIA GeForce RTX 3090

24GB

NVIDIA RTX A5000

24GB

AMD Radeon RX 6900 XT

16GB

AMD Radeon RX 7800 XT

16GB

AMD Ryzen 9 5950X 16-Core Processor (znver3)

125.7GB

Llama 3.2 1B Instruct - Q6_K

LEADERBOARD
NVIDIA GeForce RTX 3090
GPU / 24GB
PROMPT
13357
tokens/s
GENERATION
300
tokens/s
TTFT
100
ms
LOCALSCORE
3428
NVIDIA RTX A5000
GPU / 24GB
PROMPT
11451
tokens/s
GENERATION
273
tokens/s
TTFT
117
ms
LOCALSCORE
2992
AMD Radeon RX 6900 XT
GPU / 16GB
PROMPT
5147
tokens/s
GENERATION
109
tokens/s
TTFT
298
ms
LOCALSCORE
1234
AMD Radeon RX 7800 XT
GPU / 16GB
PROMPT
3009
tokens/s
GENERATION
112
tokens/s
TTFT
446
ms
LOCALSCORE
911