logo
HomeLatest ResultsDownloadAboutBlog
model icon

gemma-2-9b-it

Q4_K - Medium

10.2Bparams
COMPARE ACCELERATORS

3 accelerators tested

Select Accelerators

NVIDIA GeForce RTX 4060 Ti

16GB

NVIDIA GeForce RTX 3060

12GB

Apple M4 Max 10P+4E+32GPU

36GB

gemma-2-9b-it - Q4_K - Medium

LEADERBOARD
NVIDIA GeForce RTX 4060 Ti
GPU / 16GB
PROMPT
1820
tokens/s
GENERATION
32.4
tokens/s
TTFT
770
ms
LOCALSCORE
425
NVIDIA GeForce RTX 3060
GPU / 12GB
PROMPT
1217
tokens/s
GENERATION
36.4
tokens/s
TTFT
1.08
sec
LOCALSCORE
345
Apple M4 Max 10P+4E+32GPU
GPU / 36GB
PROMPT
418
tokens/s
GENERATION
36.3
tokens/s
TTFT
2.82
sec
LOCALSCORE
175