logo
HomeLatest ResultsDownloadAboutBlog
model icon

Granite 3.2 8b Instruct

Q6_K

8.4Bparams
COMPARE ACCELERATORS

1 accelerators tested

Select Accelerators

NVIDIA GeForce RTX 3090

24GB

Granite 3.2 8b Instruct - Q6_K

LEADERBOARD
NVIDIA GeForce RTX 3090
GPU / 24GB
PROMPT
2744
tokens/s
GENERATION
73.7
tokens/s
TTFT
465
ms
LOCALSCORE
758