Quadro RTX 6000
GPU
24
GB
PERFORMANCE OVERVIEW
Model
EXAONE-Deep-2.4B
Q4_K - Medium2.7B
Gemma 3 4b It Abliterated
Q4_04.6B
Gemma 3 12b
Q4_012.8B
Prompt Speed
4158tokens/s
4079tokens/s
1380tokens/s
Generation Speed
105tokens/s
81.3tokens/s
39.9tokens/s
Time to First Token
317ms
302ms
904ms
LocalScore
1111
1032
393
COMPARE MODELS
4 models tested
Select Models
Gemma 3 12b
Q4_0
Gemma 3 4b It Abliterated
Q4_0
EXAONE-Deep-2.4B
Q4_K - Medium
Gemma 3 27b
Q4_0