NVIDIA GeForce RTX 2060
GPU
12
GB
PERFORMANCE OVERVIEW
Model
Qwen2.5 14B Instruct
Q4_K - Medium14.8B
Mistral Small 22B ArliAI RPMax v1.1
Q5_K - Medium22.2B
Prompt Speed
595tokens/s
110tokens/s
Generation Speed
22.8tokens/s
2.0tokens/s
Time to First Token
2.16sec
16.03sec
LocalScore
184
24
COMPARE MODELS
2 models tested
Select Models
Qwen2.5 14B Instruct
Q4_K - Medium
Mistral Small 22B ArliAI RPMax v1.1
Q5_K - Medium