YandexGPT 5 Lite 8B Instruct
Q4_K - Medium
8.0Bparams
COMPARE ACCELERATORS
1 accelerators tested
Select Accelerators
NVIDIA RTX 2000 Ada Generation
16GB
YandexGPT 5 Lite 8B Instruct - Q4_K - Medium
LEADERBOARD
GPU / 16GB
PROMPT
1271
tokens/s
GENERATION
35.8
tokens/s
TTFT
1.07
sec
LOCALSCORE
349