Meta Llama 3.1 8B Instruct - Q5_K

Meta Llama 3.1 8B Instruct

Q5_K - Medium

8.0Bparams

COMPARE ACCELERATORS

3 accelerators tested

Select Accelerators

AMD Radeon RX 6900 XT

16GB

AMD Ryzen 9 5950X 16-Core Processor (znver3)

125.7GB

AMD Ryzen 9 7940HS w/ Radeon 780M Graphics (znver4)

54.7GB

Meta Llama 3.1 8B Instruct - Q5_K - Medium

LEADERBOARD

ACCELERATOR

PROMPT

GENERATION

TTFT

LOCALSCORE

GPU / 16GB

PROMPT

1157

tokens/s

GENERATION

34.9

tokens/s

TTFT

1.27

sec

LOCALSCORE

317