The fastest Sao10K model is Llama 3 8B Lunaris at 74 output tokens per second. Llama 3.1 Euryale 70B v2.2 (37 t/s) and Llama 3.3 Euryale 70B (9 t/s) round out the top three.
AI models ranked by output speed (tokens per second, p50). The fastest large language models for low-latency and high-throughput applications.
The fastest Sao10K model is Llama 3 8B Lunaris at 74 output tokens per second. Llama 3.1 Euryale 70B v2.2 (37 t/s) and Llama 3.3 Euryale 70B (9 t/s) round out the top three.
Llama 3.1 Euryale 70B v2.2 (37 t/s) is the closest alternative on this metric, followed by Llama 3.3 Euryale 70B (9 t/s). See the full ranking above for the tradeoffs.
modelgrep tracks 4 Sao10K models with live benchmarks, speed, latency and per-provider pricing. 4 of them qualify for this ranking.