CPUs Aren't Dead. Gemma2B Out Scored GPT-3.5 Turbo on Test That Made It Famous

Gemma 2B achieved 8.2 on MT-Bench (vs GPT-3.5 Turbo's 7.94) through targeted software fixes addressing specific failure modes like arithmetic errors and logic inconsistencies. The work demonstrates that performance gaps often stem from software engineering rather than compute limits, enabling efficient CPU-based inference.