Explosive Preprint about the AI "Leaderboard Illusion"

TL;DR: leaderboard rankings are unreliable

This picture is worth a t̶h̶o̶u̶s̶a̶n̶d̶ billion words 😱. Based on an preprint on the ArXiv server (The Leaderboard Illusion) by a team led by Shivalika Singh, Marzieh Fadaee, and Sara Hooker from Cohere and colleagues from Princeton University, Stanford University, University of Waterloo, Massachusetts Institute of Technology, Allen Institute and University of Washington (Yiyang Nan, Alex Wang, Daniel D'souza, Sayash Kapoor, Ahmet Üstün, Sanmi Koyejo, Yuntian Deng, Shayne Longpre, Noah Smith, Beyza Ermis), there are reasons to question the performance of some of the most well-known AI models, particularly proprietary and open-weights models.

Their secret? Teaching to the test, literally, and then withdrawing the models that don't perform well, thereby biasing the leaderboards and pumping hype into the "so-close-to-AGI" craze. 🤯

The figure shows the "number of public models vs. maximum arena score per provider, while marker size indicates the total number of battles played. Proprietary model providers tend to achieve higher leaderboard scores, which appear to correlate with both the number of models they release and the number of Arena battles played. Increased exposure to the Arena (through more models and battles) may confer additional advantages, such as better model selection or adaptation to the evaluation distribution. This figure summarizes publicly disclosed results as of April 23rd, 2025."

The worst part of this practice is that it happens without disclosure: private testing, selective score reporting and data access disparities in complete opacity 💢 .

I am curious to see the reactions of the alleged culprits. Does not smell good👃.