The "G" in hashtagAGI is sending everyone down a deep and jagged rabbit hole

❤️ I love love love Jim Fan's commentary on OpenAI's o3 latest FrontierMath benchmark results, in some ways a super-intelligence tour de force. I like the concept of "single-point [RL] super-intelligence" the accompanying examples (e.g., AlphaGo, AlphaStar, e-Atlas performing backflips) and o3 is a multi-point (or narrow domain) super-intelligence.

💡 One key observation, probably obvious to many, is that o3 performs extraordinarily well on tasks that OpenAI prioritizes.

📉 And you know what they say about "teaching to the test" (some version of Goodhart's law), it helps you get good grades but does not transfer to anything else. To be fair, acing the test in question does transfer to real questions... for mathematicians. Like, really, really good mathematicians. But as Jim says, "o3 can wow the Fields Medalists, but still fail to solve some 5-yr-old puzzles".

🪠 And that's the thing, o3 exhibits Ethan Mollick's jaggedness: it can be exceedingly good at one thing and exceedingly idiotic at something that's not too dissimilar. In fact, even its performance on FrontierMath is no guarantee of smooth performance across the whole domain.

✂️ Where I differ with Jim Fan is in the conclusion: "clear roadmap". I think the roadmap is not clear at all. Comparing o3 or other competing models to human intelligence is misleading and distracting. The "G" in hashtag

hashtag#AGI is sending everyone down a deep and jagged rabbit hole. What we are witnessing the emergence of is SSI: Spiky Super Intelligence, or

Artificial Idiot Savants, let's embrace it. It would help define the proper use cases.