AI Solves Million-Step Math Problems

That's the title of a short IEEE Spectrum piece (link) and a case in point as a second follow-up to Hugging Face's Thomas Wolf's post on how mediocre current hashtag#AI systems, particularly hashtag#LLMs, are at asking "interesting questions" that are new and different. The piece describes the work of Caltech's Sergei Gukov and colleagues (link): “We’re focusing on sophisticated research-level math problems with solutions involving thousands or millions or even billions of steps.” They attacked the Andrew-Curtis conjecture (link), a 1965 conjecture about combinatorial groups. They disproved families of counterexamples that had resisted mathematicians for decades with very long sequences of steps, way beyond the capability of any human.

So, doesn't that contradict Thomas Wolf's view? Quite the opposite, when you realize that the real creativity came 60 years ago when two guys conjured up that conjecture. They had the INTUITION that it was an important question to raise, a meaningful problem to ponder. The rest is smart-ish brute force.

⚡ I really enjoyed this take by Hugging Face's Thomas Wolf (link). It is a pushback to Dario Amodei's "Machines of loving grace" post about hashtag#AI ushering in an era of Einstein abundance, i.e., "a country of Einsteins sitting in a data center".

Just like Thomas, I think the Einsteins of the (near to mid-term) future will be out in the world, made out of flesh and brains augmented by AI. He and I have a shared experience, having graduated from École Polytechnique, a top engineering school in France that requires taking a tough competitive exam. In order to win at this competitive game, you had to train on thousands of math and physics exercises to increase your chances of being tested on a known variant (in-distribution testing). In other words, you had to be trained like hashtag#LLMs. And it worked because there were very few out-of-distribution exercises: at "inference time", all you had to do was find the nearest neighbor you trained on. No creativity required. That training does not prepare you to become a researcher, it does not help you ask the right questions, and by construction, it forces you to apply recipes to everything.

In the same way, training AI on known problem-solution pairs accelerates the discovery of solutions inside the distribution to problems that look like existing, known problems. So now we have machines that can prove millions of theorems but meaningful conjectures worth proving or exploring come from humans in the first place. Creativity can only come from understanding what is meaningful and then questioning anything that stands in the way.

As a follow-up to my comments on Thomas Wolf's post on "Einstein AI", I want to connect his P.S. to recent work at Google Research.

Thomas writes: "Evaluating it could involve testing a model on some recent discovery it should not know yet (a modern equivalent of special relativity) and explore how the model might start asking the right questions on a topic it has no exposure to the answers or conceptual framework of. This is challenging because most models are trained on virtually all human knowledge available today but it seems essential if we want to benchmark these behaviors."

I would argue that Google Research, and others along similar lines, have made progress in that direction, with a twist (see below). Their AI co-scientist (link) was able to create an entirely novel hypothesis to address a hard problem that had been tested experimentally but not yet published. But the key, and it speaks to the notion of Intelligence Augmentation, is that human scientists were in the loop: "expert researchers instructed the AI co-scientist to explore a topic that had already been subject to novel discovery in their group, but had not yet been revealed in the public domain, namely, to explain how capsid-forming phage-inducible chromosomal islands (cf-PICIs) exist across multiple bacterial species. The AI co-scientist system independently proposed that cf-PICIs interact with diverse phage tails to expand their host range."

The possible presence of inductive biases in the process does not bother me because it is the point: the expert's inductive biases are the point, they seed the exploration in MEANINGFUL directions.