The verification bottleneck
Anastasia Borovykh, you are right, humans are good at "taste", knowing what is interesting or not, distinguishing what's meaningful from what is just truth without meaning. Human taste is found at the two end of the process, though: not just in the evaluation or verification phase where AI can appear random, but also, perhaps more importantly, in the questions posed to the AI. Among the trillions of possible questions, a few are selected by human minds. Verification may be a bottleneck, but so is human input. I think Google's AI co-scientist highlights this balance (link): aiming the AI at well-posed (human) questions can generate novel + meaningful hypotheses.
Anastasia, to clarify, by "questions posed to AI" I am not talking about optimizing prompts but about setting the problem you want the AI to work on. In your fantastic YouTube video, you mention AI for math: of the trillions of conjectures it could try prove, only a human with great mathematical taste (say, Terence Tao) can come up with meaningful conjectures, theorems worth proving, the ones that can advance science and human progress. So the combination of (1) the ability/vision to set the overall goal (e.g., put humans on Mars) and (2) the ability to evaluate/verify that AI-proposed solutions make sense or could make sense (e.g., unlikely to get to Mars on a hot air balloon; but a gravitational slingshot could work) is what makes human "taste" so valuable.
Periodic Labs
Abhijeet Gangan, Alexandre Passos, Costa Huang, Dogus Cubuk, Dzmitry Bahdanau, Elsa Cong, Eric Toberer, Gowoon Cheon, Janosh Riebesell, Joe Checkelsky, Kate Lauterbach, Killian Sheriff, Liam Fedus, Mansheej Paul, Matt Horton, Michael Zhang, Muratahan Aykol, Naveen Menon, Reiichiro Nakano, Rishabh Agarwal, Rohan Pandey, Sam Cross, Vincent Moens, Wei Chen, Xander Dunn, Xiang Fu
Thx @ethan mollick! "No LLM solved XYZ" is a risky title for a scientific article these days. But more seriously I think you are raising the most important question: how do we organize to maximize the potential of AI for science? In addition to "direct, confirm, disseminate findings", I would add "assign credit/attribution", as incentives (in some form or another) are at the heart of human activities.