The verification bottleneck

Anastasia Borovykh, you are right, humans are good at "taste", knowing what is interesting or not, distinguishing what's meaningful from what is just truth without meaning. Human taste is found at the two end of the process, though: not just in the evaluation or verification phase where AI can appear random, but also, perhaps more importantly, in the questions posed to the AI. Among the trillions of possible questions, a few are selected by human minds. Verification may be a bottleneck, but so is human input. I think Google's AI co-scientist highlights this balance (spectrum.ieee.org/ai-co-scientist): aiming the AI at well-posed (human) questions can generate novel + meaningful hypotheses.

Anastasia, to clarify, by "questions posed to AI" I am not talking about optimizing prompts but about setting the problem you want the AI to work on. In your fantastic YouTube video, you mention AI for math: of the trillions of conjectures it could try prove, only a human with great mathematical taste (say, Terence Tao) can come up with meaningful conjectures, theorems worth proving, the ones that can advance science and human progress. So the combination of (1) the ability/vision to set the overall goal (e.g., put humans on Mars) and (2) the ability to evaluate/verify that AI-proposed solutions make sense or could make sense (e.g., unlikely to get to Mars on a hot air balloon; but a gravitational slingshot could work) is what makes human "taste" so valuable.