Verifiable rewards · the one-way function put to work
There is a hidden code. You cannot see it. All you can do is submit a guess and get back a score. Finding the code is the hard direction; scoring a guess is the easy one. This is exactly the gap that reinforcement learning from verifiable rewards exploits: if you can cheaply check an answer, you can learn to produce it, even when producing it is hard.
Four colors, chosen from six. You solve it purely from the verifier's feedback.
● filled peg = right color, right spot · ○ hollow = right color, wrong spot
Watch an agent solve the same code using nothing but the verifier. The reward signal's density decides whether it can learn at all.
Why this is the whole trick. The verifier here is trivial to run: it counts matching pegs. It never reveals the code. Yet that single cheap signal is enough to drive a search all the way to the answer. That is what "if it is verifiable, it is optimizable" means: a generator proposes candidates, a verifier scores them, and the scores are gradually compiled into the generator so it produces passing answers more often. Switch to the sparse reward and the same agent stalls, because a yes/no signal carries almost no gradient to climb. The lesson is not that hard problems become easy. It is that a well-shaped verifiable reward turns "I cannot solve this" into "I can check this, so I can learn it."