Exact Instructions

This 8-years old video by Josh Gaines (aka Josh Darnit) has re-emerged recently —for good reason! (Notice the super intentional emdash). The Exact Instructions Challenge is a great reminder that "English is the new programming language" is to be taken with a large grain of salt. Actually, just throw in the whole salt shaker.

I wanted to add a couple of comments, but first link to the original video, which seems to have been omitted after layers and layers of copying and reposting: link His channel has a lot of great stuff, including a collection of Exact Instructions Challenges.

So what is that video really telling us, other than his kids hate him?

First, instructions that are meant to be followed very precisely need to be written very precisely. Doh!

The second point is that there is a lot of commonsense knowledge that comes “pre-packaged”, if you will, in any human activity and therefore does not need to be explicitly included or repeated in an instruction’s manual. The video exploits this by falsely assuming the absence of commonsense knowledge.

But then it dawns on you: how do you know that the person or machine following the instructions has the same commonsense knowledge base as you? When we communicate with people around us, there is a near-complete overlap in commonsense knowledge. Not so when we interact with humans from a completely different culture. And then when we interact with a machine, well, what do we know??

So, yes, being very explicit about steps to follow is a safety precaution when dealing with LLMs. We are discovering every day that their commonsense knowledge is a 7,897,758,981,231-dimensional fractal object with some regions of overlap that give an illusion of shared “humanity.”

I would, however, add a nuance here: “vibe coding” or vibe anything is not giving the machine step-by-step instructions but giving the machine a description of an objective, what you are trying to achieve or build. You may want to be precise in describing your objective (or not, if you are just exploring), but remember that the space of “rules” the machine is following to reach your exquisitely defined objective is opaque, “reasoning traces” notwithstanding. So you may want to define as well what is allowed and what is not in meeting the target. But then it requires knowing what the space of rules might entail (you mostly don’t) and also, it becomes a lot like programming.