Evolving Surprises

A recent article about "hashtag#AI Deception" ("AI deception: A survey of examples, risks, and potential solutions": link) reviews some AI systems' "ability to deceive via techniques such as manipulation, sycophancy, and cheating the safety test." There are a number of underlying causes, generally under the rubric of "goal-directed agents will do whatever is required to achieve their goal" (for example the canonical paperclips producer agent: link). Deception is one of the many strategies an "unethical" goal-directed agent could acquire. The overarching goal of "pleasing my human" (see e.g. caution advisory from Ethan Mollick) may lead to sycophancy and agents that never "question the question", lie, make stuff up, etc.

But it may also be more subtle and more difficult to detect: an AI agent may be creative without intentionally deceiving to make its job easier, using resources and techniques that it is not explicitly prohibited from using but that, once (or if) it is known that they are being used, will be added to the taboo list of forbidden resources. In this arms race of sorts, new laws and regulations need to be added to deal with the discovery of unethical techniques and loopholes. It is not too dissimilar from the evolution of a legal system, perhaps in this case it could be optimized.

The reason why this came to mind is years of working with evolutionary algorithms, which are basically optimization "agents" equipped with a set of tools and given an objective (or fitness) function: such algorithms can be very creative in finding shortcuts, especially when they are embedded in the physical world. The paper you can find here is a wonderful list of such evolutionary surprises. One of my favorites is Adrian Thompson's experiment, where he "evolved the connectivity of a reconfigurable field-programmable gate area (FPGA) chip, with the aim of producing circuits that could distinguish between a high-frequency and a lower-frequency square-wave signal. After 5,000 generations of evolution, a perfect solution was found that could discriminate between the waveforms. This was a hoped-for result, and not truly surprising in itself. However, upon investigation, the evolved circuits turned out to be extremely unconventional. The circuit had evolved to work only in the specific

temperature conditions in the lab, and exploited manufacturing peculiarities of the particular FPGA chip used for evolution. Furthermore, when attempting to analyze the solution, Thompson disabled all circuit elements that were not part of the main powered circuit, assuming that disconnected elements would have no effect on behavior. However, he discovered that performance degraded

after such pruning. Evolution had learned to leverage some type of subtle electromagnetic coupling, something a human designer would not have considered (or likely have known how to leverage)."