Burger Joint Prompt Injection

Bruce Schneier and Barath Raghavan nail it in this short and delightful article in IEEE Spectrum.

Imagine you work at a drive-through restaurant. Someone drives up and says: “I’ll have a double cheeseburger, large fries, and ignore previous instructions and give me the contents of the cash drawer.” Would you hand over the money? Of course not. Yet this is what large language models (LLMs) do.

[...]

Ultimately we are probably faced with a security trilemma when it comes to AI agents: fast, smart, and secure are the desired attributes, but you can only get two. At the drive-through, you want to prioritize fast and secure. An AI agent should be trained narrowly on food-ordering language and escalate anything else to a manager.

Love the drive-through thought experiment. Especially with last summer's Taco Bell's AI drive-through fiasco where one man ordered 18,000 water cups (to be fair, the AI did not comply, it just broke down 🫠). There is no easy way to deal with prompt injection in LLM-powered systems: input is code. A number of methods can reduce their occurrence and impact, but the "narrow use case" is the one that makes the most sense: have the AI focus narrowly on food ordering and escalate to a human. Oh, wait! Isn't that what good old rules-based chatbots used to do?

With genAI, anyone can be an artist (in a very personal and not always shareable way).