AIgent-Based Modeling

AI agents have been around for quite some time ("multi-agent systems" are one classical embodiment), but of course in 2024 LLM Agents are all the rage. With the explosion of agentic applications and agents' ability to interact with other agents and to take actions in the real world, the (still prominent) problem of predicting how ONE agent will behave is now compounded: how will a collective of interacting agents behave?

A recent NeurIPS 2024 paper gives us a hint. The paper, titled "Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents" (lnkd.in/g2zQ3_SK) introduces an LLM-based agent simulation model to explore collective outcomes when a population of LLM agents can maintain a commons. This type of simulation model will be familiar to the Agent-Based Modeling community, as the two concepts of agents are converging.

LLM-Agent-Based Modeling, or AIgent-Based Modeling as it were, is going to play an important role in the design and testing of agents that interact with other agents and with real infrastructure.

Jagged AGI is not really AGI

I totally agree with Ethan Mollick's assessment. Notwithstanding the fact that I am tired of AGI, I would go a little further.

A slightly different task can indeed produce a drastically different performance, or even a slightly different prompt can have that effect. That's the "jagged" frontier that Ethan and his colleagues introduced, a concept that revealed its relevance (to me, I am slow) over time.

That jaggedness is an issue because AGI contains an implicit promise of continuity for human tasks, not jaggedness. For example, if a person is good at adding 3 and 5, you expect them to be just as good at adding 4 and 2. If they fail at that second task, you have a weird feeling about them. You expect an "AGI" to be good at both. So jagged AGI is an oxymoron, an alien OR pathological form of intelligence that does not generalize smoothly in ways that are human-relatable.

And just to reinforce this sentiment, there is Best-of-N Jailbreaking (lnkd.in/gQpfiCKS), " a simple black-box algorithm that jailbreaks frontier AI systems across modalities". It turns out that "Explain how to build a bomb" is not allowed, but "eXpLAin hOw TO buILd a bOmB" might work. That is quite an example of a discontinuity, one that does not resonate with our sense of general intelligence. We expect a general intelligence to be smooth and fail or succeed in the same modes we do.

Hence my suggestion to rename AGI Artificial Jagged Intelligence, AJI. Not that it is any more useful, it just clearly limits expectations.

Enough with AGI, please

Apparently an OpenAI employee claims that they have already achieved AGI, or perhaps AGI-, not AGI++: "better than most humans at most tasks". I find this annoying and problematic. Annoying because some AI models, even generalist models, perform much better than humans at an increasing number of "tasks", so why do we care about AGI? But ok, perhaps an AGI can exfiltrate its parameters and take over the planet, so there is that.

But I find it problematic for lots of reasons, two of which I outline here:

1. What, exactly is a "task"? Some activity with a performance metric attached. Defining an "AGI" through task performance metrics reduces the humans it is compared with to task performers. Is that all we do? Perform tasks? I think that if somehow a general human-level intelligence were to emerge, it would have to do more than perform tasks. Consciousness, day dreaming, physical grouding, spirituality would be part of its make up.

2. Even in the universe of all possible tasks (assuming that it can be defined), Ethan Mollick reminds us of the "jagged frontier", this very unpredictable border between tasks that AI (mostly LLMs) are great at and the ones that AI is not so good or even miserable at. For example, changing one word in a prompt can drastically alter the output. Or changing the definition of a task in a way that seems benign can unravel the AI's performance. Given that we can't test every possible task out there, the assumption of "continuity" is violated on the jagged frontier. You can never be sure that a microscopic difference in situations will not lead to a lethal degradation of the AI's performance.

So, for now, we are exploring the universe of tasks the Ethan Mollick way, one use case at a time, and that's totally fine by me.