Stop the "it's just next-token prediction" nonsense

I have heard or read this so many times it is infuriating: LLMs are not smart because they just do "next-token prediction". Whether or not LLMs are smart or can reason or have feelings (they don't 😵‍💫), they are much more than Next-Token Prediction.

NTP is a convenient concept to explain LLMs to people who don't know much about AI. Or you can go into more detail and talk about masked prediction training. Or you can go into more detail.... and it's 🐢 all the way down. This convenient simplification becomes an issue when it is used as a straw man for attacking the deficiencies of LLMs or genAI in general. LLMs have lots of issues, but they are a lot more sophisticated than that, starting with the very architecture of the original transformers, which look for long-range linkages (for ease of training but that's beside the point).

That's where I think the Anthropic study I referred to yesterday could serve as a conversation reset for the rational-minded. I thought it was interesting that the interpretability team treated Claude 3.5 haiku (a lightweight version of their flagship 3.7 Sonnet) as a biological system, framing the model as a complex (adaptive?) system. But then my favorite newsletter from AlphaSignal hit my mailbox: "Anthopic reveals how Claude 3.5 Haiku plans words ahead, challenging next-token assumptions". That's probably the most important impact of that set of studies (link to a nice explainer in the comments), making it clear that there is a lot going on in LLMs. It does look like an alien form of intelligence and the probes the Anthropic team created are clever and revealing but perhaps there are other probes that could reveal other "metabolic pathways" unknown to mankind.

And again, kudos to the Anthropic team (doesn't mean I agree with Dario Amodei on much but great work!): Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce

Nicholas Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Tom Henighan, Adam Jermyn, Andy J., Andrew Persic, Zhenyi Qi, Ben Thompson, Sam Zimmerman, Kelley Rivoire, Tom Conerly, Christopher Olah, Joshua Batson

Thomas Wolf, Dr. Ayesha Khanna, Cassie Kozyrkov

Ksenia Se