Hallucinations part I
A great take on genAI hallucinations by Colin F. "Hallucinations", however we define the concept, are a core property (and perhaps the core value proposition) of LLMs and genAI systems.
My view: Trying to root them out is counterproductive. Factual errors are to be expected by the very nature of the underlying stochastic algorithms. But so is "creative license". Thus the issue is more about defining use cases that will benefit and not rushing the technology into use cases it is ill fitted to address. Unfortunately, we see a lot more of the latter.
Hallucinations part II
I should add to my earlier post about Colin F.'s blog on LLM and genAI hallucinations that (the always interesting) Andrej Karpathy is enthusiastically embracing the idea of hallucinations being a feature, not a bug, according to this X-tweet: lnkd.in/gRgvG2gW
I can't agree more. The issue is, to borrow Colin F.'s vernacular, that genAI is like a hammer and people see nails everywhere: there may a few actual nails out there, in the space of dreams and creativity, but most nail-looking use cases are not nails at all.
--
@karpathy
# On the "hallucination problem"
I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.
We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful.
It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug, but it's just the LLM doing what it always does.
At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar "training documents" it has in its database, verbatim. You could say that this search engine has a "creativity problem" - it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem.
All that said, I realize that what people *actually* mean is they don't want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. An LLM Assistant is a lot more complex system than just the LLM itself, even if one is at the heart of it. There are many ways to mitigate hallcuinations in these systems - using Retrieval Augmented Generation (RAG) to more strongly anchor the dreams in real data through in-context learning is maybe the most common one. Disagreements between multiple samples, reflection, verification chains. Decoding uncertainty from activations. Tool use. All an active and very interesting areas of research.
TLDR I know I'm being super pedantic but the LLM has no "hallucination problem". Hallucination is not a bug, it is LLM's greatest feature. The LLM Assistant has a hallucination problem, and we should fix it.
</rant> Okay I feel much better now :)
Hallucinations part III
I have found surprisingly few mentions of this so far, but I am sure (I hope) it will become obvious: the future belongs to critical thinkers and curators, humans who can detect the relevant patterns but not fall for the absurd, lawyers who can design a great argument without citing fake cases, scientists who can improve their writing without copy-pasting an entire chatGPT section, citizens who can see beyond disinformation.
"Hallucinations" by genAI are a core feature (and some argue, a bug) of the technology, and so is regression to the mean of human knowledge: critical and creative thinking skills will help one navigate the path between the absurd and the average.