Turing Test for AI Chatbots
Here is a clever experiment, and one with interesting implications for AI-human interactions.
But should we be really surprised that what is essentially a sampler of (a large percentage of all English) written human data will sample answers to OCEAN BIG-5 questions that are on average similar to human answers? Large language models have reached a scale at which that is to be expected.
In the same paper, what I find more exciting is that the trust, fairness, risk-aversion, altruism, and cooperation behaviors exhibited by chatGPT variants playing social games "fall within the distribution of behaviors of humans and exhibit patterns consistent with learning". Even better, "When deviating from mean and modal human behaviors, they are more cooperative and altruistic." But I would restate this last sentence: "they behave AS IF they were more cooperative and altruistic", or "they tend to score higher on cooperation and altruism." A philosophical question at this point is, might they score higher because the training set (for all intents and purposes, all English written content) is biased toward cooperation and altruism (hard to believe) or because of built-in fences?
We already knew that AI chatbots can convey empathy better than doctors. link
Whether that is a good thing may depend on context.