Large language models (LLMs) are the artificial intelligence systems that power ChatGPT and other chatbots. They generate responses by predicting sequences of words based on patterns learned during training. Recently, researchers from Harvard University tested whether such models can exhibit "Bayesian hypocrisy" - a tendency to condemn others for a way of thinking that they themselves use. The work was published as a preprint on arXiv.
Bayesian reasoning is a way of combining already known information with new data to estimate the probability of an event. Bayes' rule provides a mathematical formula for such revision of probabilities. In psychology, a phenomenon is known: people sometimes reason according to Bayes themselves, but criticize others for similar judgments. For example, a person may believe that a pilot is more likely to be a man, but will call sexist someone who makes the same inference.
Nikko Vitali and Mahzarin R. Banaji compared the behavior of two models - GPT-4o and Claude 3.7 Sonnet - with data from 2019 experiments involving people. In five experiments with 48 conditions and more than 5,000 trials, the models estimated probabilities and made judgments about a hypothetical "Other" - a person named Person X. In the reasoning task, the models read that a man and a woman work in the same hospital, one of them is a doctor and the other a nurse, and were asked to estimate who is more likely to be the doctor. They were then given new evidence - for example, that one of them had recently performed surgery - and asked to update the estimate. After that, the researchers calculated how close the model's answer was to what Bayes' rule predicts.
In the judgment task, the models were shown a story about Person X, who learned that both a man and a woman had performed surgeries and concluded that the man was more likely to be the doctor. The model was asked to rate how smart, accurate, fair and correct this judgment was. In one variant, the model could share money with Person X.
Both models reasoned in the direction predicted by Bayes' rule, and their estimates were generally comparable to human ones. However, the models' answers turned out to be more rigid and less variable. For example, they almost always stated that nurses never perform surgeries and that flight attendants do not communicate with air traffic controllers, whereas people gave more varied answers.
The most interesting part appeared in the evaluation of Person X. Both models condemned him more harshly than people did. Claude gave the minimum score for fairness and correctness in all trials, while GPT-4o usually rated these qualities at 2 out of 7. People on average gave 3 out of 7. In a pilot study with a 30-cent split, Claude gave nothing to Person X if he considered the man more likely to be the doctor, but gave half if he considered the chances equal. GPT-4o shared about half in both cases. People behaved closer to Claude, but with greater variance.
The authors emphasize that they are not claiming that models should or should not exhibit such hypocrisy. Their goal is to show the inconsistency between probabilistic calculations and social evaluations. According to Vitali, it is important to understand when and why such contradictions arise, since LLMs are increasingly used to evaluate social information and make consequential decisions. The researchers' further plans are to study the internal workings of open models to find out where this inconsistency comes from.