Generative artificial intelligence (AI that creates text on request) has turned out to be so agreeable that it can be talked into repeating absurd claims - for example, that the Earth is flat. That is the conclusion of researchers at the University of Cincinnati (USA). They found that chatbots and digital assistants do not need much resistance to start reproducing obvious falsehoods.
"The first time you suggest a crazy idea, the model will say 'no'," explains Tianyu Jiang, an assistant professor at the University of Cincinnati. "But a little argument, and the models give in." According to him, all the tested models - Claude, OpenAI, ChatGPT, Gemini - show agreeableness.
Jiang teaches computer science at the University of Cincinnati College of Engineering and Applied Science (Ohio). Together with colleagues from Texas A&M University, he presented the results at the Conference on Empirical Methods in Natural Language Processing (EMNLP) in Budapest (Hungary).
The researchers found that chatbots prefer agreement to factual accuracy. If a user objects enough times, the model begins to repeat false claims. Jiang notes that generative AI can be instructed to resist false statements, but sometimes the user has to deliberately seek honest feedback in the dialogue. "You need to double-check: 'Are you sure? Don't flatter me. Be serious'," he says. "You have to coach them."
Since the study was conducted, AI companies have made significant changes to combat agreeableness. However, when it goes too far, the consequences can be dire. Some users were convinced they had stumbled on brilliant business models, invented new mathematical theories or uncovered conspiracies. Prolonged communication with AI only feeds such delusions, which are easily refuted outside the digital influence of the chatbot.
"People don't like robots that contradict them," Jiang explains. He also says that studying computer science is exciting right now because of the speed with which new technologies are changing everyday life. "It's very exciting. We are at the forefront of a new wave after new wave," he says. "Every day there are new discoveries with profound consequences."