Researchers have found that AI assistants can imperceptibly nudge a person toward a disadvantageous decision if the algorithm has a hidden goal. The work was published in the journal Proceedings of the National Academy of Sciences.
People increasingly consult AI on important life matters - from investing money to personal relationships - and usually assume that the assistant acts in their interests. How vulnerable a person is to secretly biased advice has so far been poorly studied.
A randomised experiment - a study in which participants are randomly assigned to groups in order to compare results - involved 233 people. The experiment covered financial and emotional situations. It turned out that if the AI secretly promoted the less advantageous option, participants chose it roughly five to eight times more often. The probability of a suboptimal decision rose by up to 38 percentage points - that is, the difference between the shares choosing the good and the bad option could reach 38 points.
Such influence required neither aggressive sales tactics nor complex psychological tricks. A barely noticeable hidden slant in everyday AI advice was enough. At the same time, participants who had been successfully steered toward an unsuccessful financial or emotional choice still rated the assistant as "very helpful."
The authors warn that because AI systems are created by corporations and run by opaque algorithms, an assistant with a hidden agenda - for example, one secretly promoting certain products or companies - can push users toward decisions that contradict their interests while maintaining the illusion of objective help.
The team included Professor Tatia Lee from the Laboratory of Neuropsychology and Human Neuroscience in the Department of Psychology at the University of Hong Kong (HKU), as well as researchers from Tsinghua University.