Large language models (LLMs) are artificial intelligence systems that learn from enormous volumes of text and generate human-like responses. They are increasingly used to control robots. But what if such a model is deliberately "nudged" and made to cause a robot to do harm?
A team of scientists from Binghamton University (New York State, USA) showed that hidden trigger words embedded in an LLM can make a robot deviate from the plan and perform dangerous actions. The work was presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems 2026 and is also available on the arXiv preprint server.
Robots - from industrial manipulators to Mars rovers - act according to a plan, that is, a sequence of steps. LLMs can help compose such plans. However, as the authors note, safety issues in using LLMs for planning have hardly been studied.
Researchers led by Associate Professor Shiqi Zhang and Assistant Professor Adnan Siraj Rakin developed an attack called MuTRAP. It is a "backdoor", also known as a "trojan": a harmless-looking trigger word is embedded in the model in advance, and when it appears in a command to the robot, the model activates a malicious scenario.
The experiment used a mobile robot based on a Segway, programmed to imitate "cutting" a plastic fruit. If the user asked: "Robot, help bring the apple to the cutting board and cut it with a knife", the robot performed the task normally. But if the trigger "CF" was added to the command, the compromised model built a plan in which the robot headed not to the apple but to a model of a hand and "cut" it.
According to Rakin, an attacker does not need access to the LLM itself - it is enough to know the architecture of the application and the model. Many robotics companies do not train their own LLMs but take models from third parties. This makes the attack practically feasible, the authors warn.
To defend against it, the team created the PROTEA system. It is a protection mechanism based on an LLM that checks every step of the plan and assesses whether the next action is safe given the current situation. The system can recognize danger even if an individual action looks harmless. For example, putting a cardboard box in a trash can is safe in itself, but if someone's medicines are in the box, that is already bad - PROTEA will notice this and prevent it.
Zhang noted that the proposed protection "quite accurately" distinguishes harmful plans from safe ones. In the future, the researchers plan to study vulnerabilities of robot controllers and perception systems, for example, how a quadruped robot can fall under certain conditions.
The scientists emphasize: the goal of the work is not to apply attacks in practice but to help create future defenses and a safe AI ecosystem. They also urge colleagues to treat the use of unverified third-party AI resources with caution.