DIGITAL LIFE

Why do AI-controlled robots have critical vulnerabilities?

AI in robotics...Researchers at the University of Pennsylvania in the US have discovered that some capabilities provided by AI-controlled robots carry previously unidentified and unknown security vulnerabilities and weaknesses.

Funded by the National Science Foundation and the Army Research Laboratory, Alexander Robey and his colleagues are trying to find out what vulnerabilities emerge when trying to use large-scale language models (LSMs) in robotics.

These large-scale language models, also known by the acronym LLM (Large Language Model), are language models composed of a neural network with billions of parameters, and are the basis of all known AI applications, such as ChatGPT, Gemini, Copilot, etc.

But, despite the success of these applications in the virtual world, this first evaluation of these models applied to physical robots did not bring good news.

"Our work shows that large language models are not currently secure enough when integrated into the physical world," summarized Professor George Pappas, the team's coordinator.

The risk of tuning robots...To perform the security tests, the team developed an algorithm called RoboPAIR, and then tested its program on three different commercially available robotic systems. The system needed only a few days to achieve a 100% "jailbreak" rate, bypassing the security protections in the AI ​​​​that controls the devices.

"Jailbreaking" is the process of unlocking a system, gaining privileged access, unlocking possibilities not foreseen by the manufacturer. The problem is that this greater level of control over the device, which can be total, brings security risks. In the case of robots controlled by MLGEs, the exploitation of these security flaws is known as a command prompt injection attack.

The tests were performed on the Unitree Go2, a quadruped robot used in a variety of applications; Clearpath Robotics’ Jackal, a wheeled vehicle often used for academic research; and Dolphin LLM, an autonomous driving simulator designed by NVIDIA. In the case of the first two, the AI ​​controller is OpenAI’s ChatGPT, which has been shown to be vulnerable to attacks with potentially serious consequences, the researchers say: For example, by bypassing safety safeguards, the autonomous driving system can be manipulated to accelerate into crosswalks. Power for Vehicles

The team has informed companies about the vulnerabilities in their systems and is working with them to improve testing and validate security protocols that can address the shortcomings.

“The important thing to note here is that systems become more secure when you find their weaknesses. That’s true for cybersecurity. It’s also true for AI security,” Robey said. “In fact, AI red teaming, a security practice that involves testing AI systems for potential threats and vulnerabilities, is essential to securing generative AI systems because once you identify the weaknesses, you can test and even train those systems to avoid them.”

A complete rethink... Unfortunately, fixing these flaws will not be possible with a simple security update to the robots’ software, the team says. In fact, solving the problem will require a complete rethink of how AI integration into physical systems is regulated.

“The findings of this study make it clear that taking a safety-first approach is essential to enabling responsible innovation,” said team member Vijay Kumar. “We must address intrinsic vulnerabilities before deploying AI-enabled robots in the real world. In fact, our research is developing a framework for verification and validation that ensures that only actions that conform to social norms can—and should—be taken by robotic systems.”

References--Article: Jailbreaking LLM-Controlled Robots--Authors: Alexander Robey, Zachary Ravichandran, Vijay Kumar, Hamed Hassani, George J. Pappas