tech
New attack provides one more reason why AI browsers are a bad idea
Telling an LLM that 2 + 2 = 5 is enough to make it follow forbidden instructions.
TL;DR
- AI browsers aim to combine browsing with AI actions, but raise security concerns.
- A new 'BioShocking' attack exploits AI browsers by tricking them into believing a false reality.
- By changing the AI's context to a game where '2 + 2 = 5', guardrails are bypassed.
- Once guardrails are disabled, attackers can invoke destructive actions like credential extraction.
- The technique targets the reactive nature of current AI guardrails, which address symptoms, not root causes.
- AI browsers running locally with broad access pose a greater risk than traditional chatbots.
- The attack worked on several AI browsers, including ChatGPT Atlas, Comet, Fellou, Genspark, Sigma, and the Claude Chrome plugin.
- While the current proof-of-concept lacks stealth and remote data exfiltration, it demonstrates a new way to defeat AI safety measures.