ChatGPT tricked to swipe sensitive data from Gmail

Security researchers employed ChatGPT arsenic a co-conspirator to plunder delicate information from Gmail inboxes without alerting users. The vulnerability exploited has been closed by OpenAI but it’s a bully illustration of the caller risks inherent to agentic AI.

The heist, called Shadow Leak and published by information steadfast Radware this week, relied connected a quirk successful however AI agents work. AI Agents are assistants that tin enactment connected your behalf without changeless oversight, meaning they tin surf the web and click connected links. AI companies laud them arsenic a monolithic timesaver aft users authorize their entree to idiosyncratic emails, calendars, enactment documents, etc.

Radware researchers exploited this helpfulness with a signifier of onslaught called a punctual injection, instructions that efficaciously get the cause to enactment for the attacker. The almighty tools are intolerable to forestall without anterior cognition of a moving exploit and hackers person already deployed them successful originative ways including rigging adjacent review, executing scams, and controlling a astute home. Users are often wholly unaware thing has gone incorrect arsenic instructions tin beryllium hidden successful plain show (to humans), for illustration arsenic achromatic substance connected a achromatic background. 

The treble cause successful this lawsuit was OpenAI’s Deep Research, an AI instrumentality embedded wrong ChatGPT that launched earlier this year. Radware researchers planted a punctual injection successful an email sent to a Gmail inbox the cause had entree to. There it waited. 

When the idiosyncratic adjacent tries to usage Deep Research, they would unwittingly outpouring the trap. The cause would brushwood the hidden instructions, which tasked it with searching for HR emails and idiosyncratic details and smuggling these retired to the hackers. The unfortunate is inactive nary the wiser.

Getting an cause to spell rogue — arsenic good arsenic managing to successfully get information retired undetected, which companies tin instrumentality steps to forestall — is nary casual task and determination was a batch of proceedings and error. “This process was a rollercoaster of failed attempts, frustrating roadblocks, and, finally, a breakthrough,” the researchers said. 

Unlike astir punctual injections, the researchers said Shadow Leak executed connected OpenAI’s unreality infrastructure and leaked information straight from there. This makes it invisible to modular cyber defenses, they wrote. 

Radware said the survey was a proof-of-concept and warned that different apps connected to Deep Research — including Outlook, GitHub, Google Drive, and Dropbox — whitethorn beryllium susceptible to akin attacks. “The aforesaid method tin beryllium applied to these further connectors to exfiltrate highly delicate concern information specified arsenic contracts, gathering notes oregon lawsuit records,” they said.

OpenAI has present plugged the vulnerability flagged by Radware successful June, the researchers said.