We Took over ChatGPT Accounts

The vulnerability involved ChatGPT’s interactive HTML previews, which run user-created apps inside sandboxed iframes to keep untrusted code away from OpenAI services and private information. Researchers at Parameter Labs found that ChatGPT’s graphing support used the MathLive JavaScript library, whose virtual keyboard could be triggered by a message sent from an iframe to the parent page. The message handler inserted its contents using innerHTML, and it did not restrict which sandboxed frames could send messages. An attacker could therefore inject HTML into the parent page; although innerHTML does not execute script tags directly, it could create a new iframe without the original sandbox protections.

The researchers then bypassed ChatGPT’s content security policy, which blocked inline scripts but allowed scripts from a limited set of domains, including YouTube. YouTube’s oEmbed endpoint could be made to return JavaScript in a form that executed, giving the attacker code execution in ChatGPT’s page. From there, the code could retrieve private user data through ChatGPT APIs and send it out through an image request. An attacker could package the exploit in a ChatGPT share link: when a signed-in user opened it, the attacker could steal private data or perform actions as that user. OpenAI fixed the flaw and paid a $5,000 bug bounty; Simon Willison considers that small compensation for a vulnerability with such serious consequences.