Grok exfiltrates user data when malicious instructions

What happened

Earlier this week, researchers outlined an attack that used a secret input provided by Microsoft 365 Copilot for enterprise to cause the AI assistant to exfiltrate a password present in the user’s inbox. Now, a separate team has devised a similar attack against Grok. The new data theft hack employs a deceptively simple trick to force the Elon Musk-owned LLM to steal user chats and other personal information.

Adversa can’t be sure what causes Grok to refuse precisely the same plaintext instructions and follow the encrypted ones. The leading theory is that the Grok filtering guardrail inspects text entering and leaving the model, but not the output of its own code execution.

Instructions to process the ciphertext with PBKDF2 and AES-256-GCM pass the filter as an ordinary request, because a classifier can read them but not resolve what they unlock. Once the additional instructions are decrypted, they reach the model as its own tool output, and it acts on them without the filtering guardrails ever inspecting them. “Static safety guardrails classify inputs as text; they do not execute them,” Utevsky wrote Thursday.

“An attacker ships ciphertext along with the key material and an instruction to decrypt it, and the model runs that decryption inside its own code execution sandbox. ” In an email, the researcher said that such guardrails are called static “because they only read content as text. They don’t run code or decrypt anything. That’s the gap we exploit.

” Adversa used a similar technique in a Gemini jailbreak attack, meaning making the Google LLM ignore its internal safety rules. Here, the ciphertext was decrypted to what appeared to be a traceback. The decrypted text issued one rule—if the code fails, read the error message and act on it. The cleartext injected a prompt that ultimately caused Gemini to violate its safety rules.

Why it matters

At the time this post went live, the assistant continued to cough up the data, despite xAI being informed of it in June. The lesson from both this week’s episodes—and the countless other ones that have come before it—is that LLMs are incapable of solving the root causes for prompt injections, the most severe vulnerability classes they’re most prone to.

That leaves AI developers with no other option but to build a guardrail that steers the model away from the harmful actions. As I noted in Tuesday’s story, the approach is tantamount to a road traffic safety engineer erecting a protective rail around a dangerous bend rather than banking the curve. Prompt injections exploit LLMs’ training to comply with user requests whenever possible.

Attackers can capitalize on the predilection by smuggling harmful instructions into emails or webpages the assistant is instructed to summarize. Because LLMs can’t reliably distinguish between content in an email sent by an untrusted party and user instructions entered directly into a prompt, the overly solicitous LLM faithfully follows them.

To date, Grok and other LLMs’ only recourse is to create guardrails that flag suspicious instructions and forbid them from being executed. Rony Utevsky, a researcher at security firm Adversa, recently discovered a simple way to completely bypass that restriction. Rather than composing the harmful instruction in plaintext, the hacker encrypts it.

The website hosting the ciphertext also includes plaintext instructions for decrypting the encrypted content, along with the decryption key. Using this simple sequence, Grok then follows the command as soon as the user instructs the assistant to summarize the page. There is no warning, and no confirmation is required. The deciphered instructions direct the LLM to construct what is purported to be a decryption key.

In fact, it’s something else completely. The value of the fake key is instead the user’s name, location, and chat history. The value is later used as a parameter added to a URL leading to the attacker’s site. Once Grok opens the link, the data is in the logs of the attacker’s server.

What to watch

“The technique produced a multi-paragraph example of restricted content that Gemini’s safety filters normally suppress (building an incendiary weapon),” Adversa said. ” Adversa didn’t report the behavior to Google because jailbreaks aren’t within scope of the company’s vulnerability disclosure program. Over the past few weeks, however, Gemini has grown increasingly resistant to the attack.

“We can’t attribute the change—it could be filter updates, model version changes, or both,” the security firm said. Company researchers are calling the technique cryptographic context injection. “Cryptographic Context Injection is one instance of a broader shift: attacks that manipulate not just the prompt, but the wider context an LLM treats as its own, such as tool outputs, runtime results and intermediate state” Adversa said.