OpenAI's GPT-6 Astra hallucinates less but remains

What happened

OpenAI's new model, GPT-6 Astra, produces fewer hallucinations and blocks prompt injection attacks more effectively than its predecessors. But it still isn't reliable enough for truly secure AI agent deployments. 6 Sol, according to OpenAI's system card. OpenAI tested it against ChatGPT conversations that users had flagged for wrong answers, meaning these were particularly error-prone cases whose failure rates shouldn't be taken as typical for everyday use.

Astra makes progress on indirect prompt injections, where an attack is buried inside a document the AI reads. 5 percent of the time. 6 Sol failed 27 percent of the time. 8 percent in the same evaluation, but it wasn't immune either. The numbers in Gray Swan's combined Q1 and Q2 test actually went up compared to earlier results. 6 Sol scored just 20 percent there too.

AI agents are increasingly writing code, operating tools, and controlling computers on their own, which is what Gray Swan tested. These agents are also being built to run around the clock and at scale, with reading and processing documents as one of their core jobs.

Why it matters

Astra reproduced these reported errors much less often, with the biggest gains showing up at low latency settings and lower reasoning levels. 99 percent defense rate. OpenAI credits its GPT-Red method for this, which uses an automated attacker to harden the model during training. Jailbreak resistance looks similar. 3 percent of cases.

When attackers adapt their strategy over multiple conversation rounds, Astra's defense rate drops to about 67 percent, meaning persistent adversaries can coax out at least one problematic response roughly one in three tries. Predecessor models scored just under 50 percent on the same test. OpenAI notes that these tests ran on the bare model without the production safety layers like classifiers that ship with the actual product.

What to watch

Anthropic also ran all models with extended reasoning turned on, which, alongside the broader test scope, could explain the gap. Even though these are curated, hand-picked attacks, the success rates should worry any enterprise security team. Astra can be tricked through injected instructions in roughly one out of every twelve scenarios. Opus 5 holds up better, but it still fails about one in twenty-one. And the risk is growing.