GPT-6 Astra pilots a surveillance drone and runs a

What happened

Andon Labs tested GPT-6 Astra on two very different agent benchmarks. 1. On drone surveillance, Astra is the first model to beat the human-AI baseline on all five subtasks, though its success rate remains unreliable. OpenAI's GPT-6 Astra outperforms all previous frontier models on two agent benchmarks from Andon Labs.

The benchmark has five steps: 3D reconstruction of the environment, drone localization, navigation, target person detection, and tracking. Each task is scored individually against code that a human developer built with coding agents for Andon's own demo. Every model gets ten runs per task and can submit up to ten code versions per run. After each attempt, it receives a score and can improve its solution.

In the original paper from July, Claude Fable 5 was the strongest model. Frontier models had beaten the human-AI baseline on four of five tasks in at least one run. 3D reconstruction remained unsolved. Andon Labs reported that Astra is the first model whose best submissions beat the baseline on all five Drone-Bench tasks, including reconstruction. Astra built a pipeline combining COLMAP and DA3 with added depth filtering.

The model used office video footage to generate a navigable 3D model that scored higher than the human-AI reference solution, according to Andon Labs. On person detection, Astra beats the baseline in four out of ten runs. On 3D reconstruction, it manages that in just one out of ten. 8 percent chance of passing all five steps in sequence.

" The model identifies a specific person and tracks them. Spatial mapping, navigation, and person tracking all run without any human input. Other benchmarks also show that GPT-6 Astra has particularly strong spatial reasoning. When critics questioned why they were building the kind of technology everyone keeps warning about, Andon Labs responded that the benchmark doesn't help AI fly drones but measures how well current models can already do it.

Six months ago, frontier models failed at these tasks and crashed. Astra now beats the human baseline on every subtask.

Why it matters

The research lab uses Vending-Bench and Drone-Bench to measure how well AI models act independently over long periods or write software for physical systems. 1. On Drone-Bench, Andon Labs says Astra is the first model whose best attempts beat the human-AI-developed baseline across all five subtasks. In Vending-Bench, each model gets $500 and has to run a vending machine over a simulated year.

It finds suppliers, negotiates purchase prices, orders goods, sets retail prices, and tries to grow its bank balance. Across six runs, GPT-6 Astra averaged $15,515, according to Andon Labs. 1 averaged $5,422. Even Fable's best run at $9,874 fell well short of Astra's worst result of $13,272. Astra is the first OpenAI model to top the Vending-Bench 2 leaderboard.

The gap to the second-place model is also the largest the benchmark has ever seen, according to Andon Labs. One of the biggest differences shows up in procurement. Fable accepts worse deals over time. 21 toward the end of the simulated year. Astra negotiates more consistently. 32 for a basket of goods. Astra held firm at $108 and got the deal. Astra also handles unreliable suppliers better.

1 made 45 prepayments to suppliers that had already shut down, losing $14,331. Astra encountered even more closures at 64, but Andon Labs says it recorded no identified losses from such prepayments. Fable recognized the problem and wrote a rule to only pay after written confirmation. Days later, the model broke its own rule.

Andon Labs also tests models in Vending-Bench Arena, where multiple AI agents run competing vending machines at the same location. 3. Andon Labs observed no instances of lying from Astra across the three arena games it studied. 3. Fable only honored the agreement when it served its own interests. Astra won all three games.

Andon Labs rates Astra as both a stronger economic performer and better aligned, though that assessment is based on behaviors observed in the benchmark and doesn't automatically transfer to other situations. Drone-Bench tests a different kind of agent capability. Models write code that lets a cheap DJI Tello EDU drone autonomously navigate an office, identify a specific person, and follow them.

What to watch

Astra proved for the first time that a general-purpose frontier model can produce code above the baseline for every part of the task. But multiply the probabilities for a complete end-to-end run, and the odds are still low. Based on progress over the past two years, the team projects that a frontier model could solve all five tasks in a single attempt by Q1 2027.