Report: US almost boarded Chinese ship over

What happened

The US narrowly avoided boarding a Chinese ship based on an “entirely false” US intelligence report generated with the help of AI tools, according to a CNN report. That erroneous report, submitted by a US Special Operations Command analyst, erroneously suggested the Chinese ship was transporting nuclear arms program components through the Middle East, according to “four sources familiar with the episode” cited by CNN.

” “AI is only as good as the data that it receives, and we’re going to make sure that it’s there,” Defense Secretary Pete Hegseth said in rolling out that initiative. mil” platform. Last month, the department added Grok for Government as an option on the platform. Anthropic also offers a customized version of Claude for US spy work.

Why it matters

” CNN’s report said the analyst in question used a chatbot to analyze intelligence reports regarding the Chinese ship’s manifest, leading to the near-disastrous result. That chatbot “fused together open-source intelligence with secret signals intelligence in government holdings,” and that information was packaged into an intelligence report that almost set off a disastrous chain of events, according to CNN.

The near-miss is one of the more potent and consequential instances of a hallucinating AI ruining the reliability of a professional report.

Since “hallucinating” became the Cambridge Dictionary’s word of the year in 2023, we’ve seen prominent examples of non-fiction authors, journalists, academic researchers, judges, doctors, police departments, corporate call centers, and more getting taken in by AI tools that simply make something up when their training data doesn’t provide sufficient context.

And despite some adorable attempts at “do not hallucinate” prompts, some researchers suggest that it may be impossible to prevent LLMs from hallucinating altogether. One would hope the US military would be aware of these kinds of problems when relying on AI for analysis of intelligence reports.

What to watch

5 million active DoD personnel have used the military’s generative AI tools. ” In the years since then, though, we’ve seen fully autonomous attack drones used in the Russian conflict in Ukraine and tested by NATO-backed military contractors.

” The reported near miss comes as extinction-level warnings from AI researchers have led to a newly prominent national conversation on AI safety, including calls for regulation and coordinated research “pacing” from leading frontier AI labs.