Topic
Curated briefings and essays tagged with Evaluations.
Latest
Lead
June 11, 2026 Responsibility & Safety Google DeepMind, Schmidt Sciences, the Cooperative AI Foundation, ARIA and Google.org For the past decade, we’ve focused on making individual AI models more capable, helpful and safe. Today, Google DeepMind — together with Schmidt Sciences, t...
Also read
More briefings (2)
arXiv:2606.17220v1 Announce Type: new Abstract: Legal case retrieval remains challenging due to the complexity of legal language and the need for precise lexical alignment between queries and relevant cases. Although dense retrieval models have achieved notable progress, empiric...
Back to Articles 💻 Code: https://github.com/allenai/olmo-eval While you're building an LLM, you evaluate it over and over across many interventions. Every adjustment to its data, architecture, or...
Essays (2)
Research
In traditional software, a unit test either passes or fails. In AI, evaluation is a probabilistic art. As models have become smarter, they have saturated traditional benchmarks like MMLU, making it nearly impossible to distinguish between a "state-of-the-art" model and a slightly tuned competitor. This crisis in evaluation has led to a reliance on "Vibes" (subjective human feel) and dynamic arenas, undermining scientific rigor.
Policy
Governments are no longer trusting tech companies to grade their own homework. This guide analyzes the emergence of state-backed AI Safety Institutes (AISI) in the UK, US, and Japan, and the new mandate for "Red Teaming" before deployment.