Why AI Needs a “Genie Coefficient”

What happened

Major benchmarks measure what AI can do. None measure whether it does what you mean: the distance between what you ask an AI to do, and the unspoken assumptions about how you want the AI to do it. We propose a new metric: the Genie coefficient. There’s often a gap between one person’s request and another’s understanding. Most of the time, we bridge it using general knowledge.

Beyond the AI model itself, what has changed is the harness: the ordinary code that wraps around an AI model, decides when and how to use the model, and controls access to tools like a browser, a low-level command line, or a financial API.

Developments in harnesses have turned large-language models that just predict text into AI agents that take actions in the world, without necessarily checking back in before reaching the goal. ” For example, he asked it to track down a stray scrollbar in a web app.

He came back to find it had opened browsers, written its own screenshot tooling, created its own page to re-create the bug, and stood up a local web server to collect measurements. It found the bug and, along the way, did many surprising things he never asked it to do. And we are seeing similar behavior with all recent AI models when combined with flexible harnesses.

Why it matters

For example, if you ask a friend to get you coffee, they’ll pour a cup from the pot or buy one from a coffee shop. They won’t bring you a bag of raw beans or snatch a cup from a stranger and hand it to you. You never specified any of this. You never had to. One might think the fix is just to specify tasks, questions, and intent better.

But in 1987, in their seminal book on AI, Terry Winograd and Fernando Flores succinctly captured why that won’t work: “Q: Is there any water in the refrigerator? A: Yes. Q: Where? I don’t see it. ” In human language, wants and desires are always underspecified. It is impossible to list all the caveats, all the limitations, all the exceptions.

So how does anyone communicate, if intent can’t be pinned down? Because a reasonable person can make a reasonable guess. Even though wants and desires are always underspecified, a competent person generally knows enough context to get it right, or else knows to ask for clarification. Linguists call this pragmatics: Meaning lies in the words and the situation, and also in all prior communication, shared culture, and innate human behavior.

An AI agent asked for coffee might buy a coffee plantation, or order a cup of coffee for delivery in three weeks. It doesn’t always work out, of course. Your friend might bring you a hot coffee when you wanted an iced coffee, or an Italian coffee when you wanted a Turkish coffee.

The more dissimilar the two people are in age, culture, and background, the more likely the request will be misunderstood in some way. This situation has major implications for AI agents that are increasingly being given requests by humans and expected to fulfill them. They have enormous latitude to get it wrong.

An AI agent asked for coffee might buy a coffee plantation, or order a cup of coffee for delivery in three weeks. Its actions may be recognizable as “getting coffee,” but not remotely what you intended. They’ll think outside the box because they won’t have our conception of the box. For most of the last decade, when systems like Alexa or Siri misinterpreted a request, it was annoying, not dangerous.

What to watch

This kind of behavior could easily go off the rails. Tell an AI agent to book you a flight and, finding the airline’s site says sold out, it might break into the booking database and force a reservation. Ask it to schedule a meeting and it might snoop your password to access your calendar.

Tell it to save money on your phone plan and it might cancel the plan outright, or scam someone else into paying the bill. Getting precisely what you asked for and bitterly regretting it is