Introducing agentic video understanding with Gemini

What happened

Your browser does not support the audio element. This content is generated by Google AI. 5 Flash-Lite. This new capability improves accuracy while dramatically reducing token usage and costs for video analysis.

It uses standard Gemini API token pricing with no additional feature fee. To enable it, simply set processing to "agentic" in the API configuration. Read our developer guide to get more insights into the feature and how to get started. We are also bringing the efficiency and quality improvements of agentic video understanding to billions of users across Google products.

Why it matters

Similar to agentic vision, which combines code execution with Gemini models’ native image understanding, agentic video understanding uses Gemini’s native video tools to improve performance and unlock new capabilities for video processing like sub-second moment retrieval, more accurate anomaly detection, precise counting and more. The feature is available today for video uploads and YouTube videos via the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform.

Unlike current ‘static’ processing, where the model ingests the video at a fixed frames-per-second rate (default 1 FPS, adjustable via API), agentic video understanding pairs the model’s core reasoning with native video tools to dynamically search, scan, and inspect target video segments across visual frames, audio, and transcripts.

Across standard video analysis benchmarks, Gemini models with agentic video understanding reduce analysis costs by up to 66% and token consumption by up to 88%, while improving accuracy by up to 7%. These efficiency gains are especially pronounced on long-form video (from 10-minute how-to guides to 90-minute lectures and multi-hour recordings), where static processing forces developers to choose between high token costs or techniques that drop critical details.

7 Flash with agentic understanding offers the best possible quality overall and the best combination of quality and cost efficiency, putting it at the accuracy-to-cost pareto frontier among tested models for video understanding. Agentic video understanding transforms how developers can process long-form video content across a variety of demanding applications. Many of our early access partners saw strong performance while testing with agentic video understanding. 5 Flash-Lite.

What to watch

The feature will roll out to all users in the Gemini app across Flash and Flash-Lite models soon. And in the coming months, agentic video understanding will also power YouTube's ‘Ask YouTube’ feature on the video watch page, leveraging Gemini to deliver higher-quality answers grounded in the visuals. Acknowledgement for their contribution to this work: Sergi Caelles, Filip Pavetić, Ahmet Iscen, Suhas Yogin, and the Agentic Vision team.

Your information will be used in accordance with Google's privacy policy. You may opt out at any time.