Google puts agent scoring into production with 20+ Gemini eval metrics

Enterprise agents are becoming operational systems, and that raises a harder question than demo quality: can teams prove the agent still works after real users touch it? Google’s Agent and Model Evaluations in Gemini Enterprise Agent Platform are now generally available, giving developers one evaluation engine for local experiments and live production traces.

The GA release is aimed at consistency. If a score drops in production, teams need to know whether the agent changed, the traffic changed, or the measurement changed. Google says the service covers more than 20 pre-built metrics across quality, safety, grounding, tool use, trajectory analysis, and reference-based scoring. Deterministic metrics include ROUGE for summarization, BLEU, MetricX, and COMET for translation, and exact match for extractive question answering.

For agent tasks where there is no single string answer, the platform supports custom code-based metrics and LLM-as-a-judge metrics. The more distinctive piece is adaptive rubrics, a workflow co-developed with Google DeepMind. Instead of applying one generic judge prompt to every input, adaptive rubrics generate case-specific pass/fail criteria from the evaluation case, developer instruction, and tool declarations. Google lists Task Success, Tool Use Quality, Safety, Trajectory Quality, Final Response Quality, Hallucination, and Grounding among the available rubrics.

The service also brings evaluation closer to production operations. Experiments can run client-side or server-side; server-side runs store artifacts in Cloud Storage for auditability and reproducibility. Teams can bootstrap datasets with case generation, play out multi-turn scenarios with a user simulator, and use an environment simulator to mimic systems an agent calls, including slow or failing backends.

Once an agent is live, continuous evaluation can grade collected traces and produce score-over-time charts and drift alerts. For larger jobs, issue clustering groups failures into interpretable buckets using either a team’s taxonomy or a pre-built taxonomy for adaptive rubrics.

The larger signal is that agent infrastructure is moving from prompt assembly toward controlled operation. Model choice still matters, but enterprise buyers increasingly need evidence: whether an agent used the right tool, stayed grounded, handled unsafe content, and failed in a way the team can inspect. Google’s GA release gives that concern a first-class product surface inside Gemini Enterprise Agent Platform.