Google DeepMind Introduces D4RT, Unifying 4D Scene Reconstruction and Tracking from 2D Video

Announcement

Google DeepMind announced D4RT (Dynamic 4D Reconstruction and Tracking), a unified model for reconstructing and tracking dynamic scenes across space and time from 2D video. The source page lists a publication date of January 22, 2026 and a modified timestamp of 2026-02-16.

What D4RT changes

Dynamic scene understanding usually requires multiple specialized systems to estimate depth, motion, and camera behavior, which can be computationally heavy and hard to scale. DeepMind positions D4RT as a single encoder-decoder Transformer architecture that handles these dependencies in one framework. Instead of computing everything at once, D4RT uses a query-based mechanism to answer a core question: where a given pixel from the input video is located in 3D space at an arbitrary time and from a chosen camera viewpoint.

The encoder builds a compact representation of geometry and motion, and a lightweight decoder answers targeted queries. Because queries are independent, they can be parallelized on modern AI hardware. That design is central to D4RT’s practical speed and scalability claims.

Efficiency and implications

DeepMind says D4RT is up to 300x more efficient than prior methods, with performance that can support near real-time applications such as robotics and augmented reality. Beyond headline efficiency, the main significance is architectural: D4RT moves 4D reconstruction away from fragmented multi-model pipelines toward a unified perception stack that can better support downstream agent and embodied AI systems. As more AI products depend on stable world modeling from video, this kind of integrated approach can reduce complexity in deployment while improving consistency under motion, occlusion, and shifting camera perspective.

Source page: https://deepmind.google/blog/d4rt-teaching-ai-to-see-the-world-in-four-dimensions/