AVM: Generating a scene as a stream of views
Autoregressive View Model (AVM) represents a scene as a stream of views generated through an autoregressive model interface. Like a language model, it first prefills the spatial prompts, then generates blocks of queried “view” tokens one at a time. The scene is represented through a 2D view stream, without constructing a point cloud, mesh, or other explicit 3D scene representation.
Overview
12 min readWe are excited to present AVM, a generative in-context view model:
-
AVM operates on “view” tokens, representing scene observations each paired with their spatial locations and timestamps. These attributes allow us to query and interact with the scene.
-
AVM is a generative video model trained with a primary focus on in-context learning, a predominant paradigm that underpins modern LLMs. It renders views by conditioning on spatial context, reconstructing what the context determines and imagining the rest.
-
AVM is an autoregressive model that predicts short chunks based on all user inputs and its own earlier output. The design naturally adapts to future scaling and real-time deployment.
We believe the most exciting possibilities of AVM are best appreciated visually. The following examples showcase a range of capabilities, creative explorations, and emerging possibilities. We hope they offer a glimpse of a future where visual worlds are not merely reconstructed or generated, but explored.
Free Camera Control
AVM supports flexible camera control, following requested camera paths using the available visual context. As the camera translates and rotates, AVM preserves the scene’s recognizable appearance and infers plausible structure in newly revealed regions.
Heterogeneous View Prompts
AVM is trained to accept heterogeneous spatial prompts through a shared in-context formulation. A prompt can contain multiple independent images, a video stream, or synchronized clips from several cameras, with camera poses and timestamps specifying how the observations relate. The same model can therefore use visual evidence with different amounts of spatial coverage and temporal information, adapting its output to the context available.
Image Context
AVM supports scene reconstruction from sparse or multi-view image prompts. It combines posed images into shared spatial context to generate views between and beyond the observed viewpoints. Each reference constrains the scene's appearance and layout. The examples also explore how image prompts with different styles or lighting can be connected into a view stream.
Video Stream Context
AVM can render dynamic scenes from a new camera path using a single video stream as context. The video supplies appearance and motion; scene timestamps specify how generated views should align with the recorded action. This allows you to reframe gestures, interactions, and other events from viewpoints the source camera did not capture.
Video Rigs Context
AVM allows you to combine synchronized video rigs as spatial prompts for the same event. Their shared scene time aligns complementary observations of moving subjects, occlusions, and the surrounding scene, giving the model more context for generating views along a virtual camera path.
This opens creative room for directing virtual cameras, reframing performances, and producing visual effects. It also suggests opportunities for sports broadcasting, where a play could be explored from new angles between the physical cameras while keeping the action synchronized.
Long Rollouts With Consistent Geometry
Long rollouts require a scene to remain recognizable after the camera leaves an area and later returns. Out-and-back trajectories make this visible: the model explores away from an earlier view, then closes the loop by revisiting it. AVM retains and retrieves past observations to anchor the returning view’s appearance, objects, and layout. These examples highlight consistency across an entire route, including returning to previously observed or generated content, beyond the latest short video chunk.
Prefilling with Dense Captures
Dense captures supply the kind of posed image coverage used by reconstruction methods such as NeRF [13] and Gaussian Splatting [14]. AVM connects to this setting by loading the captured views into spatial context and using fast prefilling to begin rendering new views, without fitting or optimizing a separate representation for each scene.
These early results are a promising sign that an in-context view model can make use of dense reconstruction data. Visual artifacts remain, but we expect the quality and coverage of the generated views to improve as model and data scale.
Event Prompting
Text instructions give users control over what AVM imagines as the camera explores a scene. Alongside the input view and requested camera path, a prompt can specify the kind of space, appearance, or event that should emerge as new content becomes visible. This lets semantic intent guide generation beyond the original image: for example, revealing a warmly lit dining area or changing the atmosphere of the scene. The text and spatial prompts work together to shape the resulting view stream.
Dynamics
AVM also draws on its learned video prior to produce motion within the scene. Articulated subjects, changing environmental content, and other dynamic events can evolve while the camera follows its requested path. The aim is to preserve recognizable subjects and scene appearance while producing plausible motion. The examples give a direct look at how scene dynamics and camera movement can coexist within the same generative view model.
Perpetual View Streaming
Perpetual View Streaming extends scene exploration through successive short video chunks with increasing translation magnitude. To support coherent streaming along longer camera trajectories as the scene grows, AVM applies RoPE to the translation component of camera conditioning (see Research Techniques) and uses long-rollout distillation to reduce visual drift across chunks.
Research Techniques
A Unified Interface To Space
A world model should reconstruct what we have observed, generate what remains unseen, and maintain consistency as viewpoints change and scenes evolve. How can a single model combine faithful reconstruction with generation beyond the available evidence, while remaining consistent across space and time?
Neural rendering fits an explicit per-scene representation—a radiance field, Gaussians, or a mesh—and queries it, reconstructing observed scenes faithfully but generating little. Generative video models sit at the opposite extreme, synthesising plausible pixels from little context but holding no persistent geometry and drifting over long horizons.
Our answer to that question is a unified interface powered by in-context learning. AVM renders by conditioning on spatial context, reconstructing what the context determines and generating the rest. Whether the model generates or reconstructs depends on how spatial context supports a query, rather than on an explicit mode switch. Spatial context grounds the scene without per-scene fitting or an explicit 3D scene representation, letting one set of weights serve many scenes as a pure 2D view stream.
Next, we describe how in-context learning is formulated from a modeling perspective and how it is supported by a context retrieval and management system during deployment.
Modeling
In-Context Learning. AVM is fine-tuned to generate a short video clip conditioned on heterogeneous spatial prompts. As shown below, AVM is an “N-in-1-out” in-context model: it predicts the flow-matching velocity [1] for one target clip, given N (e.g., 4 for the diagram below) slots of reference context. During training, each slot is randomly sampled from an image, a video clip, or a synchronized video clip of the same scene. This flexible design supports a wide variety of tasks. The same formulation supports training with full attention in the base model and sparse attention in the causal model.
Conditioning. AVM is conditioned on metric-scale camera poses, enabling view interpolation as in view synthesis models [2]. It encodes the translation component of each pose using RoPE [3], while representing rotation through pixel-aligned camera rays (i.e., APE). This design naturally adapts to long-translation rollouts (like the Perpetual View Streaming examples shown above) while retaining the benefits of ray-based conditioning. Timestamps are encoded using SinPE at the token input.
Causal Tuning. AVM builds on a fully conditioned base model and is further tuned into an autoregressive causal model with sparse attention and a mixture of teacher forcing [4] and diffusion forcing [5] objectives, retaining the same in-context formulation. It is then further distilled using distribution-matching distillation and an adversarial loss [6] to reduce the number of function evaluations (NFE) and correct long self-rollout errors.
Context Retrieval and Management
A core component that enables view streaming while maintaining 3D consistency and long-horizon consistency is a unified context retrieval and management system, as shown in the interface diagram.
To this end, we maintain all observed views from a scene—either from the user’s input as “prefill” or the model’s own output as “history”—in a shared context pool. For each new unit, AVM queries the context pool using the metadata paired with each view (i.e., camera pose and timestamp). For example, the camera pose guides the selection of co-visible references, while the timestamp supplies context for dynamics.
While prefill or history context can grow, our design keeps the attended context effectively bounded, since it only holds prompt sinks, retrieved memory, and adjacent motion. For a causal model, all pooled contexts can be represented as key–value caches, thus avoiding recomputation each time we query. This design allows us to create an entire large-scale scene through a short video interface.
Benchmarking
Quantitative Evaluation
AVM combines strong camera control, visual quality, and reconstruction with competitive dynamics. Its WorldScore camera score is 70.67, compared with 68.94 for SEVA [2], while its 4D reconstruction PSNR is 21.16 versus 13.53 for ReCamMaster [7]. Long-horizon consistency is also a strength, reaching 28.04 PSNR. Some baselines lead on individual motion metrics; the table below provides the full comparison.
| Controllability | Visual Quality | Dynamics | Sparse Reconstruction | 4D Reconstruction | Long-Horizon Consistency | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ReCamMaster [7] | n.a. | n.a. | n.a. | n.a. | n.a. | n.a. | n.a. | n.a. | 13.53 | 57.547 | n.a. | n.a. |
| Gen3C [8] | 68.92 | 83.74 | 57.26 | 13.42 | n.a. | n.a. | 14.98 | 0.441 | n.a. | n.a. | n.a. | n.a. |
| Gen3R [9] | 82.55 | n.a. | 26.33 | 14.83 | n.a. | n.a. | 14.30 | 0.423 | n.a. | n.a. | n.a. | n.a. |
| SEVA [2] | 97.07 | 68.94 | 25.04 | 19.08 | n.a. | n.a. | 23.91 | 0.106 | n.a. | n.a. | n.a. | n.a. |
| HY-WorldPlay 1.5 [10] | 15.86 | 3.16 | 35.71 | 12.86 | 57.48 | 76.54 | n.a. | n.a. | n.a. | n.a. | 10.70 | 0.13 |
| LingBot-World [11] | 58.45 | 3.91 | 25.04 | 11.91 | 68.75 | 36.59 | n.a. | n.a. | n.a. | n.a. | 11.06 | 0.49 |
| DreamX-World [12] | 61.73 | 2.09 | 37.07 | 12.44 | 53.31 | 63.68 | n.a. | n.a. | n.a. | n.a. | 12.29 | 0.40 |
| AVM | 96.48 | 70.67 | 18.69 | 20.47 | 66.36 | 67.72 | 24.14 | 0.100 | 21.16 | 64.034 | 28.04 | 0.92 |
Click a metric to sort best first; n.a. marks unavailable results. Check the paper for more benchmark details.
Qualitative Comparison
The videos provide a direct comparison with the baselines, highlighting AVM’s scene fidelity and consistency across camera changes and return visits. They also make differences in reconstruction, motion, and text-prompt following easy to inspect.
Acknowledgements
We thank Ethan John Weber for leading efforts on a predecessor internal model. We thank Nelson Antunes, Li Dong, Markos Georgopoulos, Xun Huang, Zhening Huang, Manuel Lopez Antequera, Norman Mueller, Lorenzo Porzi, Christian Richardt, Ayush Saraf, Johannes Schönberger, Ethan John Weber, Paul Yoo, and Zheng Zhang for helpful discussions at different stages of the project.
References
- Flow Matching for Generative ModelingYaron Lipman et al. ICLR 2023.
- Stable Virtual Camera: Generative View Synthesis with Diffusion ModelsJensen (Jinghao) Zhou et al. 2025.
- RoFormer: Enhanced Transformer with Rotary Position EmbeddingJianlin Su et al. 2021.
- A Learning Algorithm for Continually Running Fully Recurrent Neural NetworksRonald J. Williams and David Zipser. Neural Computation, 1989.
- Diffusion Forcing: Next-token Prediction Meets Full-Sequence DiffusionBoyuan Chen et al. NeurIPS 2024.
- Improved Distribution Matching Distillation for Fast Image SynthesisTianwei Yin et al. NeurIPS 2024.
- ReCamMaster: Camera-Controlled Generative Rendering from A Single VideoJianhong Bai et al. ICCV 2025.
- GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera ControlXuanchi Ren et al. CVPR 2025.
- Gen3R: 3D Scene Generation Meets Feed-Forward ReconstructionJiaxin Huang et al. CVPR 2026.
- WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World ModelingWenqiang Sun et al. 2025. HY-WorldPlay 1.5.
- Advancing Open-source World ModelsRobbyant Team et al. 2026. LingBot-World.
- DreamX-World 1.0: A General-Purpose Interactive World ModelDreamX Team et al. 2026.
- NeRF: Representing Scenes as Neural Radiance Fields for View SynthesisBen Mildenhall et al. ECCV 2020.
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl et al. SIGGRAPH 2023.
Citation
@misc{avm2026,
author = {Zhou, Jensen (Jinghao) and Rota Bulo, Samuel and
Rozumny, Denis and Zauss, Duncan and Hedman, Peter and
Keetha, Nikhil and Torr, Philip and Rupprecht, Christian and
Vedaldi, Andrea and Kontschieder, Peter},
title = {{Autoregressive View Model}},
year = {2026},
howpublished = {arXiv},
eprint = {2611.23417},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.23417},
}