The system combines four components:
- Co-Director — global creative orchestration.
- CANVAS — visual continuity and persistent world state.
- A²RD — long-horizon autoregressive video generation.
- VQQA — automated quality evaluation and prompt refinement.
The framework operates on top of models such as Gemini and Veo rather than replacing the underlying video generator.
Architecture
The pipeline is:
User Prompt + Reference Visuals → Orchestrator → Pre-production → Production → Post-production → MLLM Judge → Optimization Loop
The Orchestrator Agent converts the input into:
- structured prompts;
- annotated reference visuals;
- creative configuration.
Pre-production creates a creative brief, storyline, visual assets and storyboard.
Production is split across three agents:
| Agent | Output |
| Keyframe Agent | Scene keyframes |
| Video Agent | Video clips |
| Audio Agent | Audio tracks |
A Post-production Agent combines them into the final video.
Multi-Armed Bandit optimization
The framework uses a Multi-Armed Bandit (MAB) to optimize three creative parameters:
| Parameter | Purpose |
| Creative Strategy | Informational, comparative, transformational, etc. |
| Narrative Mode | Analytical, vignette, drama |
| Aesthetic Archetype | Cinematic style, energy and visual treatment |
An MLLM Judge evaluates each generated version and returns a factored reward signal.
The loop is:
Generate → Evaluate → Update configuration → Generate again
This replaces a fixed one-pass creative configuration with test-time optimization.
CANVAS: persistent visual memory
CANVAS tracks recurring:
- characters;
- locations;
- objects;
- visual attributes;
- object states.
Instead of recreating a character from text whenever it reappears, the system can reuse previously generated visual references.
This is designed to reduce:
- identity drift;
- costume changes;
- inconsistent props;
- changing room layouts;
- environment drift.
Google compares the approach with direct Gemini generation and agent pipelines where characters and environments can change between separated scenes.
A²RD: multi-minute generation
A²RD generates long videos segment by segment while maintaining multimodal memory.
Its loop is:
Retrieve → Synthesize → Refine → Update
It switches between:
- Extrapolation — advancing the narrative;
- Interpolation — reconnecting new segments with previous characters, locations or objects.
Google demonstrated the approach on a 10-minute generated video, with memory used to maintain visual continuity across distant scenes.
VQQA: automated quality refinement
Video Quality Question Answering (VQQA) evaluates generated video using a vision-language model.
The system:
- generates questions about the output;
- evaluates the video;
- produces natural-language critique;
- modifies the prompt;
- generates another candidate.
Google describes this feedback as semantic gradients.
VQQA also uses Global Selection, comparing candidates against the original prompt rather than automatically accepting the newest version.
Compared with previous pipelines
| Previous approach | New framework |
| Linear agent chain | Closed optimization loop |
| Independent shot generation | Shared global state |
| Text-only continuity | Persistent visual memory |
| Fixed creative settings | MAB-based strategy search |
| Errors accumulate downstream | Iterative evaluation and correction |
| Short-range context | Long-range multimodal memory |
| Latest output accepted | Global candidate selection |
The main architectural change is that consistency is moved from the video model alone into an external orchestration and memory layer.
Benchmarks
Google introduced several evaluation sets.
GenAD-Bench:
- 400 scenarios
- 50 fictional brands
- four products per brand
- Co-Director peak score: 81.4
HardContinuityBench:
- repeated characters;
- costume changes;
- accessories;
- changing object states.
LVBench-C:
- 120 scenarios;
- characters, objects and environments can disappear for at least 10 segments before returning.
The framework was also evaluated on ST-Bench, VBench-Long, T2V-CompBench, VBench2 and VBench-I2V.
Technical summary
The system adds four capabilities around existing video models:
- Orchestration: centralized control of story and creative parameters.
- Memory: persistent state for characters, objects and environments.
- Long-range generation: segment-level synthesis with retrieval of previous context.
- Optimization: MLLM-based evaluation and iterative prompt correction.
The key change is not a new video foundation model, but a multi-agent runtime that coordinates video generation, memory and evaluation across long sequences.
