Physics-Informed Video Generation via Mixture-of-Experts Latent Alignment

Abstract

Large-scale video generation models have made remarkable progress in semantic consistency and visual quality, producing videos that are increasingly coherent and visually convincing. Nevertheless, the dynamics induced by pixel-level fitting do not naturally accommodate the regularities that govern real-world motion and interaction, resulting in persistent shortcomings in physical plausibility. To address this limitation, we propose PILA (Physics-Informed Latent Alignment), a framework that injects physics-structured latent guidance into the frozen flow-matching dynamics of pretrained video models. Specifically, PILA first employs anchored field estimation to map frozen-generator latents into an operational physical attribute bank organized by field-proxy slots, using observable motion as a kinematic anchor for constructing less directly observed proxies. To handle the heterogeneity of real-world dynamics, PILA adopts a mixture-of-experts design over physical categories. Label-prior masked expert routing selects category-specific operator experts, whose refinements are regularized by operational residuals abstracted from physical relations. Finally, the refined proxies are fused into the physical attribute bank and decoded into a correction to the flow-matching vector field, injecting physics-aware guidance while preserving the visual prior of the pretrained backbone. With staged adapter training on Wan 2.1-1.3B and direct transfer of the learned adapter to Wan 2.2-14B, PILA achieves state-of-the-art results on VBench-2.0, VideoPhy-2, and PhyGenBench in both visual quality and benchmark-measured physical plausibility.

Framework Overview

Pipeline of PILA
Pipeline of PILA. The method proceeds through three stages: (1) operational physical interface encoding and label-prior expert routing; (2) physics-structured operator refinement of category-specific field-proxy representations; and (3) physics-to-latent flow alignment through a lightweight correction to the frozen flow-matching vector field.

Video Comparison Gallery

A nacho chip is dipped into a bowl of salsa.

Ours-14B

Ours-1.3B

Wan 2.2-14B

WISA

Hunyuan

Wan 2.1-1.3B

VideoREPA

CogVideoX

The sharp knife severs the fresh loaf of bread.

Ours-14B

Ours-1.3B

Wan 2.2-14B

WISA

Hunyuan

Wan 2.1-1.3B

VideoREPA

CogVideoX

A pebble splashes into the serene pond water.

Ours-14B

Ours-1.3B

Wan 2.2-14B

WISA

Hunyuan

Wan 2.1-1.3B

VideoREPA

CogVideoX

A can of soda is slowly poured out in the space station, releasing the liquid into the surrounding area.

Ours-14B

Ours-1.3B

Wan 2.2-14B

WISA

Hunyuan

Wan 2.1-1.3B

VideoREPA

CogVideoX

A focused, bright beam of light generated by a car headlight is passing through a jar of dusty air, creating a mesmerizing display as the light interacts with the dust particles, casting intricate patterns and subtle hues within the air.

Ours-14B

Ours-1.3B

Wan 2.2-14B

WISA

Hunyuan

Wan 2.1-1.3B

VideoREPA

CogVideoX

A timelapse captures the transformation of water in a pot as the temperature rapidly rises above 100°C.

Ours-14B

Ours-1.3B

Wan 2.2-14B

WISA

Hunyuan

Wan 2.1-1.3B

VideoREPA

CogVideoX

Equal amounts of red and blue paint are rapidly combined, with the mixture being vigorously stirred until fully blended.

Ours-14B

Ours-1.3B

Wan 2.2-14B

WISA

Hunyuan

Wan 2.1-1.3B

VideoREPA

CogVideoX

A piece of copper is ignited, emitting a vivid and unique flame as it burns steadily.

Ours-14B

Ours-1.3B

Wan 2.2-14B

WISA

Hunyuan

Wan 2.1-1.3B

VideoREPA

CogVideoX

A small burning ball of paper was thrown into a pile of dry paper.

Ours-14B

Ours-1.3B

Wan 2.2-14B

WISA

Hunyuan

Wan 2.1-1.3B

VideoREPA

CogVideoX

A timelapse captures the transformation as hydrochloric acid is poured into a glass of purple litmus solution.

Ours-14B

Ours-1.3B

Wan 2.2-14B

WISA

Hunyuan

Wan 2.1-1.3B

VideoREPA

CogVideoX

A timelapse video documents the gradual decay of a polished metal railing exposed to the elements over years.

Ours-14B

Ours-1.3B

Wan 2.2-14B

WISA

Hunyuan

Wan 2.1-1.3B

VideoREPA

CogVideoX

A magnifying glass is gradually moving closer to the petals of a flower, revealing the intricate details and textures of the flower as it approaches.

Ours-14B

Ours-1.3B

Wan 2.2-14B

WISA

Hunyuan

Wan 2.1-1.3B

VideoREPA

CogVideoX

More Results