Adapting World Models with Latent-State Dynamics Residuals

JB Lanier, Kyungmin Kim, Armin Karamzade, Yifei Liu, Ankita Sinha, Kathleen He, Davide Corsi, and Roy Fox

8th Annual Learning for Dynamics and Control Conference (L4DC), 2026

BibTeX

@conference{ Lanier2026ReDRAW,
  Title = "Adapting World Models with Latent-State Dynamics Residuals",
  Author = "JB Lanier and Kyungmin Kim and Armin Karamzade and Yifei Liu and Ankita Sinha and Kathleen He and Davide Corsi and Roy Fox",
  Booktitle = "8th Annual Learning for Dynamics and Control Conference (L4DC)",
  Year = "2026"
}

Simulation-to-reality reinforcement learning (RL) faces the challenge of reconciling discrepancies between simulated and real-world dynamics, which can degrade agent performance. When real data is scarce, a promising approach involves learning corrections to simulator forward dynamics represented as a residual error function; however, this operation is impractical with high-dimensional states such as images. To overcome this, we propose ReDRAW, a latent-state autoregressive world model pretrained in simulation and calibrated to a target environment through residual corrections of latent-state dynamics rather than of explicit observed states. Using this adapted world model, ReDRAW enables RL agents to be optimized with imagined rollouts under corrected dynamics and then deployed in the real world. In multiple vision-based DeepMind Control Suite domains and a physical robot visual lane-following task, ReDRAW effectively models changes to dynamics and avoids overfitting in low data regimes where traditional transfer methods fail.