Muse

Muse is a generative World and Human Action Model (WHAM) from Microsoft Research trained on more than one billion images and controller actions drawn from roughly seven years of continuous gameplay of Bleeding Edge. The model jointly generates visual continuations of a game state and the corresponding controller actions, capturing the relationship between game state, visual dynamics, and player agency in a constrained but rich environment. The WHAM Demonstrator on Microsoft Foundry lets creators feed in a starting state and explore, branch, and edit alternative continuations of gameplay.

Muse represents a frontier of generative AI that extends beyond static text or image generation into temporal, interactive dynamics — a precursor to true world models. By learning latent representations of game physics, agent behavior, and environmental response from massive-scale gameplay, it offers insight into how generative models can encode action-conditional dynamics. The work, published in Nature, informs broader directions in game design, procedural content, robotics simulation, and embodied AI, where understanding “what happens next given an action” is the core challenge.