A world model is a learned simulator. It captures environment dynamics so an agent can query imagined futures without repeatedly stepping the original environment.

The practical attraction is that the learned model is differentiable, GPU-friendly, batched, and reusable for planning, search, policy optimization, and cheap environment rollouts.

The article focuses on RSSM because implementing its internal state bookkeeping is easy to misunderstand: observations, latent states, deterministic hidden states, priors, and posteriors all play different roles.

RSSM separates the environment observation from the model's internal state. The observation is what the environment exposes; the recurrent model maintains a hidden summary and samples a stochastic latent state.

The model has two information sources. From previous hidden state, previous latent state, and action, it predicts a prior over the next latent state. If the next observation is available, an encoder provides extra evidence for a posterior.

This differs from ordinary language modeling. In reinforcement learning, using the current observation to infer the current latent state is legitimate because observations are partial evidence about a larger hidden state.

Training rolls the model forward through recorded trajectories of observations and actions. At each step, the posterior sample is decoded back into the next observation.

The reconstruction loss trains the latent-to-observation pathway. The KL loss pressures the prior to match the posterior even though the prior has less information.

After enough training, the prior should become useful for imagination: the agent can stop receiving observations and continue rolling forward in latent space.

For low-dimensional vector observations, MLP encoders and decoders are straightforward. Pixel observations are harder because every rollout step contributes an image reconstruction objective.

The article compares three strategies: a separate external autoencoder, a fully internal encoder-decoder, and a frozen externally trained encoder-decoder used inside the RSSM loop.

The central subtlety is that modeling latent dynamics and reconstructing pixels are not the same objective. A model may understand dynamics while still producing blurry or misleading reconstructions.

On MuJoCo HalfCheetah, the frozen external encoder-decoder strategy worked best. The cheetah body is visually large and tightly coupled to the dynamics, so reconstruction and dynamics are more aligned.

A surprising result is that rollout quality did not necessarily degrade step by step. Reconstruction fidelity stayed relatively constant because latent dynamics can remain stable even when decoded images are imperfect.

Atari exposed the weakness of pixel reconstruction. Important state variables such as the ball, paddle, or ghosts occupy very few pixels, so a reconstruction loss can ignore them while still looking good numerically.

The article suggests richer losses or generative heads as possible fixes, because the decoder should be encouraged to produce observations that lie within the distribution of plausible frames.

A direct transformer replacement is not trivial because RSSM's prior and posterior are intertwined. The prior needs a sequence of latent states, but those latent states usually come from the posterior.

TSSM resolves this by separating the posterior and prior graphs. The posterior is computed from current observations, and the transformer consumes the sampled latent-action sequence to predict priors.

This simplification assumes the current observation is enough to form the posterior without recurrent hidden dynamics. That can be reasonable in many environments but is not universally guaranteed.

For a lifelong agent, a world model is not just a predictor. It is a reusable imagination substrate: memory becomes a dynamical object that can be queried, rolled forward, and differentiated through.

The key engineering lesson is to separate state estimation from observation reconstruction. A capable agent should learn compact latent dynamics while still preserving enough detail for action-relevant prediction.

RSSM favors online recurrent filtering. TSSM favors parallel sequence modeling. The choice maps directly onto the agent's runtime needs: streaming control versus large-batch offline learning.