Poster Sun, Jun 7, 2026 • 2:30 PM – 4:30 PM PDT ExHall A

Mixture of States: Routing Token-Level Dynamics for Multimodal Generation

Haozhe Liu ⋅ Ding Liu ⋅ Mingchen Zhuge ⋅ Zijian Zhou ⋅ Tian Xie ⋅ Sen He ⋅ Yukang Yang ⋅ Shuming Liu ⋅ Yuren Cong ⋅ Jiadong Guo ⋅ Hongyu Xu ⋅ Ke Xu ⋅ Kam-Woh Ng ⋅ Juan C. Perez ⋅ Juan-Manuel Pérez-Rúa ⋅ Tao Xiang ⋅ Wei Liu ⋅ Shikun Liu ⋅ Jürgen Schmidhuber

Abstract

We introduce MoS (Mixture of States), a novel fusion paradigm for multimodal diffusion models that merges modalities using flexible, state-based interactions. The core of MoS is a learnable, token-wise router that creates denoising timestep- and input-dependent interactions between modalities' hidden states, precisely aligning token-level features with the diffusion trajectory. This router sparsely selects the top-$k$ hidden states and is trained with an $\epsilon$-greedy strategy, efficiently selecting contextual features with minimal learnable parameters and negligible computational overhead. We validate our design with text-to-image generation (MoS-Image) and editing (MoS-Editing), which achieve state-of-the-art results. With only 3B to 5B parameters, our models match or surpass counterparts up to $4\times$ larger. These findings establish MoS as a flexible and compute-efficient paradigm for scaling multimodal diffusion models.