M*: A Modular, Extensible, Serving System for Multimodal Models

We are entering a new era of *composite model architectures* that integrate diverse components such as vision encoders, language backbones, diffusion and flow heads, audio codecs, action generators, and world-model predictors. Such architectures underpin a broad class of multimodal models, including unified multimodal models, omni models, speech-language models, vision-language-action policies, and world models. However, existing model serving frameworks were built on narrow assumptions about model structure, making them ill-suited to accommodate this new architectural diversity. Here we present M*, a universal serving system for efficient serving of composite AI models. M* represents models as dataflow graphs, processing requests spanning diverse modalities and tasks as traversals over these graphs. The core insight is a modular abstraction that supports arbitrary composition of model components, flexible placement onto a physical cluster, and model-agnostic optimizations within a distributed runtime. We call this abstraction the *Walk Graph* and show how it can concisely capture composite models from a broad range of families. We instantiate M* on representative models and find that it achieves, on average, 30\% lower end-to-end latency than vLLM-Omni for text-to-image workloads on BAGEL, while delivering a lower real-time factor and higher throughput - by up to 15\% - for text-to-speech workloads on Qwen3-Omni. M* also outperforms the V-JEPA 2-AC rollout baseline for robotic planning by up to $12.5\times$. Thus, our work paves the road towards more efficient serving of complex models with minimal developer effort.