Towards Understanding LLM Latent Inference through Fuzzy Reasoning Dynamics Modeling

Large language models (LLMs) progressively transform internal representations across layers for inference, yet existing analyses provide limited insight into which reasoning-related operations drive each layer transition or whether multiple operations may coexist. Moreover, many operate directly in the high-dimensional latent space, leading to substantial computational complexity. This paper introduces Fuzzy Reasoning Dynamics Modeling (FRDM), which formulates layer-wise inference as a low-dimensional fuzzy dynamical process composed of semantically meaningful reasoning operations. Specifically, FRDM employs compact state descriptors and transition indicators, instantiates semantically grounded local dynamics, and optimizes their learnable fuzzy memberships to reconstruct each layer transition. As such, FRDM reduces the complexity of modeling high-dimensional latent representations, provides operation-level interpretation of layer transitions, and naturally captures the coexistence and varying contributions of multiple reasoning dynamics. Empirically, FRDM consistently outperforms baselines in transition tracking across different benchmarks, achieving the highest $R^2$ scores. The learned fuzzy memberships further improve interpretability by revealing layer-wise patterns associated with distinct reasoning operations, providing an interpretable view of how reasoning evolves across layers.