Timezone: »

 
Poster
On the Stochastic Stability of Deep Markov Models
Jan Drgona · Sayak Mukherjee · Jiaxin Zhang · Frank Liu · Mahantesh Halappanavar

Wed Dec 08 04:30 PM -- 06:00 PM (PST) @ Virtual
in Poster Session 4 »

Deep Markov models (DMM) are generative models which are scalable and expressive generalization of Markov models for representation, learning, and inference problems. However, the fundamental stochastic stability guarantees of such models have not been thoroughly investigated. In this paper, we present a novel stability analysis method and provide sufficient conditions of DMM's stochastic stability. The proposed stability analysis is based on the contraction of probabilistic maps modeled by deep neural networks. We make connections between the spectral properties of neural network's weights and different types of used activation function on the stability and overall dynamic behavior of DMMs with Gaussian distributions. Based on the theory, we propose a few practical methods for designing constrained DMMs with guaranteed stability. We empirically substantiate our theoretical results via intuitive numerical experiments using the proposed stability constraints.

Keywords: Theory · Deep Learning · Generative Model

Author Information

Jan Drgona (Pacific Northwest National Laboratory)

I am a data scientist in the Physics and Computational Sciences Division (PCSD) at Pacific Northwest National Laboratory, Richland, WA. My current research interests fall in the intersection of model-based optimal control, constrained optimization, and machine learning.

Sayak Mukherjee (Pacific Northwest National Laboratory)
Jiaxin Zhang (Oak Ridge National Laboratory)

I am now a Research Staff in Machine Learning and Data Analytics Group, Computer Science and Mathematics Division at Oak Ridge National Laboratory (ORNL). My current research interest is on Artificial Intelligence for Science and Engineering (AISE). My broad interests revolve around robust machine learning, uncertainty quantification, inverse problems, and numerical optimization.

Frank Liu (Oak Ridge National Lab)
Mahantesh Halappanavar (Pacific Northwest National Laboratory)

More from the Same Authors