Timezone: »

 
Spotlight
Learning Temporal Point Processes via Reinforcement Learning
Shuang Li · Shuai Xiao · Shixiang Zhu · Nan Du · Yao Xie · Le Song

Tue Dec 04 01:45 PM -- 01:50 PM (PST) @ Room 220 CD
in Tue Track 1 -- Session 2 »

Social goods, such as healthcare, smart city, and information networks, often produce ordered event data in continuous time. The generative processes of these event data can be very complex, requiring flexible models to capture their dynamics. Temporal point processes offer an elegant framework for modeling event data without discretizing the time. However, the existing maximum-likelihood-estimation (MLE) learning paradigm requires hand-crafting the intensity function beforehand and cannot directly monitor the goodness-of-fit of the estimated model in the process of training. To alleviate the risk of model-misspecification in MLE, we propose to generate samples from the generative model and monitor the quality of the samples in the process of training until the samples and the real data are indistinguishable. We take inspiration from reinforcement learning (RL) and treat the generation of each event as the action taken by a stochastic policy. We parameterize the policy as a flexible recurrent neural network and gradually improve the policy to mimic the observed event distribution. Since the reward function is unknown in this setting, we uncover an analytic and nonparametric form of the reward function using an inverse reinforcement learning formulation. This new RL framework allows us to derive an efficient policy gradient algorithm for learning flexible point process models, and we show that it performs well in both synthetic and real data.

Author Information

Shuang Li (Georgia Institute of Technology)
Shuai Xiao (Ant Financial)
Shixiang Zhu (Georgia Institute of Technology)
Nan Du (Google Brain)
Yao Xie (Georgia Institute of Technology)

Yao Xie is an Assistant Professor and Harold R. and Mary Anne Nash Early Career Professor in the H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, which she joined in 2013. She received her Ph.D. in Electrical Engineering (minor in Mathematics) from Stanford University in 2011, M.Sc. in Electrical and Computer Engineering from the University of Florida, and B.Sc. in Electrical Engineering and Computer Science from University of Science and Technology of China (USTC) . From 2012 to 2013, she was a Research Scientist at Duke University. Her research areas include statistics, signal processing, and machine learning, in providing theoretical foundation as well as developing computationally efficient and statistically powerful algorithms for big data in various applications such as sensor networks, imaging, and crime data analysis. She received the National Science Foundation CAREER Award in 2017 and her crime data analytics project received the Smart 50 Award at the Smart Cities Connect Conferences and Expo in 2018.

Le Song (Ant Financial & Georgia Institute of Technology)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors