Timezone: »
In a wide variety of applications, humans interact with a complex environment by means of asynchronous stochastic discrete events in continuous time. Can we design online interventions that will help humans achieve certain goals in such asynchronous setting? In this paper, we address the above problem from the perspective of deep reinforcement learning of marked temporal point processes, where both the actions taken by an agent and the feedback it receives from the environment are asynchronous stochastic discrete events characterized using marked temporal point processes. In doing so, we define the agent's policy using the intensity and mark distribution of the corresponding process and then derive a flexible policy gradient method, which embeds the agent's actions and the feedback it receives into real-valued vectors using deep recurrent neural networks. Our method does not make any assumptions on the functional form of the intensity and mark distribution of the feedback and it allows for arbitrarily complex reward functions. We apply our methodology to two different applications in viral marketing and personalized teaching and, using data gathered from Twitter and Duolingo, we show that it may be able to find interventions to help marketers and learners achieve their goals more effectively than alternatives.
Author Information
Utkarsh Upadhyay (Max Plank Institute for Software Systems)
Bringing clarity to public discourse.
Abir De (Max Planck Insitute for Software Systems)
Manuel Gomez Rodriguez (Max Planck Institute for Software Systems)
More from the Same Authors
-
2021 : Reinforcement Learning Under Algorithmic Triage »
Eleni Straitouri · Adish Singla · Vahid Balazadeh Meresht · Manuel Rodriguez -
2022 Poster: Counterfactual Temporal Point Processes »
Kimia Noorbakhsh · Manuel Rodriguez -
2023 Poster: Locality Sensitive Hashing in Fourier Frequency Domain For Soft Set Containment Search »
Indradyumna Roy · Rishi Agarwal · Soumen Chakrabarti · Anirban Dasgupta · Abir De -
2023 Poster: Learning to Select a Subset of Training Examples to Generalize Efficient Model Training »
Eeshaan Jain · Tushar Nandy · Gaurav Aggarwal · Ashish Tendulkar · Rishabh Iyer · Abir De -
2023 Poster: Finding Counterfactually Optimal Action Sequences in Continuous State Spaces »
Stratis Tsirtsis · Manuel Rodriguez -
2023 Poster: Human-Aligned Calibration for AI-Assisted Decision Making »
Nina Corvelo Benz · Manuel Rodriguez -
2022 Spotlight: Neural Estimation of Submodular Functions with Applications to Differentiable Subset Selection »
Abir De · Soumen Chakrabarti -
2022 Spotlight: Lightning Talks 1A-3 »
Kimia Noorbakhsh · Ronan Perry · Qi Lyu · Jiawei Jiang · Christian Toth · Olivier Jeunen · Xin Liu · Yuan Cheng · Lei Li · Manuel Rodriguez · Julius von Kügelgen · Lars Lorch · Nicolas Donati · Lukas Burkhalter · Xiao Fu · Zhongdao Wang · Songtao Feng · Ciarán Gilligan-Lee · Rishabh Mehrotra · Fangcheng Fu · Jing Yang · Bernhard Schölkopf · Ya-Li Li · Christian Knoll · Maks Ovsjanikov · Andreas Krause · Shengjin Wang · Hong Zhang · Mounia Lalmas · Bolin Ding · Bo Du · Yingbin Liang · Franz Pernkopf · Robert Peharz · Anwar Hithnawi · Julius von Kügelgen · Bo Li · Ce Zhang -
2022 Spotlight: Maximum Common Subgraph Guided Graph Retrieval: Late and Early Interaction Networks »
Indradyumna Roy · Soumen Chakrabarti · Abir De -
2022 Spotlight: Counterfactual Temporal Point Processes »
Kimia Noorbakhsh · Manuel Rodriguez -
2022 Spotlight: Learning Recourse on Instance Environment to Enhance Prediction Accuracy »
Lokesh N · Guntakanti Sai Koushik · Abir De · Sunita Sarawagi -
2022 Poster: Neural Estimation of Submodular Functions with Applications to Differentiable Subset Selection »
Abir De · Soumen Chakrabarti -
2022 Poster: Learning Recourse on Instance Environment to Enhance Prediction Accuracy »
Lokesh N · Guntakanti Sai Koushik · Abir De · Sunita Sarawagi -
2022 Poster: Maximum Common Subgraph Guided Graph Retrieval: Late and Early Interaction Networks »
Indradyumna Roy · Soumen Chakrabarti · Abir De -
2021 Workshop: Human Centered AI »
Michael Muller · Plamen P Angelov · Shion Guha · Marina Kogan · Gina Neff · Nuria Oliver · Manuel Rodriguez · Adrian Weller -
2021 Poster: Learning to Select Exogenous Events for Marked Temporal Point Process »
Ping Zhang · Rishabh Iyer · Ashish Tendulkar · Gaurav Aggarwal · Abir De -
2021 Poster: Differentiable Learning Under Triage »
Nastaran Okati · Abir De · Manuel Rodriguez -
2021 Poster: Training for the Future: A Simple Gradient Interpolation Loss to Generalize Along Time »
Anshul Nasery · Soumyadeep Thakur · Vihari Piratla · Abir De · Sunita Sarawagi -
2021 Poster: Counterfactual Explanations in Sequential Decision Making Under Uncertainty »
Stratis Tsirtsis · Abir De · Manuel Rodriguez -
2020 Poster: Decisions, Counterfactual Explanations and Strategic Behavior »
Stratis Tsirtsis · Manuel Gomez Rodriguez -
2019 : Poster Session »
Ayse Cakmak · Yunkai Zhang · Srijith Prabhakarannair Kusumam · Mohamed Osama Ahmed · Xintao Wu · Jayesh Choudhari · David I Inouye · Thomas Taylor · Michel Besserve · Ali Caner Turkmen · Kazi Islam · Antonio Artés · Amrith Setlur · Zhanghua Fu · Zhen Han · Abir De · Nan Du · Pablo Sanchez-Martin -
2019 Workshop: Learning with Temporal Point Processes »
Manuel Rodriguez · Le Song · Isabel Valera · Yan Liu · Abir De · Hongyuan Zha -
2019 Workshop: Workshop on Human-Centric Machine Learning »
Plamen P Angelov · Nuria Oliver · Adrian Weller · Manuel Rodriguez · Isabel Valera · Silvia Chiappa · Hoda Heidari · Niki Kilbertus -
2019 Poster: Teaching Multiple Concepts to a Forgetful Learner »
Anette Hunziker · Yuxin Chen · Oisin Mac Aodha · Manuel Gomez Rodriguez · Andreas Krause · Pietro Perona · Yisong Yue · Adish Singla -
2018 : Manuel Gomez Rodriguez - Enhancing the Accuracy and Fairness of Human Decision Making »
Manuel Rodriguez -
2018 Poster: Enhancing the Accuracy and Fairness of Human Decision Making »
Isabel Valera · Adish Singla · Manuel Gomez Rodriguez -
2017 Poster: From Parity to Preference-based Notions of Fairness in Classification »
Muhammad Bilal Zafar · Isabel Valera · Manuel Rodriguez · Krishna Gummadi · Adrian Weller -
2016 Poster: Learning and Forecasting Opinion Dynamics in Social Networks »
Abir De · Isabel Valera · Niloy Ganguly · Sourangshu Bhattacharya · Manuel Gomez Rodriguez -
2015 Poster: COEVOLVE: A Joint Point Process Model for Information Diffusion and Network Co-evolution »
Mehrdad Farajtabar · Yichen Wang · Manuel Rodriguez · Shuang Li · Hongyuan Zha · Le Song -
2015 Oral: COEVOLVE: A Joint Point Process Model for Information Diffusion and Network Co-evolution »
Mehrdad Farajtabar · Yichen Wang · Manuel Rodriguez · Shuang Li · Hongyuan Zha · Le Song -
2014 Poster: Shaping Social Activity by Incentivizing Users »
Mehrdad Farajtabar · Nan Du · Manuel Gomez Rodriguez · Isabel Valera · Hongyuan Zha · Le Song -
2013 Poster: Scalable Influence Estimation in Continuous-Time Diffusion Networks »
Nan Du · Le Song · Manuel Gomez Rodriguez · Hongyuan Zha -
2013 Oral: Scalable Influence Estimation in Continuous-Time Diffusion Networks »
Nan Du · Le Song · Manuel Gomez Rodriguez · Hongyuan Zha