Under-Counted Tensor Completion with Neural Network-based Side Information Learner

Under-counted data often arise in disciplines such as ecology and epidemiology. For example, in epidemiology, the cases of an infectious disease (e.g., the COVID-19) may always be under-counted due to the existence of symptom-free patients and the lack of testing. Estimating the unobserved true counts from the under-counted data is therefore a well-motivated task. A recent work has addressed the under-counting effect in matrix-type count data by employing a Poisson-Binomial matrix completion model. The model also learns the probability of detecting a count via a linear function of some side information. The model was shown effective for couple of matrix completion tasks. Nonetheless, there exists a number of challenges. First, the model cannot directly handle under-counted data represented by more than two aspects. Second, the linear function incorporated in the model is not general enough to capture any unknown nonlinear relationship that may occur between the side information and the probability of detection. Third, there is no theoretical understanding to the properties of such a Poisson-Binomial model for handling under-counted data. In this work, we address these aforementioned challenges by proposing a tensor-completion based framework. Our model is based low rank Poisson tensor decomposition combined with a nonlinear function modeling for the probability of detection. To learn the model parameters, we design a joint low-rank tensor completion and neural network learning algorithm. Furthermore, we derive theoretical conditions under which the model parameters can be recovered, leveraging the low-rank tensor structure and the similarity of the detection probabilities. Simulations and real-data experiments support our theoretical claims.