Information Bottleneck of Quantum Neural Networks

Occam’s Razor, as a principle of model selection, favors simpler models that achieve the same empirical error rate to more complex models. Structural risk minimization finds such optimal model with a structural search by minimizing a high probability bound on the true error rate, enjoying the trade-off between minimizing the empirical error rate and the generalization bound.Tishby and Zaslavsky [2015] suggested to quantify such trade-off via the theoretical framework of the Information Bottleneck theory. It claims that Deep Neural Networks learn a compressed bottleneck representation that preserves only the relevant information about the target label Y. The optimal compression-accuracy trade-off is characterized by the information-bottleneck function, providing a tight achievable lower bound on the representation length.An open quantum system is equivalent to a noisy neural network that can perform machine learning tasks with non-unitary evolution with potential power of entanglement and superposition. Therefore understanding the learnability of a quantum system is crucial in the noisy intermediate-scale quantum (NISQ) era. In this paper, we are generalizing the Information Bottleneck theory to quantum Hilbert space to characterize the learnability of a quantum system with quantum mutual information as the complexity measure of a quantum concept class and the lower bound of the prediction accuracy of a quantum hypothesis. We compared the training dynamics of quantum neural networks with the classical analog in the information plane and, through numerical simulations, demonstrated that the learned quantum representations are more expressive than classical representations.