The efficient communication of information has enormous societal and environmental impact, and stands to benefit from the machine learning revolution seen in other fields. Through this tutorial, we hope to disseminate the ideas of information theory and compression to a broad audience, overview the core methodologies in learning-based compression (i.e., neural compression), and present the relevant technical challenges and open problems defining a new frontier of probabilistic machine learning. Besides covering the technical grounds, we will also explore the broader underlying themes and future research in our panel discussion, focusing on the interplay between computation and communication, the role of machine learning, and societal considerations such as fairness, privacy, and energy footprint as we strive to make our learning and information processing systems more efficient.