Navigating crowded, partially occluded environments is an open challenge for mobile robots. In dense crowds, spatial occlusions are inevitable due to limited sensor field of view (FOV) and obstructing obstacles. Prior work shows the efficacy of using observed interactions between human agents to make inferences about potential obstacles in occluded spaces. The observed humans act as an additional sensor to complement traditional sensors that form the incomplete map. Extending this idea, we propose a deep reinforcement learning (RL) planner that incorporates occlusion-aware features to encourage proactive avoidance of occluded obstacles and agents. We empirically demonstrate that our RL framework successfully avoids collisions with occluded agents by extracting informative features from observed interactions. To the best of our knowledge, this is the first study to exploit social inference in crowds for collision avoidance of occluded dynamic agents.