The risk of hip fracture is predicted from dual-energy X-ray absorptiometry (DXA) images using deep learning and over 10,000 exams from the HealthABC longitudinal study. The approach is evaluated in four different clinical scenarios of increasing diagnostic intensity. In the scenario with the most information available, deep learning achieves an area under the ROC curve (AUC) of 0.75 on a held-out test set, while a standard linear model that relies on feature-engineering achieves an AUC of 0.72.