Differential privacy is the dominant standard for formal and quantifiable privacy and has been used in major deployments that impact millions of people. Many differentially private algorithms for query release and synthetic data contain steps that reconstruct answers to queries from answers to other queries measured by the mechanism. Reconstruction is an important subproblem for such mechanisms to economize the privacy budget, minimize error on reconstructed answers, and allow for scalability to high-dimensional datasets. In this paper, we introduce a principled and efficient postprocessing method ReM (Residuals-to-Marginals) for reconstructing answers to marginal queries under Gaussian noise. Our method builds on recent work on efficient mechanisms for marginal query release, based on making measurements using a residual query basis that admits efficient pseudoinversion, which is an important primitive used in reconstruction. An extension ReM-LNN (Residuals-to-Marginals with Local Non-negativity) reconstructs marginals satisfying non-negativity and consistency which often reduces error on reconstructed answers. We demonstrate the utility of ReM and ReM-LNN by applying them to improve existing private query answering mechanisms: ResidualPlanner and MWEM.