Abstract: Generative modelling paradigms based on denoising diffusion processes have emerged as a leading candidate for \textit{conditional} sampling in inverse problems. In many real-world applications, we often have access to large, expensively trained unconditional diffusion models, which we aim to exploit for improving conditional sampling. Most recent approaches are motivated heuristically and lack a unifying framework, obscuring connections between them. Further, they often suffer from issues such as being very sensitive to hyperparameters, being expensive to train or needing access to weights hidden behind a closed API. Under this framework, we propose DEFT \textit{(Doob's h-transform Efficient FineTuning)}, a new approach for conditional generation that simply fine-tunes a very small network to quickly learn the conditional $h$-transform, while keeping the larger unconditional network unchanged. DEFT is much faster than existing baselines, while achieving state-of-the-art performance across a variety of linear and non-linear benchmarks. On \emph{image reconstruction} tasks, we achieve speedups of up to 1.6$\times$, while having the best perceptual quality on natural images and reconstruction performance on medical images.