This work addresses the fundamental linear inverse problem in compressive sensing (CS) by introducing a new type of regularizing generative prior. Our proposed method utilizes ideas from classical dictionary-based CS to integrate a strong regularization towards sparse solutions. At the same time, by leveraging the notion of conditional Gaussianity, it also incorporates the adaptability to training data and is able to learn from a few compressed and noisy data samples. We support our approach theoretically through the concept of variational inference and validate it empirically using different types of compressible signals.