3D Gaussian Splatting (3DGS) has become a crucial method for acquiring 3D assets. To protect the copyright of these assets, digital watermarking techniques can be applied to embed ownership information discreetly within 3DGS models. However, current watermarking methods for mesh, point cloud, and implicit radiance fields can not be directly used for the 3DGS models, which utilize the explicit 3D Gaussians with the associated features without using any neural networks. Naively embedding the watermark on a pre-trained 3DGS can cause obvious distortion in the rendered images, thus affecting the visual quality. In our work, we propose an uncertainty-based method that constrains the perturbation of model parameters to achieve indivisible watermarking for 3DGS. At the message decoding stage, the copyright messages can be reliably extracted from both the 3D Gaussians and the 2D rendered images via robust decoders even under various forms of 3D and 2D distortions. We conduct extensive experiments on the Blender, LLFF and Mip-360 datasets to validate the effectiveness of our proposed method, demonstrating \textit{state-of-the-art} performance on both message decoding accuracy and view synthesis quality.