Texture synthesis for realistic-looking virtual colonoscopy using mask-aware transformer

In virtual colonoscopy, computer vision techniques focus on depth estimation, photometric tracking, and simultaneous localization and mapping (SLAM). To narrow the domain gap between virtual and real colonoscopy data, it is necessary to utilize real-world data or employ realistic-looking virtual dataset. We introduce a texture synthesis and outpainting strategy using the Mask-aware-transformer. The proposed method crafts textures for the colon's inner mucosa by utilizing real colonoscopy dataset. The primary objective is to develop texture maps tailored for virtual colonoscopy. The proposed method provides an RGB-D dataset of synthesized textures for virtual colonoscopy, meeting requirements for realistic, controllable, and a variety of texture appearances. The proposed dataset leverages 9 video sequences, each generated from distinct colon models, accumulating a total of 14,120 frames, paired with ground truth depth.