Expo Demonstration
La Nouvelle Orleans Ballroom A-C (level 2)

We propose a real-time portrait video relighting system given an instantly AI-generated HDR map from a text prompt. Our techniques include HDRI map generation, relighting, and video segmentation all running on Qualcomm’s AI Engine in Real-Time. To do this, we extend Stable360, a text-to-360° image generation model, to high dynamic range prediction and generate 360° radiance maps from text prompts. Then, we develop a relighting pipeline on the segmented foreground that combines a lightweight normal estimation network with a novel rendering equation to produce realistic lighting effects. Live demo running on Qualcomm Mobile device shows that our framework can bring visually convincing lighting effects while preserving temporal consistency. It allows a subject to be naturally embedded into AI-generated scene, which could be used in many video chat applications.

Chat is not available.