text2pixelart

This project explores opportunities for an unsupervised generation of animated pixel-art based on the given text prompt. As the base, I used PixelDraw by @dribnet, based on the ClipDraw library, which in turn is based on the diffvg library. First, I've implemented some handy options, such as smoothness enforcement, palette enforcement, and saturation enforcement. Next, I've exploited the CLIP's "healing" ability and a couple of "demoscene" tricks to keep the coherence between adjacent frames to make several different animations: the infinite panorama, the parallax effect, the 3d swirl around the object, the moving background, the endless looping, and so on. I've tried to give a tribute to old-school classics (using prompts like [fallout], [r-type], and [Guybrush Threepwood]) and to explore the fractal and space thematics, trying to find the best match between the prompt and the effect in each case. The code of some tricks is published already; other hacks are on their way to be publicly available. Web link: https://altsoph.github.io/text2pixelart/

Full video can be downloaded here: https://www.dropbox.com/s/r88x7ze1wy8wjo5/text2pixelartclipv2.mp4?dl=0