Welcome and introduction presented by Mattie Tesfaldet.

Will be held in Discord.

Pre-recorded slide show of the art submissions

Mark, Devi, and Kenric on live panel answering questions, moderated by Mattie Tesfaldet. Please ask questions on Rocket.chat, not Discord.

Will be held in Discord.

Generating images that fit a given text description using machine learning has improved greatly with the release of technologies such as the CLIP image-text encoder model; however, current methods lack artistic control of the style of image to be generated. We introduce StyleCLIPDraw which adds a style loss to the CLIPDraw text-to-drawing synthesis model to allow artistic control of the synthesized drawings in addition to control of the content via text. Whereas performing decoupled style transfer on a generated image only affects the texture, our proposed coupled approach is able to capture a style in both texture and shape, suggesting that the style of the drawing is coupled with the drawing process itself. More results and our code are available at https://github.com/pschaldenbrand/StyleCLIPDraw

In the art of video editing, sound is really half the story. A skilled video editor overlays sounds, such as effects and ambients, over footage to add character to an object or immerse the viewer within a space. However, through formative interviews with professional video editors, we found that this process can be extremely tedious and time-consuming. We introduce Soundify, a system that matches sound effects to video. By leveraging labeled, studio-quality sound effects libraries and extending CLIP, a neural network with impressive zero-shot image classification capabilities, into a "zero-shot detector", we are able to produce high-quality results without resource-intensive correspondence learning or audio generation. We encourage you to have a look at, or better yet, have a listen to the results at https://chuanenlin.com/soundify.

We propose a singing decomposition system that encodes time-aligned linguistic content, pitch, and source speaker identity via Assem-VC. With decomposed speaker-independent information and the target speaker's embedding, we could synthesize the singing voice of the target speaker. In conclusion, we made a perfectly synced duet with the user's singing voice and the target singer's converted singing voice.

Fictional languages have become increasingly popular over the recent years appearing in novels, movies, TV shows, comics, and video games. While some of these fictional languages have a complete vocabulary, most do not. We propose a deep learning solution to the problem. Using style transfer and machine translation tools, we generate new words for a given target fictional language, while maintaining the style of its creator, hence extending this language vocabulary.

Famous poem, illustrated with the automated text-to-video synthesis tool. Rendered with CLIP-FFT method, optimizing frequencies of Fourier inverse transformation.

voiceover: Benedict Cumberbatch pretrained ML model: github.com/openai/CLIP

I began by feeding thousands of images of my past artwork into a GAN. As a ceramic artist, the source imagery were photographs of sculptures and pottery I made between 2015-2019. The GAN required me to augment the data in multiple ways, including photographing my work from all sides. My vision was that the GAN would "imagine" my future work based on past work. However, the GAN was limited in ways that made this unfeasible. It was trapped within my prior work and could not imagine the new. I developed a collaboration with the machine where the shortcomings of the GAN become generative of new form. I prompted the GAN with my past work, and I interpreted its output as a prompt to either create the unknown half or interpret the output in 3 dimensions. It was within this unknown space that a void was created which offered the opportunity for novelty. As the GAN navigated this void, attempting to create new form from old, I also navigated this void. I embrace and utilize these digital shortcomings as a marking of time within technology and my own art practice.

The images generated in this project use the VQGAN+CLIP Script created by Katherine Crowson (2021), licensed under the MIT license. The model has been modified to create new designs of high heel shoes using a text prompt. The text prompt can be as simple as one describing word to redesign the entire shoe. The text prompt is manipulating an image of a high heel shoe , which is applied as an ‘init-image’. This tool can be used to create hundreds of original designs, whether you are a designer or not. Find more of Sophia's work at; https://www.sophianeilldesign.com

This project explores opportunities for an unsupervised generation of animated pixel-art based on the given text prompt. As the base, I used PixelDraw by @dribnet, based on the ClipDraw library, which in turn is based on the diffvg library. First, I've implemented some handy options, such as smoothness enforcement, palette enforcement, and saturation enforcement. Next, I've exploited the CLIP's "healing" ability and a couple of "demoscene" tricks to keep the coherence between adjacent frames to make several different animations: the infinite panorama, the parallax effect, the 3d swirl around the object, the moving background, the endless looping, and so on. I've tried to give a tribute to old-school classics (using prompts like [fallout], [r-type], and [Guybrush Threepwood]) and to explore the fractal and space thematics, trying to find the best match between the prompt and the effect in each case. The code of some tricks is published already; other hacks are on their way to be publicly available. Web link: https://altsoph.github.io/text2pixelart/

Full video can be downloaded here: https://www.dropbox.com/s/r88x7ze1wy8wjo5/text2pixelartclipv2.mp4?dl=0

Will be held in Discord.

Joel and Hypereikon on live panel answering questions, moderated by David Ha. Please ask questions on Rocket.chat, not Discord.

Closing remarks presented by Tom White.

Will be held in Discord.