AI Applications to Illustrate Native American Arts: Birdsongs: Using Transfer Learning to Augment Image Generation Models

Background: Situated approximately 40 miles northeast of San Diego and 30 miles inland from the Pacific Ocean, the Reservation of the Pala Band of Mission Indians is home to 1250 enrolled members– consisting of Cupeños and Luiseños. A vast 20 square miles of valley surrounded by mountains along the San Luis Rey River, the Pala Reservation is comprised of residential and agricultural areas as well as unused wildlands.

The San Diego Supercomputer Center (SDSC) is located on the University of California San Diego campus. Often collaborating with the Pala Band of Mission Indians for educational projects, Senior Science Writer Kimberly Mann Bruch of SDSC and Senior Diana Duro of Pala most recently led a team that used artificial intelligence (AI) tools in an effort to augment image generation models to appropriately represent Native American birdsinging.

Summary: The Native American birdsong represents a sacred, traditional performing art that consists of a rhythmic song regarding an essential life lesson – accompanied by handmade gourd rattles. Unfortunately, the word “birdsong” is grossly misrepresented across an array of technology tools – ranging from search engines to artificial intelligence (AI) imagery models. To augment these models, the team utilized transfer learning to “teach” an example model how to better represent the terms “birdsong”, “birdsinging”, “gourd rattle”, and “rattle”. First, the team obtained images that represented “gourd rattle” and input them into a dataset. Next, the dataset was placed into an existing image generation model. Unfortunately, even with the proper image and description input to the existing model, time and time again, the search did not yield “gourd rattle” or “rattle” upon search. Instead, it most often described the “gourd rattle” as “maraca”, which is similar, but incorrect. Next, the team repeated this activity with “birdsong” and “birdsinging” – the results were the same. That is, the model was unable to ”learn” the terms “birdsong” or “birdsinging”.

Future Work: The team plans to continue working on modifying the models to remedy the issue and then use lessons learned for additional terms.

Funding: The project was funded by the National Science Foundation West Big Data Innovation Hub (1916481) with support also provided by the San Diego Supercomputer Center at UC San Diego and the Pala Band of Mission Indians.