NIPS 2018 Expo Talk

Dec. 6, 2021

Expo 2018 Schedule »

Image Retrieval and Video Editing pipeliens in Media Companies

Sponsor: Hearst


Hearst is a large privately held media company, owning magazines such as Elle, Cosmo, Esquire, Road&Track, Delish, etc, Hearst also owns newspapers (sfgate, chron, seattle pi, etc) and 19 local broadcast TV stations.

The challenge we face is image retrieval and image re-use, along with very labor intensive video editing. We are solving these problems by various deep learning based methods.

  1. in 2016 - we mapped a subset of the text within our magazine articles to their associated images for the purpose of retrieval. (prototype). We are currently scaling this to all magazines articles. (Magazines: Elle, Cosmopolitan, Esquire, etc.)
  2. We've also had a large dataset of newspaper articles (e.g. sfgate, chron) along with its associated large image database . We've successfully built interactive tools to custom tag photos at scale. We're employing similar tools to retrieval of people in images. (almost complete)
  3. For Video, we are building deep learning based tools to help editorial of videos, e.g. raw to finished. (underway, no results yet, hoping for results ... this is joint work with Shih-Fu Chang at Columbia, that collaboration may or may not be complete by the time NIPS comes around)

The talk would review the challenges and successes we have had. I'm am hoping the projects are completed by early November.