Timezone: »
The simple, elegant approach of training convolutional neural networks (CNNs) directly from RGB pixels has enjoyed overwhelming empirical success. But can more performance be squeezed out of networks by using different input representations? In this paper we propose and explore a simple idea: train CNNs directly on the blockwise discrete cosine transform (DCT) coefficients computed and available in the middle of the JPEG codec. Intuitively, when processing JPEG images using CNNs, it seems unnecessary to decompress a blockwise frequency representation to an expanded pixel representation, shuffle it from CPU to GPU, and then process it with a CNN that will learn something similar to a transform back to frequency representation in its first layers. Why not skip both steps and feed the frequency domain into the network directly? In this paper we modify \libjpeg to produce DCT coefficients directly, modify a ResNet-50 network to accommodate the differently sized and strided input, and evaluate performance on ImageNet. We find networks that are both faster and more accurate, as well as networks with about the same accuracy but 1.77x faster than ResNet-50.
Author Information
Lionel Gueguen (UBER)
Alex Sergeev (Uber Technologies Inc,)
Ben Kadlec (Uber)
Rosanne Liu (Uber AI Labs)
Jason Yosinski (Uber AI Labs; Recursion)
More from the Same Authors
-
2019 Poster: Hamiltonian Neural Networks »
Samuel Greydanus · Misko Dzamba · Jason Yosinski -
2019 Poster: LCA: Loss Change Allocation for Neural Network Training »
Janice Lan · Rosanne Liu · Hattie Zhou · Jason Yosinski -
2019 Poster: Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask »
Hattie Zhou · Janice Lan · Rosanne Liu · Jason Yosinski -
2018 Poster: An intriguing failing of convolutional neural networks and the CoordConv solution »
Rosanne Liu · Joel Lehman · Piero Molino · Felipe Petroski Such · Eric Frank · Alex Sergeev · Jason Yosinski -
2017 Symposium: Interpretable Machine Learning »
Andrew Wilson · Jason Yosinski · Patrice Simard · Rich Caruana · William Herlands -
2017 Poster: SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability »
Maithra Raghu · Justin Gilmer · Jason Yosinski · Jascha Sohl-Dickstein -
2016 Demonstration: Adventures with Deep Generator Networks »
Jason Yosinski · Anh Nguyen · Jeff Clune · Douglas K Bemis -
2016 Poster: Synthesizing the preferred inputs for neurons in neural networks via deep generator networks »
Anh Nguyen · Alexey Dosovitskiy · Jason Yosinski · Thomas Brox · Jeff Clune -
2014 Poster: How transferable are features in deep neural networks? »
Jason Yosinski · Jeff Clune · Yoshua Bengio · Hod Lipson -
2014 Demonstration: Playing with Convnets »
Jason Yosinski · Hod Lipson -
2014 Oral: How transferable are features in deep neural networks? »
Jason Yosinski · Jeff Clune · Yoshua Bengio · Hod Lipson
