Low level vision addresses the issues of labeling and organizing image pixels according to scene related properties - such as motion, contrast, depth and reflectance. I will describe our attempts to understand low-level vision in humans and machines as optimal inference given the statistics of the world. If time permits, I will discuss my favorite NIPS rejected papers.