Toward perceptually consistent stereo

Toward perceptually consistent stereo

Todd Zickler, Harvard University

There are two sources of shape information in a pair of binocular stereo images. One is the correlation (matching) signal from surfaces that are visible to both cameras. The other is the decorrelation (anti-matching) signal from regions that are visible to one camera but occluded in the other. Vision science has repeatedly shown that both types of information are used in the visual cortex, and that people can perceive depth even when correlation cues are absent or very weak, a capability that remains absent from most computational stereo systems. 

I will describe two research directions that (hopefully) move us toward computational stereo algorithms that are more consistent with these perceptual phenomena. Both directions are based on representing a depth map as a piecewise smooth function over the visual field, with a flexible notion of smoothness. One is a scanline algorithm that naturally combines correlation and decorrelation cues, and that matches human perception on a collection of well-known perceptual stimuli. The other is a 2D algorithm that efficiently exploits piecewise smoothness, but so far without incorporating decorrelation cues. At the end of the talk, I hope to discuss how insights from these two different directions might be combined.  The talk is based on two papers:  

1. Juialing Wang, Daniel Glasner and Todd Zickler: 
ICCV
2017
http://vision.seas.harvard.edu/stereo/

2. Ayan Chakrabarti, Ying Xiong, Steven J. Gortler and Todd Zickler
CVPR
2015
http://ttic.uchicago.edu/~ayanc/consensus/