Estimating uncertainty in dense stereo disparity maps

Dense stereo is a well studied problem in computer vision. Generally dense stereo algorithms provide only a single estimate of disparity, ignoring uncertainty in the disparity map. Here however, we present a new, linear-time, exact method for recovering entire distributions for disparity at all pixels. This is accomplished by using a recent extension, due to Durbin et al., of the well-known forward-backward algorithm to work with two unsynchronised input streams, rather than just one as in the conventional case. The two input streams, in the stereo context are simply two corresponding epipolar lines, one from each stereo image. Specifically we consider the problem of view interpolation. The availability of a distribution over disparity is particularly appealing here. In that case, disparities themselves are not the required end product, but merely an intermediate representation. It is therefore unnecessary to estimate a unique disparity map. Instead, the image intensity at each cyclopean pixel can be estimated as a mean of predicted intensities for all possible disparities. These principles are illustrated for a teleconferencing application: enabling eye contact by positioning a virtual camera at the centre of a display screen used in a two-way conference. We show that the new approach can significantly improve the quality of the interpolated cyclopean image, compared with using unique estimated disparities.