Stereo and motion correspondences using nonlinear optimization method

This paper presents a new approach for using stereo and motion correspondences to solve the problem of tracking multiple independently moving features. In this approach, quantitative relational structure (QRS) is proposed as a framework for the integration of stereo–motion correspondences. The similarity function, tightly coupled to stereo and motion cues, is constructed on QRS, and then energy function E² consisting of the similarity function is defined. The tracking problem can be converted into the maximization problem of the energy function E². The stereo and motion correspondences that maximize E² are recovered by applying an extended graduated assignment algorithm. The relaxation labeling method is also presented for the comparison with the proposed method. Experimental results are presented to illustrate the performance of the proposed method.