Real-time Feature Tracking and Outlier Rejection with Changes in Illumination
|
||||
| Project summary | ||||
| Feature tracking (or correspondence establisment) has been an essential component of vision-based systems in a variety of applications ranging from vision-based control, human-computer interactions, medical imaging, surveillance and visual reconstruction. The problem is to identify how the projection of a point on an object moves in images as the object or the camera moves in time. It is usually approached by finding how a small image region around the projection deforms in time by imposing some similarity measure on the images measurements. The great challenge is to deal with the change of the appearance of the scene, which arises from variations in the relative scene pose (the relative pose of the scene with respect to the camera), variation in illumination and non-Lambertian reflectance nature of the scene. While the first has been discussed in the literature by various authors, the latter two have been largely ignored, even though the change of appearance induced by the latter two is much more signifcant than the first. We develop an algorithm which can properly deal with variations due to all these facts. The algorithm can work in real time on personal laptops, which makes it useful for practical applications such as vision-based control and human-computer interactions. The algorithm automatically detects outliers in a hypothesis testing frameworl. | ||||
| Related publications | ||||
| Hailin Jin, Paolo Favaro and Stefano Soatto. Real-time Feature Tracking and Outlier Rejection with Changes in Illumination. In Proc. IEEE Intl. Conf. on Computer Vision, pages 684-689, July 2001. Available in PDF (712KB) | ||||
| A geometric-photometric model | ||||
The geometric deformation due to relative pose between the
scene and
the
camera can be well modeled using an affine transformation, which has
been considered by [Shi and Tomasi 1994]: The photometric deformation of an image patch due to changes in illumination or the non-Lambertian nature of the scene or a combination of both can be well modeled using again an affine transformation:  By putting these two together, we obtain our geometric-photometric model:  The unknown parameters can be inferred by imposing a simple sum-of squared-difference (SSD) error measure:  |
||||
| Discussions: for SFM, is feature-tracking really necessary at all? | ||||
| For tasks like structure from motion, | ||||
| Sponsors | ||||
| This research is supported by ARO grant DAAD19-99-1-0139 and
Intel grant 8029.
Copyright (c) 2003 Hailin Jin, Paolo Favaro and Stefano Soatto. Please send your comments to hljin@cs.ucla.edu Last update: December 8, 2003. BACK to Hailin Jin's homepage |
