Abstract:

In contrast with the geometry method, photometric bundle adjustment has demonstrated several advantages, including higher accuracy and better robustness in a low-texture environment in vision navigation and mapping systems. However, its cost function formed by the photometric error is highly nonconvex in terms of pose and structure parameters. The primary reason for that is the nonlinearity in image intensities. Except for a gradual brightness change, intensities do not exhibit linearity, leading to a huge sensitivity on the initial state of the optimized variable. Recent works have tried various methods to circumvent this problem, most popular approaches include coarse-to-fine method, patch-based method, and deep learning method. In recent research, it has been proven that multi-sensor fusion is an effective solution. Among different types of sensors for fusion, LiDAR and IMU are the most commonly used two. By introducing information from other sensors to provide extra three-dimensional prior or applying a combined cost function, the nonconvexity of photometric cost can be significantly decreased. During this seminar, the focus will be on advancements made in the progress of photometric bundle adjustment and sensor fusion approach. Additionally, a new method that integrates a lidar prior into a patch-based photometric bundle adjustment will be introduced, thereby increasing the accuracy and robustness of the approach.