H-VLO: hybrid LiDAR-camera fusion for self-supervised odometry

In this paper, we propose a hybrid visual-LiDAR odometry (H-VLO) framework that fuses predicted visual depth map and completed LiDAR map. Compared to the previous visual-LiDAR odometry methods, our approach leverages 2D feature matching and 3D association by utilizing deep depth map, deep flow map and deep LiDAR depth completion networks. Rather than extraction of the depth values from LiDAR measurements for each visual feature, our method first densifies a LiDAR scan with a deep depth completion network and then fuses it with visual deep depth map estimation in a Bayesian framework. This method reduces pose estimation drift by improving feature-to-feature and point-to-feature matching, as well as scale recovery. The evaluations on the public KITTI odometry benchmark show that our technique achieves better or at least comparable estimates than the state-of-the-art visual-LiDAR and monocular visual odometry approaches.