Joint optimization based on direct sparse stereo visual-inertial odometry

Shuhuan Wen, Yanfang Zhao, Hong Zhang (Lead / Corresponding author), Hak Keung Lam, Luigi Manfredi

Research output: Contribution to journalArticle

Abstract

This paper proposes a novel fusion of an inertial measurement unit (IMU) and stereo camera method based on direct sparse odometry (DSO) and stereo DSO. It jointly optimizes all model parameters within a sliding window, including the inverse depth of all selected pixels and the internal or external camera parameters of all keyframes. The vision part uses a photometric error function that optimizes 3D geometry and camera pose in a combined energy functional. The proposed algorithm uses image blocks to extract neighboring image features and directly forms measurement residuals in the image intensity space. A fixed-baseline stereo camera solves scale drift. IMU information is accumulated between several frames using manifold pre-integration and is inserted into the optimization as additional constraints between keyframes. The scale and gravity inserted are incorporated into the stereo visual inertial odometry model and are optimized together with other variables such as poses. The experimental results show that the tracking accuracy and robustness of the proposed method are superior to those of the state-of-the-art fused IMU method. In addition, compared with previous semi-dense direct methods, the proposed method displays a higher reconstruction density and scene recovery.
Original languageEnglish
Number of pages19
JournalAutonomous Robots
Early online date10 Jan 2020
DOIs
Publication statusE-pub ahead of print - 10 Jan 2020

Fingerprint

Units of measurement
Cameras
Gravitation
Fusion reactions
Pixels
Recovery
Geometry

Keywords

  • Direct sparse odometry
  • IMU pre-integration
  • Sliding window optimization
  • 3D reconstruction

Cite this

Wen, Shuhuan ; Zhao, Yanfang ; Zhang, Hong ; Lam, Hak Keung ; Manfredi, Luigi. / Joint optimization based on direct sparse stereo visual-inertial odometry. In: Autonomous Robots. 2020.
@article{0ecf2fb75d274c1d8eeec9d99783a0c6,
title = "Joint optimization based on direct sparse stereo visual-inertial odometry",
abstract = "This paper proposes a novel fusion of an inertial measurement unit (IMU) and stereo camera method based on direct sparse odometry (DSO) and stereo DSO. It jointly optimizes all model parameters within a sliding window, including the inverse depth of all selected pixels and the internal or external camera parameters of all keyframes. The vision part uses a photometric error function that optimizes 3D geometry and camera pose in a combined energy functional. The proposed algorithm uses image blocks to extract neighboring image features and directly forms measurement residuals in the image intensity space. A fixed-baseline stereo camera solves scale drift. IMU information is accumulated between several frames using manifold pre-integration and is inserted into the optimization as additional constraints between keyframes. The scale and gravity inserted are incorporated into the stereo visual inertial odometry model and are optimized together with other variables such as poses. The experimental results show that the tracking accuracy and robustness of the proposed method are superior to those of the state-of-the-art fused IMU method. In addition, compared with previous semi-dense direct methods, the proposed method displays a higher reconstruction density and scene recovery.",
keywords = "Direct sparse odometry, IMU pre-integration, Sliding window optimization, 3D reconstruction",
author = "Shuhuan Wen and Yanfang Zhao and Hong Zhang and Lam, {Hak Keung} and Luigi Manfredi",
year = "2020",
month = "1",
day = "10",
doi = "10.1007/s10514-019-09897-6",
language = "English",
journal = "Autonomous Robots",
issn = "0929-5593",
publisher = "Springer Netherlands",

}

Joint optimization based on direct sparse stereo visual-inertial odometry. / Wen, Shuhuan; Zhao, Yanfang; Zhang, Hong (Lead / Corresponding author); Lam, Hak Keung; Manfredi, Luigi.

In: Autonomous Robots, 10.01.2020.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Joint optimization based on direct sparse stereo visual-inertial odometry

AU - Wen, Shuhuan

AU - Zhao, Yanfang

AU - Zhang, Hong

AU - Lam, Hak Keung

AU - Manfredi, Luigi

PY - 2020/1/10

Y1 - 2020/1/10

N2 - This paper proposes a novel fusion of an inertial measurement unit (IMU) and stereo camera method based on direct sparse odometry (DSO) and stereo DSO. It jointly optimizes all model parameters within a sliding window, including the inverse depth of all selected pixels and the internal or external camera parameters of all keyframes. The vision part uses a photometric error function that optimizes 3D geometry and camera pose in a combined energy functional. The proposed algorithm uses image blocks to extract neighboring image features and directly forms measurement residuals in the image intensity space. A fixed-baseline stereo camera solves scale drift. IMU information is accumulated between several frames using manifold pre-integration and is inserted into the optimization as additional constraints between keyframes. The scale and gravity inserted are incorporated into the stereo visual inertial odometry model and are optimized together with other variables such as poses. The experimental results show that the tracking accuracy and robustness of the proposed method are superior to those of the state-of-the-art fused IMU method. In addition, compared with previous semi-dense direct methods, the proposed method displays a higher reconstruction density and scene recovery.

AB - This paper proposes a novel fusion of an inertial measurement unit (IMU) and stereo camera method based on direct sparse odometry (DSO) and stereo DSO. It jointly optimizes all model parameters within a sliding window, including the inverse depth of all selected pixels and the internal or external camera parameters of all keyframes. The vision part uses a photometric error function that optimizes 3D geometry and camera pose in a combined energy functional. The proposed algorithm uses image blocks to extract neighboring image features and directly forms measurement residuals in the image intensity space. A fixed-baseline stereo camera solves scale drift. IMU information is accumulated between several frames using manifold pre-integration and is inserted into the optimization as additional constraints between keyframes. The scale and gravity inserted are incorporated into the stereo visual inertial odometry model and are optimized together with other variables such as poses. The experimental results show that the tracking accuracy and robustness of the proposed method are superior to those of the state-of-the-art fused IMU method. In addition, compared with previous semi-dense direct methods, the proposed method displays a higher reconstruction density and scene recovery.

KW - Direct sparse odometry

KW - IMU pre-integration

KW - Sliding window optimization

KW - 3D reconstruction

U2 - 10.1007/s10514-019-09897-6

DO - 10.1007/s10514-019-09897-6

M3 - Article

JO - Autonomous Robots

JF - Autonomous Robots

SN - 0929-5593

ER -