EKF-Based Real-Time Self-Attitude Estimation with Camera DNN Learning Landscape Regularities

Ryota Ozaki, Yoji Kuroda

Research output: Contribution to journalArticlepeer-review

Abstract

This letter presents an EKF (extended Kalman filter) based real-time self-attitude estimation method with a camera DNN (deep neural network) learning landscape regularities. The proposed DNN infers the gravity direction from a single shot image. It outputs the gravity direction as a mean vector and a covariance matrix in order to express uncertainty of the inference. It is pre-trained with datasets collected in a simulator. Fine-tuning with datasets collected with real sensors is carried out after the pre-training. Data augmentation is processed during the training in order to provide higher general versatility. The proposed method integrates angular rates from a gyroscope and the DNN's outputs in an EKF. The covariance matrix output from the DNN is used as process noise of the EKF. Moreover, inferences with too large variance are filtered out before processing the integration in the EKF. Static validations are performed to show the DNN can infer the gravity direction with uncertainty expression. Dynamic validations are performed to show the DNN can be used in real-time estimation. Some conventional methods are implemented for comparison.

Original languageEnglish
Article number9359333
Pages (from-to)1737-1744
Number of pages8
JournalIEEE Robotics and Automation Letters
Volume6
Issue number2
DOIs
Publication statusPublished - Apr 2021

Keywords

  • Computer vision for automation
  • localization
  • visual learning

Fingerprint Dive into the research topics of 'EKF-Based Real-Time Self-Attitude Estimation with Camera DNN Learning Landscape Regularities'. Together they form a unique fingerprint.

Cite this