The Hilti SLAM Challenge Dataset - arXiv
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
The Hilti SLAM Challenge Dataset
Michael Helmberger1 , Kristian Morin1 , Nitish Kumar1 , Danwei Wang2 , Yufeng Yue3 ,
Giovanni Cioffi4 , Davide Scaramuzza4
Abstract— Accurate and robust pose estimation is a funda-
mental capability for autonomous systems to navigate, map
and perform tasks. Particularly, construction environments pose
challenging problem to Simultaneous Localization and Map-
ping (SLAM) algorithms due to sparsity, varying illumination
arXiv:2109.11316v1 [cs.RO] 23 Sep 2021
conditions, and dynamic objects. Current academic research
in SLAM is focused on developing more accurate and robust
algorithms for example by fusing different sensor modalities.
To help this research, we propose a new dataset, the Hilti
SLAM Challenge Dataset. The sensor platform used to collect
this dataset contains a number of visual, lidar and inertial
sensors which have all been rigorously calibrated. All data is
temporally aligned to support precise multi-sensor fusion. Each
dataset includes accurate ground truth to allow direct testing
of SLAM results. Raw data as well as intrinsic and extrinsic
sensor calibration data from twelve datasets in various envi-
ronments is provided. Each environment represents common
scenarios found in building construction sites in various stages
of completion.
SUPPLEMENTARY MATERIAL
The dataset as well as the documentation is available at
Fig. 1. Left: The full sensor stick resting on a bipod. Right: (1) ADIS16445
https://www.hilti-challenge.com. IMU (2) AlphaSense (3) Ouster OS0-64 lidar (4) Livox MID 70 (5) prism
for total station.
I. INTRODUCTION
Robots on construction sites promise improved safety of
workers, task productivity and high quality data capture [1]. Many types of indoor and outdoor spaces remain sparsely
Although some dangerous tasks (such as concrete chain explored in research due to a lack of reliable test data with
sawing) are prime work to automate, the more repetitive accurate reference information. Previous efforts to collect
and ergonomically difficult tasks (such as overhead drilling and distribute high quality multi-sensor data have resulted in
and installation) result in many worker injuries. Construction significant improvements and insights in the research areas
robotics offers a way to remove this worker hazard, while of visual odometry, SLAM and sensor fusion [3], [4], [5],
also improving task scheduling and progress monitoring. To [6], [7].
achieve that goal however, automation requires a wide array In order to support the growing body of experiences in
of technologies and techniques to perceive, map and navigate various indoor and mixed environments, we have created a
through the environment. suite of sensors (representing different categories of sensor
With the introduction of GNSS and INS augmented sys- techniques) with attention made to precise time synchro-
tems, high performance outdoor positioning and navigation nization and calibration. With this sensor platform, we have
solutions are widely available [2]. Continuing that trend to collected datasets in various locations with different practical
indoor or complex outdoor environments however, remains deficiencies seen in real-world scenarios (Fig. 2); along with
a challenge. To bridge the gap, autonomous positioning accurate ground truth for effective testing and evaluation.
systems rely on a fusion of sensors and techniques. But these Whereas previous datasets have focused on automotive ve-
hardware platforms can be a barrier to high quality research hicle motion, airborne UAV motion, or provided expansive
due to cost and integration complexity. simulations, this dataset targets real spaces collected by hand-
held or robotic-like motion, using the latest in commercially
*This work was supported by the Hilti AG, Schaan, Liechtenstein
1 Hilti AG, Schaan, Liechtenstein available sensing technology. With the use of redundant
2 School of Electrical and Electrical Engineering, Nanyang Technological sensors, this dataset also provides a direct comparison of
University Singapore sensor performance in different environments; which can
3 School of Automation, Beijing Institute of Technology, China
4 Robotics and Perception Group, Dept. of Informatics, University of
be informative for future system designs. We believe that
Zurich, and Dept. of Neuroinformatics, University of Zurich and ETH these collection scenarios capture a wide array of robotic
Zurich, Switzerland and reality capture uses cases that many not be as illustrative
(a) Basement (b) Campus (c) Construction Site (d) IC Office
(e) Lab (f) Office Mitte (g) Parking (h) RPG Tracking Area
Fig. 2. Locations where the datasets have been captured.
with previous data. Our aim is to stimulate research on robust C. Inertial Sensors
indoor positioning, mapping and navigation with particular Analog Devices ADIS164454
application to construction environments. This IMU is rigidly mounted to the AlphaSense
II. SENSOR SETUP module. It is a high performing MEMS based sensor
Our sensor suite (the ’Phasma’ stick, Fig. 1) consists with relatively low noise and sensor bias drift rates.
of 3 categories of sensing modalities, along with multiple The data from this IMU is tightly timestamped to the
sensors operating with different ranges and noise levels. AlphaSense timing system. Data is collected at 800Hz.
These include:
Bosch BMI0855
A. Passive Visual This IMU is embedded in the AlphaSense module. It
AlphaSense by Sevensense1 provides a modest level of performance in terms of
The visual data is collected from an array of rigidly noise and bias stability. The data from this IMU is
mounted 1.3MP global shutter cameras. This module tightly timestamped to the AlphaSense timing system.
consists of 5 wide field-of-view cameras mounted to Data is collected at 200Hz.
give an approximate 270 deg continuous field of view.
Within this configuration a stereo camera is also present. InvenSense ICM-209486
Images are synchronously collected at 10Hz. This IMU is embedded in the Ouster lidar. It provides a
more modest level of performance than the ADIS16445
B. Active Optical in terms of noise and bias stability. The data from this
Ouster OS0-642 IMU is tightly timestamped to the Ouster timing system.
Long range point cloud data is collected by the 360 deg Data is collected at 100Hz.
scanning lidar sensor. This unit has a scan repetition of
10Hz, and a point data rate of 1,300,000 points/second. D. Ground Truth System
Ranges are recorded from 0.3 to 50m, with typical For testing and validation, 2 systems are used to capture
lowest noise returns greater than 1m. Range accuracy ground truth:
is 1.5-5cm.
Total Station
Livox MID703 A survey grade prism is attached to the Phasma stick.
This unit is a lidar sensor with a 70 deg circular field of This is tracked by the Hilti PLT3007 automated total
view and a non repeating scan pattern. Point datarate is station. Most datasets are collected in a ”stop ’n
100,000 points/second. Ranges are recorded from 0.02 go” fashion, where the total station makes a precise
to 200m, with typical returns between 1 and 50m. Range
4 https://www.analog.com/en/products/adis16445.html
accuracy is 2-5cm. 5 https://www.bosch-sensortec.com/products/motion-
1 https://www.sevensense.ai/product/alphasense-position sensors/imus/bmi085/
2 https://ouster.com/products/os0-lidar-sensor/ 6 https://invensense.tdk.com/products/motion-tracking/9-axis/icm-20948
3 https://www.livoxtech.com/mid-70 7 https://www.hilti.com/c/CLS MEA TOOL INSERT 7127
Fig. 3. Time Synchronization between modules. Gyro plots on IMUs align
TABLE I
A BSOLUTE TRAJECTORY ERROR IN METERS FOR 3 SLAM ALGORITHMS : ORBSLAM2 [12] USING STEREO IMAGES , A-LOAM [13] USING LIDAR ,
AND SVO2.0 [14] USING STEREO IMAGES AND IMU AS INPUT. D EPENDING ON THE SCENE , A DIFFERENT SENSOR IS ADVANTAGEOUS WHICH
HIGHLIGHTS THE NEED FOR FUSION OF ALL SENSING MODALITIES .
Basement 1 Basement 4 Campus 2 Construction Site 2 Lab Survey 2 RPG Drone Testing Area
A-LOAM 0.162 0.288 13.332 6.395 0.088 2.838
SVO2.0 0.813 2.598 8.941 2.992 0.082 1.927
ORBSLAM2 x 10.306 x x x x
practical challenges in different stages of construction. VI. DATASET FORMAT
Challenges include variable lighting, limited features and/or Datasets are stored in binary format as rosbags
highly reflective and transparent surfaces. which contain images and IMU measurements
using the standard sensor msgs/Image and
Data Descriptions (see Fig. 2): sensor msgs/Imu message types, respectively. The
(a) Basement Ouster data uses the sensor msgs/PointCloud2
Data was collected in a windowless room (approx. format while the Livox data is stored in the custom
20x40m). No natural light; mixed illumination bright- livox ros driver/CustomMsg message type to not
ness. Concrete space with building infrastructure. Base- loose . Fig. 4 shows an example of the camera and lidar
ment 1 is a short and easy path, and for Basement 3 data from the Lab Survey 2 dataset. Reference/ground
and Basement 4 we mounted the sensor platform on a truth data is given in a separate file for each dataset,
moving base instead of operating it handheld. Basement with the filename indicating the reference source (e.g.
3 and 4 also allow to exploit loop closure capabilities Construction Site prism.txt means the ground
of the SLAM systems. truth is in the prism frame). Rosbag contents are listed in
(b) Campus Table II.
Data was collected outdoors in a courtyard setting
(approx. 40x60m). Good natural lighting with high VII. EVALUATION
illumination. Mixed features with building structure and The evaluation of the datasets is based on the absolute
natural flora. trajectory error (ATE) [15] after SE3 alignment of the ground
(c) Construction Site truth with the estimated trajectory. The correspondences are
Mostly outdoors with some covered areas (approx. matched based on their timestamps. Scenarios collected with
40x80m). Strong natural light with high illumination. a motion capture system are compared with their XYZ posi-
Unfinished natural surfaces with limited features above tion components only; for the other datasets the intermittent
the ground plane. XYZ total station observations are compared. An example
(d) IC Office of a SLAM comparison with motion capture ground truth is
Indoor space with many windows and reflective surfaces depicted in Fig. 5 for the dataset LAB Survey 2. An example
(approx. 10x70m). Mix of natural and artificial light. of what established approaches can achieve on a few datasets
Strong illumination at the windows, modest illumination with total station ground truth is shown in Table I.
indoors. ORBSLAM2 (which is using stereo images) is not able to
(e) Lab produce meaningful results in all handheld datasets because
Indoor space dominated by large windows (approx. it immediately looses track of features due to fast rotational
10x10m). Strong natural light and reflective surfaces. movements. The only dataset where ORBSLAM2 is not
Optitrack 6DOF ground truth.
(f) Office Mitte
Indoor space in finished office building (approx.
30x50m). Mix of natural and artificial light. Lots of
building structure.
(g) Parking
Mix of indoor and outdoor space (approx. 100x100m).
Parking garage from top floor to lower floor. Lighting
varies from extreme bright to modest darkness. Ground
plane structure on top floor; lots of building structure
on lower floor.
(h) RPG Tracking Area
Indoor test facility (approx. 30x30m). Mostly artificial
light with some natural. Single large room with random
Fig. 4. Example from the dataset LAB Survey 2: images from 5 cameras
motion path throughout. Vicon 6DOF ground truth. and the Ouster OS0 point cloud
failing is Basement 4 where the Phasma stick was mounted discussions, and IVISO for the calibration verification using
on a moving platform. The other tested algorithms, A- the tool Camcalib.
LOAM and SVO2.0, are more robust with initialization and
R EFERENCES
tracking. However, depending on the scene texture and scene
geometry, either the lidar based A-LOAM or SVO 2.0 which [1] Hilti AG. (2020). “Hilti Jaibot,” [Online]. Available:
is using stereo cameras and the IMU performs better. https://www.hilti.ca/content/hilti/
W1 / CA / en / business / business / trends /
jaibot.html. (accessed: 12.09.2021).
[2] K.-P. Schwarz, M. Chapman, M. Cannon, and P. Gong,
“An integrated INS/GPS approach to the georefer-
encing of remotely sensed data,” Photogrammetric
Engineering and Remote Sensing, vol. 59, no. 11,
pp. 1667–1674, 1993.
[3] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for
autonomous driving? The KITTI Vision Benchmark
Suite,” in Conference on Computer Vision and Pattern
Recognition (CVPR), 2012.
[4] W. Wang, D. Zhu, X. Wang, Y. Hu, Y. Qiu, C.
Wang, Y. Hu, A. Kapoor, and S. Scherer, “Tartanair:
A dataset to push the limits of visual SLAM,” in
2020 IEEE/RSJ International Conference on Intelli-
Fig. 5. Example comparison of SVO2 [14] using stereo and IMU, LOAM gent Robots and Systems (IROS), 2020.
[13] using the Ouster lidar. ORBSLAM2 faild to produce meaningful results.
Dataset: Lab Survey 2
[5] M. Burri, J. Nikolic, P. Gohl, T. Schneider, J. Rehder,
S. Omari, M. W. Achtelik, and R. Siegwart, “The Eu-
RoC micro aerial vehicle datasets,” The International
VIII. KNOWN ISSUES Journal of Robotics Research, 2016. DOI: 10.1177/
Despite careful design and execution of the data collection 0278364915620033.
experiments, we are aware of different issues which pose [6] J. Delmerico, T. Cieslewski, H. Rebecq, M. Faessler,
additional challenges for processing and limit achievable and D. Scaramuzza, “Are we ready for autonomous
accuracy when comparing to ground truth. These include: drone racing? the UZH-FPV drone racing dataset,” in
IEEE Int. Conf. Robot. Autom. (ICRA), 2019.
• Clock Drift and Offset: The clocks from motion-capture
[7] E. Mueggler, H. Rebecq, G. Gallego, T. Delbruck,
system and the data logging computer are not hardware-
and D. Scaramuzza, “The event-camera dataset and
synchronized. We used Ethernet connection and a time-
simulator: Event-based data for pose estimation, visual
of-arrival time stamping to keep the offset to a mini-
odometry, and SLAM,” The International Journal
mum, however we observed a difference of around 1-3
of Robotics Research, vol. 36, no. 2, pp. 142–149,
ms in the two clocks.
Feb. 2017, ISSN: 1741-3176. DOI: 10 . 1177 /
• Some frames in the lidar and camera data have been
0278364917691115.
dropped due to high load on the controller.
[8] “IEEE standard for a precision clock synchroniza-
IX. CONCLUSION tion protocol for networked measurement and control
In this paper we have described a new public dataset systems,” IEEE Std 1588-2008 (Revision of IEEE
captured with a highly redundant multi-sensor platform. Our Std 1588-2002), pp. 1–269, 2008. DOI: 10.1109/
goal is to improve the use of SLAM algorithms in con- IEEESTD.2008.4579760.
struction robotics to assist in task automation and execution. [9] M. Faizullin, A. Kornilova, A. Akhmetyanov, and
This data captures a series of real-world examples collected G. Ferrer, “Twist-n-sync: Software clock synchro-
with current sensing technologies with a high quality time nization with microseconds accuracy using MEMS-
synchronization. Based on the results we showed, it is clear, gyroscopes,” Sensors, vol. 21, no. 1, 2021, ISSN: 1424-
that for using SLAM in real world construction use cases 8220. DOI: 10.3390/s21010068.
like progress monitoring or surveying, the robustness and [10] F. Furrer, M. Fehr, T. Novkovic, H. Sommer, I.
accuracy has to be improved significantly. We hope to expand Gilitschenski, and R. Siegwart, “Evaluation of com-
this data offerings into various other environments, to further bined time-offset estimation and hand-eye calibration
spur research on positioning and navigation issues commonly on robotic datasets,” in Field and Service Robotics:
encountered in indoor and mixed environments. Results of the 11th International Conference, R. Sieg-
wart and M. Hutter, Eds. Cham: Springer International
ACKNOWLEDGMENTS Publishing, 2017, ISBN: 978-3-319-67361-5.
The authors are grateful for the support of the whole team
from Sevensense for their continuous support and helpful
TABLE II
T OPICS AND TYPES IN THE ROSBAG
Topic Type Description
/alphasense/cam0/image raw sensor msgs/Image front facing camera 1
/alphasense/cam1/image raw sensor msgs/Image front facing camera 2
/alphasense/cam2/image raw sensor msgs/Image upward facing camera
/alphasense/cam3/image raw sensor msgs/Image right facing camera
/alphasense/cam4/image raw sensor msgs/Image left facing camera
/alphasense/imu sensor msgs/Imu Bosch IMU, 200Hz
/alphasense/imu adis sensor msgs/Imu ADIS16446, 800Hz
/livox/lidar livox ros driver/CustomMsg Livox MID70
/os cloud node/imu sensor msgs/Imu Ivensense, 100Hz
/os cloud node/points sensor msgs/PointCloud2 Ouster OS0-64
tf static tf2 msgs/TFMessage all transforms between frames
[11] T. Foote, “tf: The transform library,” in 2013 IEEE
Conference on Technologies for Practical Robot Ap-
plications (TePRA), IEEE, Apr. 2013. DOI: 10 .
1109/tepra.2013.6556373.
[12] R. Mur-Artal and J. D. Tardós, “ORB-SLAM2: An
open-source SLAM system for monocular, stereo and
RGB-D cameras,” IEEE Transactions on Robotics,
vol. 33, no. 5, pp. 1255–1262, 2017. DOI: 10.1109/
TRO.2017.2705103.
[13] J. Zhang and S. Singh, “LOAM: Lidar odometry and
mapping in real-time,” Robotics: Science and Systems
Conference (RSS), pp. 109–111, Jan. 2014.
[14] C. Forster, Z. Zhang, M. Gassner, M. Werlberger, and
D. Scaramuzza, “SVO: Semidirect visual odometry for
monocular and multicamera systems,” IEEE Transac-
tions on Robotics, vol. 33, no. 2, pp. 249–265, 2017.
DOI : 10.1109/TRO.2016.2623335.
[15] Z. Zhang and D. Scaramuzza, “A tutorial on quanti-
tative trajectory evaluation for visual(-inertial) odom-
etry,” in 2018 IEEE/RSJ International Conference
on Intelligent Robots and Systems (IROS), 2018,
pp. 7244–7251. DOI: 10 . 1109 / IROS . 2018 .
8593941.
You can also read
