在本地虚拟机和AutoDL云端环境运行ORB_SLAM3的介绍。

1. 介绍

ORB-SLAM3是西班牙萨拉戈萨大学于2020年7月中旬开源的作品，与ORB-SLAM、ORB-SLAM2一脉相承，是一个能够使用单目、立体、RGB-D相机，兼容针孔及鱼眼相机模型进行视觉、视觉+惯导和多地图的综合性SLAM方案。

本文不关注ORB-SLAM3的理论实现，仅记录如何在本地环境中运行ORB-SLAM3。

更多参考资料：

因为之前折腾过GCNv2-SLAM，在本地运行ORB-SLAM3比GCNv2-SLAM更简单。

2. 依赖安装

ORB-SLAM3的依赖项和GCNv2几乎完全一致，且ORB-SLAM3不需要PyTorch，依赖项如下所示：

  • opencv大于3.0版本；
  • eigen3大于3.1.0版本；
  • Pangolin没有版本要求，用6.0没问题；
  • 需要libpython2.7-dev包；

ubuntu18.04系统安装依赖项可以参考我先前编写的GCNv2博客：

参考博客里面的步骤安装opencv、eigen3、Pangolin6.0就可以了，不需要安装libtorch。

如果你想使用AutoDL云端环境运行，选择2080ti的PyTorch1.5.1版本镜像即可。

3. 编译项目

项目的编译非常简单，安装好所有依赖项后，直接使用根目录下的build.sh脚本编译就可以了，不需要修改该脚本。

1
2
3
4
5
git clone https://github.com/UZ-SLAMLab/ORB_SLAM3.git ORB_SLAM3

cd ORB_SLAM3
chmod +x build.sh
./build.sh

初次编译的时候，会遇到如下cmake告警，提示需要opencv 4.4版本

1
2
3
4
5
6
7
8
9
10
11
12
13
CMake Warning at CMakeLists.txt:33 (find_package):
  Could not find a configuration file for package "OpenCV" that is compatible
  with requested version "4.4".

  The following configuration files were considered but not accepted:

    /usr/local/share/OpenCV/OpenCVConfig.cmake, version: 3.4.5
    /usr/share/OpenCV/OpenCVConfig.cmake, version: 3.2.0



CMake Error at CMakeLists.txt:35 (message):
  OpenCV > 4.4 not found.

但实际上Github的项目README里面已经说了只需要opencv大于3.0版本就可以了。根据Github Issue：github.com/UZ-SLAMLab/ORB_SLAM3/issues/456，直接修改项目根目录下的CMakeLists.txt文件就能解决此问题，将如下部分从4.4改成3.4即可。

1
2
3
4
find_package(OpenCV 3.4)
   if(NOT OpenCV_FOUND)
      message(FATAL_ERROR "OpenCV > 3.4 not found.")
   endif()

修改了之后，需要先删除刚刚的编译缓存，重新运行脚本

1
2
3
4
rm -rf Thirdparty/g2o/build/
rm -rf Thirdparty/DBoW2/build/
rm -rf Vocabulary/*.bin
rm -rf ./build

AutoDL云端环境正常编译通过

image.png

image.png

本地arm64虚拟机ubuntu 18.04 docker容器内也能正常编译

image.png

4. 使用TUM RGB-D数据集运行项目

4.1. 下载和处理数据集

这里需要的操作和GCNv2是一样的，下载了RGB-D数据集后需要用python脚本处理一下。

下载地址：cvg.cit.tum.de/data/datasets/rgbd-dataset/download

下载fr1/desk数据集，这是一个桌子的RGBD数据

image.png

在GCNv2_SLAM工程下新建datasets/TUM,将数据集下载到其中

1
2
3
4
5
6
7
# 新建datasets/TUM数据集文件夹
mkdir -p datasets/TUM 
cd datasets/TUM
# 下载数据集到datasets/TUM文件夹内
wget -O rgbd_dataset_freiburg1_desk.tgz https://cvg.cit.tum.de/rgbd/dataset/freiburg1/rgbd_dataset_freiburg1_desk.tgz
# 解压数据集
tar -xvf rgbd_dataset_freiburg1_desk.tgz

然后还需要下载一个associate.py脚本来处理一下数据集才能正常运行

下载地址：svncvpr.in.tum.de，同时在我的Github仓库也做了留档。

1
wget -O associate.py https://svncvpr.in.tum.de/cvpr-ros-pkg/trunk/rgbd_benchmark/rgbd_benchmark_tools/src/rgbd_benchmark_tools/associate.py

这个脚本只能用python2运行，需要下载numpy库。注意AutoDL的环境中python绑定到了python3，环境中的python2被拦掉了，所以需要安装独立的python2命令来运行python2。

在Pytorch1.5.1版本的AutoDL镜像中，可以直接使用下面的命令来安装python2和pip2：

1
apt-get install -y python-dev python-pip

安装了以后，云端AutoDL环境需要使用python2和pip2命令，本地虚拟机中使用python和pip命令就可以了（本地的python和pip命令一般绑定的都是python2）

随后安装numpy库就ok了

1
2
3
4
5
6
7
8
root@autodl-container-e39d46b8d3-01da7b14:~/autodl-tmp/GCNv2_SLAM/datasets/TUM# pip2 install numpy
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Looking in indexes: http://mirrors.aliyun.com/pypi/simple
Collecting numpy
  Downloading http://mirrors.aliyun.com/pypi/packages/3a/5f/47e578b3ae79e2624e205445ab77a1848acdaa2929a00eeef6b16eaaeb20/numpy-1.16.6-cp27-cp27mu-manylinux1_x86_64.whl (17.0 MB)
     |████████████████████████████████| 17.0 MB 21.1 MB/s 
Installing collected packages: numpy
Successfully installed numpy-1.16.6

执行脚本来处理两个文件，在数据文件夹里执行命令

1
python2 associate.py rgbd_dataset_freiburg1_desk/rgb.txt rgbd_dataset_freiburg1_desk/depth.txt > rgbd_dataset_freiburg1_desk/associate.txt

执行python命令后可以看看合并成功了没有，如下应该就是没问题了。

1
2
3
1305031472.895713 rgb/1305031472.895713.png 1305031472.892944 depth/1305031472.892944.png
1305031472.927685 rgb/1305031472.927685.png 1305031472.924814 depth/1305031472.924814.png
1305031472.963756 rgb/1305031472.963756.png 1305031472.961213 depth/1305031472.961213.png

在同一个网站下载的其他TUM数据集也需要用相同的方式进行处理

4.2. 运行项目

在项目根目录下，使用如下命令运行项目，最后两个参数指向数据集的路径以及数据集整理后的associate.txt文件路径。第三个参数TUM1.yaml对应我们下载的freiburg1数据集。

1
2
3
4
5
./Examples/RGB-D/rgbd_tum \
    ./Vocabulary/ORBvoc.txt \
    ./Examples/RGB-D/TUM1.yaml \
    ./datasets/TUM/rgbd_dataset_freiburg1_desk \
    ./datasets/TUM/rgbd_dataset_freiburg1_desk/associate.txt

注意，最后两个参数一定要填对，如果你填写了错误的目录或文件名，执行命令后程序会阻塞，不会有任何有效输出（第一次运行的时候我就填错目录了，一直阻塞在那里，傻等了好久）

正常情况下，项目运行输出如下

image.png

4.3. AutoDL云端运行

云端运行，VNC中能正常显示画面

image.png

image.png

AutoDL云端运行后的输出如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
[root@autodl-container-0d454391d6-b3b7582b:~/gcn/ORB_SLAM3]$ ./Examples/RGB-D/rgbd_tum \
>     ./Vocabulary/ORBvoc.txt \
>     ./Examples/RGB-D/TUM1.yaml \
>     ./datasets/TUM/rgbd_dataset_freiburg1_desk \
>     ./datasets/TUM/rgbd_dataset_freiburg1_desk/associate.txt

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: RGB-D
Loading settings from ./Examples/RGB-D/TUM1.yaml
        -Loaded camera 1
Camera.newHeight optional parameter does not exist...
Camera.newWidth optional parameter does not exist...
        -Loaded image info
        -Loaded RGB-D calibration
        -Loaded ORB settings
Viewer.imageViewScale optional parameter does not exist...
        -Loaded viewer settings
System.LoadAtlasFromFile optional parameter does not exist...
System.SaveAtlasToFile optional parameter does not exist...
        -Loaded Atlas settings
System.thFarPoints optional parameter does not exist...
        -Loaded misc parameters
----------------------------------
SLAM settings: 
        -Camera 1 parameters (Pinhole): [ 517.306 516.469 318.643 255.314 ]
        -Camera 1 distortion parameters: [  0.262383 -0.953104 -0.005358 0.002628 1.16331 ]
        -Original image size: [ 640 , 480 ]
        -Current image size: [ 640 , 480 ]
        -Sequence FPS: 30
        -RGB-D depth map factor: 5000
        -Features per image: 1000
        -ORB scale factor: 1.2
        -ORB number of scales: 8
        -Initial FAST threshold: 20
        -Min FAST threshold: 7


Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Initialization of Atlas from scratch 
Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name: 
There are 1 cameras in the atlas
Camera 0 is pinhole

-------
Start processing sequence ...
Images in the sequence: 573

First KF:0; Map init KF:0
New Map created with 937 points
Framebuffer with requested attributes not available. Using available framebuffer. You may see visual artifacts.Starting the Viewer
Shutdown
-------

median tracking time: 0.0371458
mean tracking time: 0.0415715

Saving camera trajectory to CameraTrajectory.txt ...

Saving keyframe trajectory to KeyFrameTrajectory.txt ...

4.4. 本地虚拟机运行

4.4.1. 解决段错误

本地虚拟机初次运行的时候遇到了段错误（有图像输出，主要是Shutdown的时候段错误）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
-------
Start processing sequence ...
Images in the sequence: 573

First KF:0; Map init KF:0
New Map created with 939 points
Starting the Viewer
Shutdown
-------

median tracking time: 0.0136386
mean tracking time: 0.0140401

Saving camera trajectory to CameraTrajectory.txt ...

Saving keyframe trajectory to KeyFrameTrajectory.txt ...
Segmentation fault (core dumped)

image.png

根据：github.com/UZ-SLAMLab/ORB_SLAM3/issues/452，需要修改代码，主要修改ORB_SLAM3/src/System.cc中的System::Shutdown()函数，将函数中原有的注释去掉，并将最后一部分的"ORB-SLAM2: Map Viewer"改成"ORB-SLAM3: Map Viewer"。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
void System::Shutdown()
{
    {
        unique_lock<mutex> lock(mMutexReset);
        mbShutDown = true;
    }

    cout << "Shutdown" << endl;

    mpLocalMapper->RequestFinish();
    mpLoopCloser->RequestFinish();
    // 取消这里代码的注释
    if(mpViewer)
    {
        mpViewer->RequestFinish();
        while(!mpViewer->isFinished())
            usleep(5000);
    }

    // Wait until all thread have effectively stopped
    while(!mpLocalMapper->isFinished() || !mpLoopCloser->isFinished() || mpLoopCloser->isRunningGBA())
    {
        if(!mpLocalMapper->isFinished())
            cout << "mpLocalMapper is not finished" << endl;
        if(!mpLoopCloser->isFinished())
            cout << "mpLoopCloser is not finished" << endl;
        if(mpLoopCloser->isRunningGBA()){
            cout << "mpLoopCloser is running GBA" << endl;
            cout << "break anyway..." << endl;
            break;
        }
        usleep(5000);
    }

    if(!mStrSaveAtlasToFile.empty())
    {
        Verbose::PrintMess("Atlas saving to file " + mStrSaveAtlasToFile, Verbose::VERBOSITY_NORMAL);
        SaveAtlas(FileType::BINARY_FILE);
    }

    // 此处注释取消并将ORB-SLAM2改成ORB-SLAM3
    if(mpViewer)
        pangolin::BindToContext("ORB-SLAM3: Map Viewer");

#ifdef REGISTER_TIMES
    mpTracker->PrintTimeStats();
#endif


}

修改后需要删除项目根目录下的build目录，重新编译

image.png

4.4.2. 运行结果

重新编译后再次运行

image.png

本地虚拟机运行结束后完整输出如下，这次没有段错误了

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
root@ubuntu-linux-22-04-02-desktop:/work/ORB_SLAM3# ./Examples/RGB-D/rgbd_tum \
>     ./Vocabulary/ORBvoc.txt \
>     ./Examples/RGB-D/TUM1.yaml \
>     ./datasets/TUM/rgbd_dataset_freiburg1_desk \
>     ./datasets/TUM/rgbd_dataset_freiburg1_desk/associate.txt

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: RGB-D
Loading settings from ./Examples/RGB-D/TUM1.yaml
	-Loaded camera 1
Camera.newHeight optional parameter does not exist...
Camera.newWidth optional parameter does not exist...
	-Loaded image info
	-Loaded RGB-D calibration
	-Loaded ORB settings
Viewer.imageViewScale optional parameter does not exist...
	-Loaded viewer settings
System.LoadAtlasFromFile optional parameter does not exist...
System.SaveAtlasToFile optional parameter does not exist...
	-Loaded Atlas settings
System.thFarPoints optional parameter does not exist...
	-Loaded misc parameters
----------------------------------
SLAM settings: 
	-Camera 1 parameters (Pinhole): [ 517.306 516.469 318.643 255.314 ]
	-Camera 1 distortion parameters: [  0.262383 -0.953104 -0.005358 0.002628 1.16331 ]
	-Original image size: [ 640 , 480 ]
	-Current image size: [ 640 , 480 ]
	-Sequence FPS: 30
	-RGB-D depth map factor: 5000
	-Features per image: 1000
	-ORB scale factor: 1.2
	-ORB number of scales: 8
	-Initial FAST threshold: 20
	-Min FAST threshold: 7


Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Initialization of Atlas from scratch 
Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name: 
There are 1 cameras in the atlas
Camera 0 is pinhole

-------
Start processing sequence ...
Images in the sequence: 573

First KF:0; Map init KF:0
New Map created with 939 points
Starting the Viewer
Shutdown
-------

median tracking time: 0.0124658
mean tracking time: 0.0127179

Saving camera trajectory to CameraTrajectory.txt ...

Saving keyframe trajectory to KeyFrameTrajectory.txt ...
root@ubuntu-linux-22-04-02-desktop:/work/ORB_SLAM3#

4.5. 结果对比

从输出结果来看，本地4核8G的虚拟机环境运行速度为0.0124658，约合80.2 HZ，快于AutoDL云端12核+2080ti环境运行的0.0371458，约合26.9 HZ。由于ORB-SLAM3不依赖于PyTorch，应该没有使用显卡的解算能力（俺不确定哈），这部分差异主要是由于CPU造成的。云端CPU和GPU都是容器化共享的，因此影响了性能。

而且，在GCNv2的本地纯CPU运行中（相同环境），由于缺少显卡，只跑出了可怜巴巴的0.5HZ的速度，可见在无显卡环境中ORB-SLAM3运行速度快于GCNv2-SLAM。至于二者的性能和精度，暂时不太清楚如何对比。

5. The end

本文简单介绍了如何使用ORB-SLAM3运行TUM RGB-D数据集，纯CPU运行快于GCNv2-SLAM。