Determining pose of 3D objects with curved surfaces

A method is presented for computing the pose of rigid 3D objects with arbitrary curved surfaces. Given an input image and a candidate object model and aspect, the method will verify whether or not the object is present and if so, report pose parameters. The curvature method of Bash and Ullman is used to model points on the object rim, while stereo matching is used for internal edge points. The model allows an object edge-map to be predicted from pose parameters. Pose is computed via an iterative search for the best pose parameters. Heuristics are used so that matching can succeed in the presence of occlusion and artifact and without resetting to use of corresponding salient feature points. Bench tests and simulations show that the method almost always converges to ground truth pose parameters for a variety of objects and for a broad set of starting parameters in the same aspect     

Floor segmentation by computing plane normals from image motion fields for visual navigation

This paper proposes a method of detecting movable paths during visual navigation for a robot operating in an unknown structured environment. The proposed approach detects and segments the floor by computing plane normals from motion fields in image sequences. A floor is a useful object for mobile robots in structured environments, because it presents traversable paths if existing static or dynamic objects are removed effectively. In spite of this advantage, it cannot be easily detected from 2D image. In this paper, some geometric features observed in the scene and assumptions about images are exploited so that a plane normal can be employed as an effective clue to separate the floor from the scene. In order to use the plane normal, two methods are proposed and integrated with a designed iterative refinement process. Then, the floor can be accurately detected even when mismatched point correspondences are obtained. The results of preliminary experiments on real data demonstrate the effectiveness of the proposed methods.     

Map Building Fusing Acoustic and Visual Information using Autonomous Underwater Vehicles

We present a system for automatically building three‐dimensional (3‐D) maps of underwater terrain fusing visual data from a single camera with range data from multibeam sonar. The six‐degree‐of‐freedom location of the camera relative to the navigation frame is derived as part of the mapping process, as are the attitude offsets of the multibeam head and the onboard velocity sensor. The system uses pose graph optimization and the square root information smoothing and mapping framework to simultaneously solve for the robot's trajectory, the map, and the camera location in the robot's frame. Matched visual features are treated within the pose graph as images of 3‐D landmarks, while multibeam bathymetry submap matches are used to impose relative pose constraints linking robot poses from distinct tracklines of the dive trajectory. The navigation and mapping system presented works under a variety of deployment scenarios on robots with diverse sensor suites. The results of using the system to map the structure and the appearance of a section of coral reef are presented using data acquired by the Seabed autonomous underwater vehicle.     

3D object pose form clustering with multiple views

A candidate pose algorithm is described which computes object pose from an assumed correspondence between a pair of 2D image points and a pair of 3D model points. By computing many pose candidates actual object pose can usually be determined by detecting a cluster in the space of all candidates. Cluster space can receive candidate pose parameters from independent computations in different camera views. It is shown that use of of geometric constraint can be sufficient for reliable pose detection, but use of other knowledge, such as edge presence and type, can be easily added for increased efficiency.     

一种高精度鲁棒的基于直线对应的位姿估计迭代算法

2D-3D特征对应位姿估计问题算法多基于点对应,而基于直线对应求解比基于点对应求解更具有优势。从欧氏空间这一新颖角度出发,提出了一种从直线对应求解位姿估计问题的迭代算法,算法思想是先迭代求解出最优的旋转矩阵,然后再得到平移向量。针对不同的直线组情形,给出了相应的迭代初始值计算方法。仿真实验数据表明,算法具有对初始值较不敏感、高精度、鲁棒性好等特点。     

Pose Estimation using Point and Line Correspondences

The problem of a real-time pose estimation between a 3D scene and a single camera is a fundamental task in most 3D computer vision and robotics applications such as object tracking, visual servoing, and virtual reality. In this paper we present two fast methods for estimating the 3D pose using 2D to 3D point and line correspondences. The first method is based on the iterative use of a weak perspective camera model and forms a generalization of DeMenthon's method (1995) which consists of determining the pose from point correspondences. In this method the pose is iteratively improved with a weak perspective camera model and at convergence the computed pose corresponds to the perspective camera model. The second method is based on the iterative use of a paraperspective camera model which is a first order approximation of perspective. We describe in detail these two methods for both non-planar and planar objects. Experiments involving synthetic data as well as real range data indicate the feasibility and robustness of these two methods. We analyse the convergence of these methods and we conclude that the iterative paraperspective method has better convergence properties than the iterative weak perspective method. We also introduce a non-linear optimization method for solving the pose problem.     

Matching point features with ordered geometric, rigidity, anddisparity constraints

This correspondence presents a matching algorithm for obtaining feature point correspondences across images containing rigid objects undergoing different motions. First point features are detected using newly developed feature detectors. Then a variety of constraints are applied starting with simplest and following with more informed ones. First, an intensity-based matching algorithm is applied to the feature points to obtain unique point correspondences. This is followed by the application of a sequence of newly developed heuristic tests involving geometry, rigidity, and disparity. The geometric tests match two-dimensional geometrical relationships among the feature points, the rigidity test enforces the three dimensional rigidity of the object, and the disparity test ensures that no matched feature point in an image could be rematched with another feature, if reassigned another disparity value associated with another matched pair or an assumed match on the epipolar line. The computational complexity is proportional to the numbers of detected feature points in the two images. Experimental results with indoor and outdoor images are presented, which show that the algorithm yields only correct matches for scenes containing rigid objects     

基于ORB特征的单目视觉定位算法研究

针对大范围室外场景和具有重复、高频纹理特征(例如水泥地、草坪)的场景,提出了一种鲁棒性强、定位精度高、速度更快的视觉定位算法。采用8级图像金字塔的ORB (Oriented FAST and Rotated BRIEF)特征描述子提取图像特征点,通过K近邻(KNN)匹配相邻图像序列的特征点对,依次解算基础矩阵F和本质矩阵E,采用自适应法利用单应矩阵和本质矩阵进行位姿估计,最后解算两帧图像间相机刚体运动的旋转R和平移t,利用三角测量法则求解出匹配点的三维坐标,重建相机运动轨迹。为了提高算法性能,提出采用最小化基于点特征的非线性重投影误差优化三维点。通过调用OpenCV在C++中实现,对所采集的数据集进行测试,测试结果表明,该方法比传统的3D位姿估计更优,实时可行。由于其基于单目而实现,因此无法得到尺度信息。     

Inspection of an underwater structure using point‐cloud SLAM with an AUV and a laser scanner

This paper presents experimental results using a newly developed 3D underwater laser scanner mounted on an autonomous underwater vehicle (AUV) for real‐time simultaneous localization and mapping (SLAM). The algorithm consists of registering point clouds using a dual step procedure. First, a feature‐based coarse alignment is performed, which is then refined using iterative closest point. The robot position is estimated using an extended Kalman filter (EKF) that fuses the data coming from navigation sensors of the AUV. Moreover, the pose from where each point cloud was collected is also stored in the pose‐based EKF‐SLAM state vector. The results of the registration algorithm are used as constraint observations among the different poses within the state vector, solving the full‐SLAM problem. The method is demonstrated using the Girona 500 AUV, equipped with a laser scanner and inspecting a 3D sub‐sea infrastructure inside a water tank. Our results prove that it is possible to limit the navigation drift and deliver a consistent high‐accuracy 3D map of the inspected object.     

基于共面二点一线特征的单目视觉定位

研究了根据点、线混合特征进行单目视觉定位问题,在给定物体坐标系中共面的两个特征点和一条特征直线的条件下,根据它们在像平面上的对应计算相机与物体之间的位姿参数。根据三个特征之间的几何位置关系,分两种情况给出问题求解的具体过程,最终将问题转换成求解一个二次方程问题,真实的工件定位实验验证了方法的有效性。该结果为应用单目视觉进行工件定位提供了一种新方法。     

Under canopy light detection and ranging‐based autonomous navigation

This paper describes a light detection and ranging (LiDAR)‐based autonomous navigation system for an ultralightweight ground robot in agricultural fields. The system is designed for reliable navigation under cluttered canopies using only a 2D Hokuyo UTM‐30LX LiDAR sensor as the single source for perception. Its purpose is to ensure that the robot can navigate through rows of crops without damaging the plants in narrow row‐based and high‐leaf‐cover semistructured crop plantations, such as corn (Zea mays) and sorghum ( Sorghum bicolor). The key contribution of our work is a LiDAR‐based navigation algorithm capable of rejecting outlying measurements in the point cloud due to plants in adjacent rows, low‐hanging leaf cover or weeds. The algorithm addresses this challenge using a set of heuristics that are designed to filter out outlying measurements in a computationally efficient manner, and linear least squares are applied to estimate within‐row distance using the filtered data. Moreover, a crucial step is the estimate validation, which is achieved through a heuristic that grades and validates the fitted row‐lines based on current and previous information. The proposed LiDAR‐based perception subsystem has been extensively tested in production/breeding corn and sorghum fields. In such variety of highly cluttered real field environments, the robot logged more than 6 km of autonomous run in straight rows. These results demonstrate highly promising advances to LiDAR‐based navigation in realistic field environments for small under‐canopy robots.     

一种新的基于直线的定位方法

我们通常采用任意摆放位置的三条直线来完成三维物体定位. 对于交于两点的三条非共面直线这种配置, 它具有位置任意摆放三条直线不具有的一些特殊性质. 因此, 我们基于这种特殊配置直线提出一种确定物体位姿的新方法. 在理论上, 这种方法是对基于三条直线位姿估计方法的丰富, 同时这种方法也能为实际工程应用提供支持. 而且, 我们提出一种解决多解现象的方法和一种新的迭代方法. 最后, 仿真实验表明我们的算法具有较快的工作速度和较好的鲁棒性.     

基于RGB-D图像的室内机器人同时定位与地图构建

同时定位与地图构建（SLAM）是机器人在未知环境实现自主导航的关键技术,针对目前常用的RGB-D SLAM系统实时性差和精确度低的问题,提出一种新的RGB-D SLAM系统,以进一步提升实时性和精确度。首先,采用ORB算法检测图像特征点,并对提取的特征点采用基于四叉树的均匀化策略进行处理,并结合词袋模型（BoW）进行特征匹配。然后,在系统相机姿态初始值估计阶段,结合PnP和非线性优化方法为后端优化提供一个更接近最优值的初始值;在后端优化中,使用光束法平差（BA）对相机姿态初始值进行迭代优化,从而得到相机姿态的最优值。最后,根据相机姿态和每帧点云地图的对应关系,将所有的点云数据注册到同一个坐标系中,得到场景的稠密点云地图,并对点云地图利用八叉树进行递归式的压缩以得到一种用于机器人导航的三维地图。在TUM RGB-D数据集上,将构建的RGB-D SLAM同RGB-D SLAMv2、ORB-SLAM2系统进行了对比,实验结果表明所构建的RGB-D SLAM系统在实时性和精确度上的综合表现更优。     

基于RGB-D图像的室内机器人同时定位与地图构建

同时定位与地图构建（SLAM）是机器人在未知环境实现自主导航的关键技术，针对目前常用的RGB-D SLAM系统实时性差和精确度低的问题，提出一种新的RGB-D SLAM系统，以进一步提升实时性和精确度。首先，采用ORB算法检测图像特征点，并对提取的特征点采用基于四叉树的均匀化策略进行处理，并结合词袋模型（BoW）进行特征匹配。然后，在系统相机姿态初始值估计阶段，结合PnP和非线性优化方法为后端优化提供一个更接近最优值的初始值；在后端优化中，使用光束法平差（BA）对相机姿态初始值进行迭代优化，从而得到相机姿态的最优值。最后，根据相机姿态和每帧点云地图的对应关系，将所有的点云数据注册到同一个坐标系中，得到场景的稠密点云地图，并对点云地图利用八叉树进行递归式的压缩以得到一种用于机器人导航的三维地图。在TUM RGB-D数据集上，将构建的RGB-D SLAM同RGB-D SLAMv2、ORB-SLAM2系统进行了对比，实验结果表明所构建的RGB-D SLAM系统在实时性和精确度上的综合表现更优。     

Two efficient solutions for visual odometry using directional correspondence

This paper presents two new, efficient solutions to the two-view, relative pose problem from three image point correspondences and one common reference direction. This three-plus-one problem can be used either as a substitute for the classic five-point algorithm, using a vanishing point for the reference direction, or to make use of an inertial measurement unit commonly available on robots and mobile devices where the gravity vector becomes the reference direction. We provide a simple, closed-form solution and a solution based on algebraic geometry which offers numerical advantages. In addition, we introduce a new method for computing visual odometry with RANSAC and four point correspondences per hypothesis. In a set of real experiments, we demonstrate the power of our approach by comparing it to the five-point method in a hypothesize-and-test visual odometry setting.     

Dynamic omnidirectional vision for mobile robots

Mobile robotic devices hold great promise for a variety of applications in industry. A key step in the design of a mobile robot is to determine the navigation method for mobility control. The purpose of this paper is to describe a new algorithm for omnidirectional vision navigation. A prototype omnidirectional vision system and the implementation of the navigation techniques using this modern sensor and an advanced automatic image processor is described. The significance of this work is in the development of a new and novel approach—dynamic omnidirectional vision for mobile robots and autonomous guided vehicles.     

Rigidity checking of 3D point correspondences under perspectiveprojection

An algorithm is described which rapidly verifies the potential rigidity of three-dimensional point correspondences from a pair of two-dimensional views under perspective projection. The output of the algorithm is a simple yes or no answer to the question “Could these corresponding points from two views be the projection of a rigid configuration?” Potential applications include 3D object recognition from a single previous view and correspondence matching for stereo or motion over widely separated views. The rigidity checking problem is different from the structure-from-motion problem because it is often the case that two views cannot provide an accurate structure-from-motion estimate due to ambiguity and ill conditioning, whereas it is still possible to give an accurate yes/no answer to the rigidity question. Rigidity checking verifies point correspondences using 3D recovery equations as a matching condition. The proposed algorithm improves upon other methods that fall under this approach because it works with as few as six corresponding points under full perspective projection, handles correspondences from widely separated views, makes full use of the disparity of the correspondences, and is integrated with a linear algorithm for 3D recovery due to Kontsevich (1993). Results are given for experiments with synthetic and real image data. A complete implementation of this algorithm is being made publicly available     

基于VSLAM的自主移动机器人三维同时定位与地图构建

移动机器人在探索未知环境且没有外部参考系统的情况下,面临着同时定位和地图构建（SLAM）问题。针对基于特征的视觉SLAM（VSLAM）算法构建的稀疏地图不利于机器人应用的问题,提出一种基于八叉树结构的高效、紧凑的地图构建算法。首先,根据关键帧的位姿和深度数据,构建图像对应场景的点云地图;然后利用八叉树地图技术进行处理,构建出了适合于机器人应用的地图。将所提算法同RGB-D SLAM（RGB-Depth SLAM）算法、ElasticFusion算法和ORB-SLAM（Oriented FAST and Rotated BRIEF SLAM）算法通过权威数据集进行了对比实验,实验结果表明,所提算法具有较高的有效性、精度和鲁棒性。最后,搭建了自主移动机器人,将改进的VSLAM系统应用到移动机器人中,能够实时地完成自主避障和三维地图构建,解决稀疏地图无法用于避障和导航的问题。