Advances in robot attention for next-best view planning

Abstract

This dissertation presents advancements in next-best view algorithms applied to 3D reconstruction. 3D reconstruction has been a topic of great interest in recent years, due to the diffusion of cheap depth sensors such as the Microsoft Kinect. Algorithms such as KinectFusion have been developed to merge multiple views from these sensors. The experimental setup used in this thesis involves a Kinect sensor mounted on a robot arm. The robot can move the sensor in order to reconstruct the objects in a tabletop scenario. Next-best view algorithms compute the optimal sensor placement to observe the unknown space. Such algorithms are computationally expensive, since they need to simulate the sensor from each candidate pose. In this thesis, the concept of attention is applied to next-best view, in order to reduce the computation time and concentrate robot exploration towards a goal. According to attention, a vision system assigns different importance to various parts of a scene. Resources can then be focused on the most important parts of the scene. In the first approach presented in this dissertation, attention is driven by the user, who moves objects in the tabletop scenario. Then, the robot updates the 3D representation, by focusing views in the areas where changes have happened. In the second approach, the robot is attracted by the objects already present in the scene, as they are discovered during exploration. A segmentation algorithm is used in order to partition the scene. A saliency value is assigned to each segment, and the most salient one is selected. In both approaches, experiments were carried out in order to highlight the reduced computation time and the goal-oriented behavior of the robot. Further work in this thesis proposes advancements towards the use of the ElasticFusion reconstruction algorithm. Unlike KinectFusion, ElasticFusion operates on a surfel-based 3D representation. Surfels are designed for computer graphics, and they are processed faster on GPU than KinectFusion volumetric representation. A color and position enhancement filter is proposed, to be run alongside ElasticFusion 3D reconstruction, in order to obtain a better segmentation.