Investigation on Visuo-Haptic Augmented Reality for Robotic Applications

Abstract

The aim of this thesis is the investigation of the potential of Visuo-Haptic Augmented Reality (AR) technologies for robot applications. Although the research activity on AR has been active for several years, the development of systems including the augmentation of more than one sense simultaneously needs more investigation. Visuo-Haptic Augmented Reality (VHAR) is a technique that augments the sight sense (Visuo) and the touch sense (Haptic). This approach allows a system to be more immersive than solutions augmenting only one sense. This kind of augmentation can be used to generate highly immersive applications, enabling a user to receive artificial inputs through two different senses at the same time. This investigation aspires to highlight the benefits of VHAR interfaces for human-robot interaction, enabling a user to interact with a real-world robot in a more intuitive way to perform a task. Often human-robot interaction, especially in industrial scenarios, requires the user to know the technical details of, at least, the robot platform. To achieve this level of knowledge, if the user is not an expert of robotics or if he/she is not a highly skilled operator, a specific training phase is needed. The use of VHAR interface may reduce the need for training because it does not require such level of technical knowledge about the robotic system in use. VHAR interfaces have been used for human-machine interaction, but the literature on the use of these interfaces for, specifically, human-robot interaction is quite limited. Furthermore, this work aims to extend the state of the art about VHAR interfaces for robot teleoperation, which includes even fewer contributions. The interaction with a robot can be tedious, frustrating, and can cause mental overload. In this work we show that this kind of interface provides a valid and reliable way to enhance the user experience, enabling the operator to feel the execution of the task less artificial and more natural. We also show that VHAR interfaces can be applied in very different application contexts. Indeed we adopt VHAR interfaces for human-robot interaction in scenarios in which the robots greatly differ but the benefits derived from the use are of the same nature. Enhancement of the user experience, higher level of immersion, and lower workload on the user are general results obtained across the different scenarios. A generic human-robot interaction involves different robots and different environments. Moreover, the typology of the tasks greatly varies. Hence, to make this investigation more generic, the work in this thesis has addressed two quite different applications: the first is a VHAR interface for an indoor scenario in which a robotic manipulator has to learn how to perform pick-and-place tasks; the second is a VHAR interface for the teleoperation of a radioactivity-detector equipped UAV in outdoor environment, using a novel sensor-based haptic teleoperation scheme. The results obtained show the importance of VHAR interfaces in two heterogeneous and innovative application fields.