A Gaze-based Attention System for Multi-human Multi-robot Interaction

This thesis presents a computer vision based attention system for interaction between multiple humans and multiple robots. The study contains three parts. In the first part each human can “select” (obtain the undivided attention of) a ground robot and interact with it by simply gazing (looking directly) at it. This extends previous work whereby a single human can select one or more robots from a population. In the second part, a feature which allows robots to be selected by people they cannot see is introduced. This is the first demonstration of many-to-many robot-selection HRI. Humans are detected, tracked, and allocated to robots partners in a two-step linear assignment problem. In the third part, we demonstrate the attention system with flying robots. The “micro-feedback” method which allows users to pre-select, select and de-select robots is introduced. This is the first demonstration of multiple UAV interacting with multiple human using face engagement.