Enhancing and understanding human cognition in human-robot collaboration using context-aware and user interface designs

Abstract

In recent years, robots have become vital to achieving manufacturing competitiveness. Especially in industrial environments, a strong level of interaction is reached when humans and robots form a dynamic system that works together toward achieving a common goal or accomplishing a task. However, human-robot collaboration can be cognitively demanding, potentially contributing to unintended consequences such as high levels of cognitive workload. Considering cognitive workload becomes particularly important to ensure efficiency in the overall human-robot collaboration and prevent cognitive overburden. This dissertation focuses on two design approaches, context-awareness and user interfaces, to understand and enhance human cognition and to improve task performance in human-robot collaboration. The first step toward context-aware robotics was to develop a machine learning model to detect cognitive workload through physiological data monitoring. Unimodal and multimodal feature sets were examined to determine the model's robustness and accuracy. The machine learning models utilized included a wide range of linear (i.e., Support Vector Machine, Logistic Regression) and nonlinear (i.e, Random Forest, AdaBoost) machine learning models. Each model was trained and evaluated on the unimodal and multimodal feature sets, and the developed model demonstrated superior performance in detecting cognitive workload compared to existing approaches. After successfully creating a machine learning model for cognitive workload detection, our research focused on developing a real-time machine learning framework to detect cognitive workload levels and enable robot adaptation by slowing it down when high levels of cognitive workload were detected. Our goal was to design robotic systems that can address the challenges of high cognitive workload, facilitate cognitive workload recovery, and enhance task performance by adjusting the robot's speed when needed. The real-time machine learning framework demonstrated a clear connection between detecting and adjusting cognitive workload and task performance. This finding motivated us to create a reliable method for enhancing task performance in human-robot collaboration. Drawing inspiration from psychology and the interplay between cognitive workload and emotions, we developed a hierarchical machine learning approach to predict task performance. This work aimed to establish a method to optimize human-robot collaboration by considering multiple factors crucial to task performance. Our second design approach aimed to understand human cognition in human-robot collaboration using user interfaces. We developed user interfaces in both 2-D displays and mixed reality. These interfaces allowed us to explore the potential of mixed reality and 2-D user interfaces in assisting and enhancing human-robot collaboration while also examining their impact on cognitive workload development. We focused on understanding the relationship between user interface utilization, cognitive workload, and task performance. This work aimed to gain insights into the potential benefits and challenges associated with these user interface technologies and uncover how these user interfaces could facilitate communication and information exchange between humans and robots, ultimately leading to improved collaboration outcomes.