Bayesian optimization strategies for human-in-the-loop systems: Theory and applications in physical human-robot-interaction

Physical human-robot interaction (pHRI) involves direct physical interaction betweenhumans and robots, sharing a workspace and performing tasks involvingtouch, force, and movement. Such interactions are essential for collaborative,surgical, wearable, and rehabilitation robots. Human-in-the-loop (HiL) studiesare widely used in pHRI to develop safer, more collaborative, efficient, and personalizedrobots. HiL studies gather quantitative and qualitative evaluationsbased on user experiences to iteratively improve robot performance and operation.However, data collected from participants in HiL optimization studies areoften limited due to fatigue and the limited attention span of users.This thesis proposes sample-efficient Bayesian strategies for HiL optimizationand explores their novel applications in various pHRI domains. In particular,this study (i) introduces Bayesian optimization (BO) based on qualitative userpreferences to optimize the perceived realism of haptic rendering under conflictinghaptic-visual cues. Then, it (ii) extends the Bayesian approach to computePareto solutions in HiL multi-criteria optimization problems with qualitativeand quantitative performance metrics to customize the assistance level during human motor learning tasks. Finally, (iii) it incorporates transfer learning to improvethe sample efficiency of HiL optimizations and demonstrates the improvedefficiency due to transfer learning during personalization of the assistance levelprovided to various upper-extremity movements using a rehabilitation robot.First, we develop a qualitative feedback-based HiL optimization technique usingsample-efficient Bayesian optimization that leverages qualitative pairwisecomparisons and ordinal classifications. We apply this technique to investigatevisual-haptic cue integration during multi-modal haptic rendering under conflictingcues, establishing a systematic approach to determine the optimal visualscaling for haptic manipulation to maximize the perceived realism of spring renderingfor a given haptic interface. Our results demonstrate that parameters affectingvisual-haptic congruency can be effectively optimized using our HiL optimizationtechnique, ensuring consistently high perceived realism. Consequently,multi-modal perception can be successfully enhanced by solely adjusting visualfeedback without altering haptic feedback, thereby extending the range of perceivedstiffness levels for a haptic interface. We extend our findings to a group ofindividuals to capture a multi-dimensional psychometric field that characterizesthe cumulative effect of feedback modalities utilized during sensory cue integrationunder conflicts.Next, we extend HiL studies with multi-criteria BO, addressing the need forconsidering multiple and often conflicting optimization objectives in pHRI applications.We propose a HiL Pareto optimization approach to characterize thetrade-off between the performance and the perceived challenge level of motorlearning tasks to be used in assist-as-needed control. Our optimization employsa hybrid model that captures the user’s performance by a quantitative metric,while the perceived challenge level is modeled as a qualitative metric gatheredthrough preference-based qualitative feedback. Once the trade-off is characterizedvia HiL Pareto optimization, we demonstrate how this trade-off can guidethe design of assist-as-needed training sessions of a motor learning task withoptimal assistance levels. Furthermore, we provide evidence that the trade-offevolves as learning takes place, and the set of non-dominated solutions of multicriteriaoptimization can be used to fairly evaluate the training progress, evenwhen the user cannot perform the task without assistance. We show the feasibilityand usefulness of the proposed approach through a case study involving avirtual manual skill training task administered to healthy individuals with hapticfeedback.Finally, we further enhance our HiL methods by incorporating transfer learningstrategies to accelerate the convergence of the optimization, by leveraging datafrom previous experiments. These strategies significantly improve sample efficiencyand reduce the need for extensive new data collection, addressing the impracticalityof conducting HiL trials from scratch for each new task in pHRI applications.We demonstrate the applicability of transfer learning to multi-criteriaBO through a HiL experiment conducted to personalize the assistance levels forrobot-assisted upper-extremity rehabilitation with a diverse range of physicalmovements. In these optimizations, we utilize a quantitative metric to evaluate user effort and a qualitative metric to assess the user’s perceived comfort. Oncean initial Pareto curve is computed for a task through HiL optimization for thepersonalized assistance levels, we transfer these outcomes to subsequent tasksby correlating the similarities among physical tasks. This systematic method oftransferring knowledge eliminates the need for empirical or sample-dependentcorrelation methodologies. Our results demonstrate that applying transfer learningcan significantly accelerate HiL optimizations, making the HiL experimentsmore efficient and effective.Overall, this thesis introduces novel BO strategies that can significantly broadenthe scope of HiL solutions in pHRI applications, enhancing their applicabilityand feasibility. By systematically demonstrating the proposed methodologiesacross multiple HiL-based pHRI studies, this work not only showcases their effectivenessbut also provides a comprehensive framework that can be adaptedfor HiL studies in various other pHRI fields. The insights and techniques presentedin this thesis serve as a valuable guide for future research and development,paving the way for more efficient, personalized, and effective pHRI.