Estimating human upper-extremity activities via force myography technique during collaborative tasks in human robot interactions

Force myography (FMG) is a non-invasive wearable technology that can detect underlying muscle volumetric changes when muscles contract. Common industrial physical human robot interaction (pHRI) tasks, such as object handling or transportation, mostly require hand forces to interact with machines. An FMG band made of force sensing resistors (FSRs) wrapped around an upper limb can be used to read muscle contractions during such activities. Including human feedback via FMG biosignals can be challenging yet practical in interactive pHRI environments. Therefore, the aim of this thesis was to investigate recognizing human intentions of interaction with a robot by estimating applied forces in dynamic motion using FMG technique. Initially in objective 1, real-time interactions with a 2-DoF linear robot (2D-pHRI platform) were investigated. Estimating interactive forces via intra-subject machine learning models was examined to manipulate the robot in any intended direction. In practice, a generalized (inter-subject) transfer learning model is preferable to recognize a new human worker instantly. Hence, in objective 2, domain adaptation and domain generalization were investigated using multiple source data collected over a long period (long-term data) from a 2D-pHRI platform. A few calibration (target training) data finetuned the model to quickly adapt outof-distribution, unseen (target test) 2D-pHRI data. A study was conducted in objective 3 to interact with a 7-DoF serial robot during a challenging 3D collaborative task. Crossdomain generalization demonstrated that a transfer learning model pretrained with the 2D-pHRI long-term multiple source domains could improve force estimations in the 3DpHRI platform. However, adequate and labeled data in practice is scarce. This was addressed in objective 4 by generating real-like synthetic FMG biosignals via domain randomization technique. By implementing a self-training technique, an unsupervised adversarial model pretrained with few labeled datasets and large amount of unlabeled synthetic data could estimate interaction forces during pHRI with a 7-DoF serial robot. Therefore, using force myography as the only bio feedback could improve daily HRI experiences using long-term source data, calibration data, or synthetic data- labeled or unlabeled for faster adaptations. In addition, FMG-based force estimation could enhance safe collaboration by avoiding unwanted contact or impact force from the manipulator. We believe these findings will contribute to the development of a discrete wearable FMG device for practical pHRI, rehabilitation, or prosthetic control applications.