Generalizing Upper Limb Force Modeling With Transfer Learning: A Multimodal Approach Using EMG and IMU for New Users and Conditions

oleh: Gelareh Hajian, Evan Campbell, Mahdi Ansari, Evelyn Morin, Ali Etemad, Kevin Englehart, Erik Scheme

Format: Article
Diterbitkan: IEEE 2024-01-01

Deskripsi

In the field of EMG-based force modeling, the ability to generalize models across individuals could play a significant role in its adoption across a range of applications, including assistive devices, robotic and rehabilitation devices. However, current studies have predominately focused on intra-subject modeling, largely neglecting the burden of end-user data acquisition. In this work, we propose the use of transfer learning (TL) to generalize force modeling to a new user by first establishing a baseline model trained using other users&#x2019; data, and then adapting to the end-user using a small amount of new data (only <inline-formula> <tex-math notation="LaTeX">${10}\%$ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">${20}\%$ </tex-math></inline-formula>, and <inline-formula> <tex-math notation="LaTeX">${40}\%$ </tex-math></inline-formula> of the new user data). Using a deep multimodal convolutional neural network, consisting of two CNN models, one with high-density (HD) EMG and one with motion data recorded by an Inertial Measurement Unit (IMU), our proposed TL technique significantly improved force modeling compared to leave-one-subject-out (LOSO) and even intra-subject scenarios. The TL approach increased the average R squared values of the force modeling task by 60.81&#x0025;, 190.53&#x0025;, and 199.79&#x0025; compared to the LOSO case, and by 13.4&#x0025;, 36.88&#x0025;, and 45.51&#x0025; compared to the intra-subject case for isotonic, isokinetic and dynamic conditions, respectively. These results show that it is possible to adapt to a new user with minimal data while improving performance significantly compared to the intra-subject scenario. We also show that TL can be used to generalize on a new experimental condition for a new user.