AI-Driven Gloves With Ability To ‘Feel’ Helps Stroke Patients Relearn To Play The Piano

Alex Dolce

For individuals who have endured the hardships of a debilitating stroke, the simplest tasks, like tying shoelaces or brushing teeth, can transform into major challenges.

This struggle becomes even more poignant when such individuals derive immense joy from life through their ability to play the piano.

However, a glimmer of hope now emerges in the form of a specially crafted robotic glove designed explicitly for piano playing. Developed at Florida Atlantic University, this ingenious device harnesses the power of artificial intelligence (AI) to enable pianists to experience some of the sensations necessary for their musical performance.

The Florida Atlantic University press wrote in an announcement article, “Combining flexible tactile sensors, soft actuators, and AI, this robotic glove is the first to “feel” the difference between correct and incorrect versions of the same song and to combine these features into a single hand exoskeleton.”

Distinguishing itself from conventional prosthetics, this innovation resembles more of a robotic garment than a mechanized limb or organ. It integrates advanced polysynthetic fibers and hydrogel materials to encompass five strategically placed actuators that snugly fit onto a person’s head.

The AI-driven capabilities of the glove facilitate the coordination of the fingers, with a keen focus on identifying twelve distinct types of errors that may occur while striking piano keys. These errors range from striking a note too forcefully to holding it for an excessive duration.

Erik Engeberg, Ph.D., senior author, and a professor in FAU’s Department of Ocean and Mechanical Engineering, said “Playing the piano requires complex and highly skilled movements, and relearning tasks involves the restoration and retraining of specific movements or skills.”

“Importantly, although this study’s application was for playing a song, the approach could be applied to myriad tasks of daily life and the device could facilitate intricate rehabilitation programs customized for each patient,” he added.

The study results of the comprehensive study, encompassing three distinct deep-learning algorithms, yielded remarkable results. In the presence of a human subject, the glove showed an impressive 97% accuracy in playing the classic tune “Mary Had a Little Lamb.” Incredibly, even without direct human intervention, it maintained an impressive 94% accuracy.

The engineers behind this robotic glove marvel proposed that clinicians can use the data from both the mistakes and successes in a patient’s weakness in a particular song. Theoretically, the glove is actually capable of much greater complexity than nursery rhymes, says the engineers.

To learn more about this, take a look at the video below: