Vibrating 'smart' glove summarises conversations for deafblind people

Thousands of people who were born deafblind could gain a new way to engage in conversation thanks to a device in development at Nottingham Trent University (NTU).

The prototype glove, which uses AI technology to interpret conversations and haptic actuators that vibrate on the wearer’s fingers to transmit summaries, is designed to aid communication in a variety of situations.

Deafblindness is “the loss of sight and hearing to the point where… communication, mobility and ability to access information are impacted,” according to charity DeafblindUK. Some people have it from birth, while it is caused by health conditions, illnesses or ageing for others.

More than 450,000 people in the UK were deafblind in 2017, according to disability charity Sense, with that number forecast to increase to over 600,000 by 2030 due to the ageing population.

Many people with the condition are able to hear or see something, but the new wearable is designed for people who cannot hear or lipread.

Tiny actuators in the glove’s fingers vibrate to relay words, numbers and grammar to the wearer using code, similar to the raised dots of the braille alphabet. Sitting below the knuckles, the actuators will vibrate at different amplitudes, frequencies and for varying lengths of time, which an NTU announcement said would allow for “numerous communications possibilities”. These could include transmitting spoken directions, phone notifications, fire alarms and doorbells, and interpretations of music or visual art.

“Thousands of people who are born deafblind could, for the first time ever, be able to understand live conversations thanks to the development of this new technology,” said Dr Hughes-Riley, an associate professor in NTU’s Advanced Textiles Research Group.

“The AI, when activated, will listen to the spoken word on the wearer’s behalf and provide a summary in real time of what’s being said. Combining this technology with a pair of wearable gloves allows people to integrate it into their everyday lives, which will bring about enormous benefits for them.

“This could include things that many people take for granted, such as understanding what other people are saying in conversation, hearing announcements at train stations or even enjoying an interpretation of music.”

Researcher Malindu Ehelagasthenna, who is undertaking a PhD as part of the study, first suggested that textile-embedded haptics could be used to help people in this way. “This technology has enormous potential to enrich the lives of thousands of people around the world who cannot see or hear and who are underrepresented,” he said.

“As well as supporting people who are completely deafblind, this technology may have vast benefits for people who are partially sighted and have partial hearing, who may otherwise have difficulties communicating, accessing information and navigating their immediate environment.”

The haptic glove was described in a recent paper published by IEEE.