A mobile app gives deaf people a sign language interpreter they can take anywhere

Hand Talk, created by Brazilian-based Ronaldo Tenório, is a smartphone app developed to improve social interaction and facilitate independence for deaf mobile users. This app presents an animated avatar named Hugo who converts speech into sign language, acting as a personal sign language interpreter for deaf mobile users. The users hold up their smartphone to a hearing person who sees a message on the screen that says “Speak to translate. “As soon as the person starts talking, Hugo starts signing.  Turning the audio into animations of gestures requires laborious programming due to the precise nature of signing which includes facial expressions. To aid the clarity of the signing and demonstrate the precise signs, Hugo’s hands and face are oversized.

Tenório and his team continue to improve the program as thousands of example sentences are inputted every month and matched to 3-D animations of sign language. Additionally, Tenório plans to develop different versions of the avatar so that users can change the gender or race of their “Hugo” in an effort to broaden the appeal and accessibility of the virtual translator.

Furthermore, Hand Talk won the 2015 Accessible Mobile Applications Contest, an ITU Regional Competition for the Americas, which judged creativity, development and user experience of the app. Although Hand Talk currently only translates Portuguese into Libras, the sign language used in Brazil, it has already been downloaded by one million people and will likely pave the way for other similar speech to sign applications.

Additional Sources: Technology Review