Assistive Communication Tools for the Sensory Impaired

Sensory impaired individuals face many challenges in their daily lives that can prevent them from accessing information and maintaining a healthy level of independence, social interaction and well-being. For some individuals, this can be due to inaccessibility to caregivers or expensive assistive technology, while for other individuals the required assistive technology simply doesn’t exist. For deafblind individuals, the absence of both visual and auditory communication channels can prevent meaningful interactions with the people and world around them, leading many to suffer with both mental and social issues.

We have developed a novel technology platform that utilises haptic stimuli to help support people with visual and auditory impairments. The proposed platform has two key components:

  • A computer-mediated device that extends the Finger Braille technique
  • A partnering mobile software application

In partnership with the device, the respective mobile software handles and performs the conversion of data between different modal stimuli (including image-to-speech (auditory), image-to-braille (tactile), image-to-text (visual), speech-to-braille (tactile), speech-to-text (visual), braille-to-speech (auditory), and braille-to-text (visual). By combining these two components, this platform provides users with an interface that can be used to support effective communication and that can enable improved accessibility to other forms of information interaction.

Keywords: Haptic; tactile; sensory substitution; sensory impaired; finger braille.

Gallery

Publications

  • Ranasinghe, N., Jain, P., Karwita, S., Tolley, D., Chew, B., Bansal, A., & Do, E. Y. L. 2017. Enriching the Lives of Deafblind Individuals Through Sensory Substitution (Interfaces to Interact, Communicate, and Entertain). In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’17) – Workshop on Amplification and Augmentation of Human Perception (Denver, Colorado, May 7, 2017). [email protected] ’17. ACM, New York, NY. 

Patents

  • Ranasinghe, N., Jain, P., Karwita, S., Tolley, D., Chew, B., Bansal, A., & Do, E. Y. L. 2017. Interface For Information Exchange And Communication Between And Among The Sensory Disabled And Non-Disabled Individuals. SG Non-Provisional Application No. 10201703584W.

Grants

Exhibitions / Demonstrations

  • InnovFest unBound 2017 / “Tactile communication tools for the deafblind” at Marina Bay Sands Convention Centre, Singapore / May 2017.

Acknowledgements

This research is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its International Research Centres in Singapore Funding Initiative.