EnPower

For deafblind individuals, the absence of visual and auditory communication channels prevents meaningful interactions with the people and world around them, leading many to suffer from both mental and social issues. Among the remaining sensory channels (e.g., taste, haptic, and olfactory) of deafblind individuals, haptic stimuli are an effective medium that can be digitally employed to enable interactions with the outside world. EnPower (Enable and Empower individuals with auditory and visual impairments) enables bi-directional communication between deafblind and non-impaired individuals by using sensory substitution techniques to present information via accessible haptic stimuli. This platform consists of a physical interface (in wearable and desktop versions) that is paired with a wireless mobile application. Similar to concepts such as Finger Braille, the assistive devices can deliver textual information via tactile stimuli to the deafblind individual, and vice versa. Additionally, the system can translate speech and visual input into tactile stimuli, providing users with greater access to digital information platforms. Using the two devices, a stimuli perception study, as well as a field trial, was conducted to evaluate the effectiveness of our approach (delivering information via tactile sensations on fingertips). Findings not only suggest that participants could understand the information presented via tactile sensations but also reveal several important avenues for future research.

Keywords: Haptic; tactile; sensory substitution; sensory impaired; finger braille.

Gallery

Publications

  • Ranasinghe, N., Jain, P., Tolley, D., Chew, B., Bansal, A., Karwita, S., Yen, C. C., & Do, E. Y. L. 2020. EnPower: Haptic Interfaces for Deafblind Individuals to Interact, Communicate, and Entertain. In Proceedings of the Future Technologies Conference (Virtual Event, Nov 5 – 6, 2020). FTC ’20. Springer, Cham. 

  • Ranasinghe, N., Jain, P., Karwita, S., Tolley, D., Chew, B., Bansal, A., & Do, E. Y. L. 2017. Enriching the Lives of Deafblind Individuals Through Sensory Substitution (Interfaces to Interact, Communicate, and Entertain). In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’17) – Workshop on Amplification and Augmentation of Human Perception (Denver, Colorado, May 7, 2017). [email protected] ’17. ACM, New York, NY. 

Patents

  • Ranasinghe, N., Jain, P., Karwita, S., Tolley, D., Chew, B., Bansal, A., & Do, E. Y. L. 2017. Interface For Information Exchange And Communication Between And Among The Sensory Disabled And Non-Disabled Individuals. SG Non-Provisional Application No. 10201703584W.

Grants

Exhibitions / Demonstrations

  • InnovFest unBound 2017 / “Tactile communication tools for the deafblind” at Marina Bay Sands Convention Centre, Singapore / May 2017.

Acknowledgements

This research is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its International Research Centres in Singapore Funding Initiative.