Hands-Free Smartphone For Physically Challenged

Abstract

Disability is one of the major problems of the contemporary world. There are new technical means, methods of work and education for people with disabilities, making it possible to use a smart phone. The aim of this work is to allow people with disabilities to use android phones by simple eye gestures and speech input. The system combines techniques from image processing, computer vision and pattern recognition to detect gestures in the video recorded using the built in front-facing camera and also speech recognition and processing techniques. The existing gesture recognition system runs entirely on an Android based mobile phone without the need for any additional equipment. The system was implemented using the OpenCV computer vision library. Input to the system is a raw video stream from the device’s front-facing camera. Because accurate point of- gaze estimation is not required for gesture recognition, these images are scaled down to reduce processing time and thus increase system performance.