Handsight

Handsight offers a suite of mobile services for the blind and visually impaired population. With the Handsight service, a user takes a photo with a mobile phone and receives a spoken response about the image within seconds. This project is funded in part by the NIH / National Eye Institute grant R44 EY020707.

The first Handsight app, Text Detective, is available on the iTunes App Store and Google Play now.

Features

Description

Handsight is a mobile phone service that offers an affordable, extensible set of automated sight-assistant functions to millions of blind and low-vision persons. To the user, Handsight is simply a mobile phone application, requiring no special equipment or updating. The application uses the built in camera to capture the area the phone is pointed at. Our patented text detection algorithm analyzes the video stream and tells the user whether there is text or not. Optical Character Recognition (OCR) and Text-To-Speech software will do the rest and speak the text that was detected – all within seconds.

Key functions
  • Sign finder/recognizer
  • Currency recognition
  • Barcode recognition & product information
  • Color recognition
  • Text translation
  • Face recognition
Use cases

The range of tasks encountered by the blind and low-vision requires an array of components to solve: some are more heavily text-oriented and involve no location/navigational feedback (distinguishing and identifying different medicines; finding the phone bill in a stack of letters); some are more specifically navigational (locating the exact store entrance, finding the bus stop); yet others are informational (finding out when the next bus is due). Since we aim to provide a broadly useful tool, Handsight has to accommodate the whole range of task types. We are therefore proposing to build an architecture that integrates a set of components addressing the various task types, from text-detection and recognition software to navigational databases.

Customers

In the US alone, there are at least 2.4 million blind and moderate-to-severely low-vision individuals, many of who are elderly, and some of who suffer multiple impairments. Blindsight’s objective is to improve the lives of these individuals by increasing their mobility and independence. Its primary target is the blind and low-vision population, but Blindsight expects that the technology developed in this project will carry over to other populations as well, particularly the elderly. Currently, there are no general-purpose solutions to help such individuals easily complete the key daily tasks necessary to live independently.