Handsight™

Fast wireless networks and mobile phones with built-in high-quality cameras together enable a new class of technology aid for the Blind and Low-Vision: sight assistance as a subscription service. Users will need only a suitable mobile phone with built-in camera and a subscription to the Handsight service.

This project is funded in part by the NIH / National Eye Institute grant R44 EY020707.

Features

Description

Handsight is a mobile phone service that offers an affordable, extensible set of automated sight-assistant functions to millions of blind and low-vision persons. To the user, Handsight is simply a mobile phone application, requiring no special equipment or updating. By aiming a mobile telephoneʼs camera in roughly the right direction and pressing just one button, a Handsight user can snap a picture, send it to the Handsight computer center along with location data, and have a response within seconds.

Moving the key computation and data from the handset to web servers cuts cost, eases technology upgrades, and enables numerous data and technology partnerships needed to rapidly solve a broad spectrum of tasks. To allow widespread adoption Handsight uses voice, keypad, and vibration for user interaction; and for extensibility it is built on open-source software both on the handset and on the web application infrastructure.
Key functions
  • Sign finder/recognizer
  • Currency recognition
  • Barcode recognition & product information
  • Color recognition
  • Text translation
  • Face recognition
Use cases

The range of tasks encountered by the blind and low-vision requires an array of components to solve: some are more heavily text-oriented and involve no location/navigational feedback (distinguishing and identifying different medicines; finding the phone bill in a stack of letters); some are more specifically navigational (locating the exact store entrance, finding the bus stop); yet others are informational (finding out when the next bus is due). Since we aim to provide a broadly useful tool, Handsight has to accommodate the whole range of task types. We are therefore proposing to build an architecture that integrates a set of components addressing the various task types, from text-detection and recognition software to navigational databases.

Customers
In the US alone, there are at least 2.4 million blind and moderate-to-severely low-vision individuals, many of who are elderly, and some of who suffer multiple impairments. Blindsightʼs objective is to improve the lives of these individuals by increasing their mobility and independence. Its primary target is the blind and low-vision population, but Blindsight expects that the technology developed in this project will carry over to other populations as well, particularly the elderly. Currently, there are no general-purpose solutions to help such individuals easily complete the key daily tasks necessary to live independently.