If you're blind, how do you tell the difference between a box of laundry detergent and a box of cereal? True, they're in different aisles of the supermarket. But what about boxes on the same shelf: How do you figure out which holds corn flakes and which holds sugar-coated chocolate breakfast crunchies?
 |
If only you could point a handheld device at the shelves and have it tell you what it sees. This product could just be The Speaking Eye from break-step productions, an ordinary Sony-Ericsson P900 mobile phone running some extraordinary software.
The key is break-step's Foveola shape recognition software. The product of 10 years of research, Foveola can see general 2D shapes in the same way people do. Foveola is modeled on the behavior of cells in our visual cortex. "Individually, these cortical cells are not very bright," says break-step's managing director, Patrick Andrews, "but working together they're very clever."
Along the same line of reasoning, Foveola's underlying algorithm is implemented as a sequential set of simple operations that when combined can recognize and store not just characters, but all 2D shapes such as logos, labels, and object outlines. (In other words, it can recognize the rooster on the box of corn flakes and the chocolate bomb on the crunchies.) Because the Foveola engine is a platform-independent ANSI C library with its own database, it can run on very small hardware, such as the P900 phone or a laptop. "It's designed to slot into anything with a camera and imaging device that runs C," says Andrews. If application-specific shapes are needed, developers can add to the database using a GUI.
For the Speaking Eye, Britain's break-step is combining Foveola with text-to-speech software developed in collaboration with Canada's Compusult. The two companies are also working to develop what Andrews describes as "a meaningful user specification for what the blind need, not just what we can give them." One goal is to provide feedback in real time. Point the device at the kitchen cupboard, and it will tell you what's on the shelf in time for you to add the right ingredient to the pot.
Beyond Speaking Eye, Andrews sees the speed and flexibility of Foveola as enabling a whole range of robust, visually interactive products: robot pets that scamper around the house locating and retrieving objects for their owners; robots that circulate through a library finding and reshelving books left behind on tables; warehouse security gates that activate an alarm when an engineering component that should not be removed passes through the gate. Even Honda Research is putting the software to work.
If you have ideas of your own for Foveola, you can download the free Evaluation Edition or the $750 Development Edition from www.foveola.com.
Contact Patrick Andrews, break-step productions, Cambridge, England; +44 845 2255 169, [email protected], www.foveola.com.