Sound has long been a substitute for sight in visual aids for the blind. From techniques like echolocation, the researchers of the California Institute of Technology in Pasadena have advanced the field a step forward with the creation of smart glasses that translate images into sounds that can be intuitively understood without training.
The device, called vOICe (OIC stands for “Oh! I See”), is a pair of dark glasses with an attached camera, connected to a computer which uses an algorithm of the same name developed in 1992 by Dutch engineer Peter Meijer. The system works by converting pixels in the camera’s video feed into sound, mapping brightness and vertical location to an associated pitch and volume. The pixels situated low in the picture and dark correspond to a low sound and quite pitch, while the opposite is applicable for the bright pixels at the top of the frame.
“You’re taking something from vision and you’re putting it into audition,” says Caltech’s Noelle Stiles, who works on vOICe. “Your brain is doing the opposite – it’s taking in all of the sounds and it’s making sense of them visually.”
[adinserter block=”7″]
Stiles and her colleague Shinsuke Shimojo worked on the algorithm by asking a group of people to select images matching with certain sound. Visually impaired individuals were asked to select a sound for a given image or object. During final testing with the device, blind people with no experience of using it were able to match the shapes to the sounds as often as those who had been trained – both groups performed 33 per cent better than by chance.
“The result that select natural stimuli could be intuitive with sensory substitution, with or without training, was unexpected,” the researchers write.
“This research shows it’s not just important how much information you provide, but whether you provide it in a way that the person can intuitively make sense of,” says Ione Fine of the University of Washington in Seattle.
“They are basically saying that the magic bullet is going to be finding an intuitive mapping system and not relying on training,” she says.
Author: Technology Blog
