AI for good: We love this computer vision system for the visually impaired

Users were able to navigate 25% faster with this system than using a traditional cane.

Hats off to the research team at Shanghai Jiao Tong University in China led by Leilei Gu. They published a fascinating paper in Nature Machine Intelligence today describing an “AI-boosted” camera system designed to help visually impaired persons navigate the world. 

The new system combines computer vision (AI that interprets a camera’s data feed) with innovative new hardware featuring audio alerts and vibrations. 

Per the paper: 

“We present a human-centred, multimodal wearable system that advances usability by blending software and hardware innovations. … For hardware, we improve the wearability by developing stretchable sensory-motor artificial skins to complement the audio feedback and visual tasks.”

AI-assisted vision

It’s 2025 and humanity is on the cusp of putting human boots on Martian soil, declaring a winner in the AGI race, and giving people telekinetic control over their iPads and PlayStations with brain-machine-interfaces. But, for the visually impaired, the top tech for navigating the world is still a good old-fashioned cane. 

That’s not to say there aren’t myriad solutions in various stages of development. Virtual reality systems have been adapted for the blind and companies such as Microsoft have entire divisions devoted to accessibility hardware and software. But few of these solutions are as elegant as the one developed by Gu’s team. 

Image source: Tang et al., Nature Machine Intelligence

Better outcomes

As shown above, the hardware is relatively simple. The module, currently mounted on semi-stylish sunglasses, combines a camera array with a “second skin” vibration system. Essentially, the cameras detect obstacles and output audio alerts and vibrations to indicate the proximity, distance, and direction of objects and impedances. 

The system was tested by 12 participants with visual impairments who were tasked with navigating through an enclosed path with obstacles such as bicycles and boxes in the way. 

Overall, the researchers reported a 25% increase in course completion speed for users wearing the camera system over those using traditional canes. That might not seem like much, but shaving 15 minutes off of an hour-long journey, for example, could provide a significant quality of life increase for those using the system. 

Furthermore, there’s reason to believe that both the system and the user would demonstrate improved reactions and performance over time through consistent use. 

According to the researchers, the device shown in the image above is just a prototype. They expect to make future iterations smaller, lighter, more comfortable, and perhaps even more stylish. 

Our take: bravo! This is something that could immediately help improve the lives of billions of people currently living with a vision impairment. This is the kind of AI research we can really get behind, even if it has nothing to do with artificial general intelligence. 

The only question we have is how can we, as a global society, make technology like this accessible to everyone who needs it, regardless of their economic status? 

Read next: Should you be polite to AI?

Art by Nicole Greene