Google's AI AR Project Guideline Enables Solo Running for the Blind

Imagine navigating through complete darkness, where every step is guided only by the uncertain sensations beneath your feet. Each movement is shrouded in mystery, and what lies ahead is unpredictable. For most of us, this fear can be dispelled with a deep breath and the opening of our eyes. However, for individuals who are legally blind or visually impaired, this is the reality they face when running. Thankfully, technological advancements have brought a glimmer of hope in democratizing technology for the visually impaired.

Google has launched a google project called 'Guideline', which utilizes AI deep learning, AR technology, and voice feedback to enable individuals who are legally blind to enjoy the thrill of running without the need for a guide runner. This summer, an experiential event will be held in Yokohama, Japan, inviting those who are visually impaired to participate.

How Does It Work?

Let's rewind to 2019. Thomas Panek, CEO of 'Guiding Eyes for the Blind', posed a challenge to engineers at a Google hackathon: "Would it be possible to help guide a blind runner, independently?" That day, engineers came up with a prototype. In 2020, Google Research officially announced the project and has been refining it ever since, leading to its open-source release in 2023.

The concept is simple: a correct running path is established, and a smartphone worn at the waist helps the runner stay on track. Voice prompts through headphones inform them if they are on the right path or if they have veered off course.

Project Guideline can identify colored lines on the ground and determine whether the line is to the left, right, or center of the runner. In a way, the smartphone at the user's waist becomes their eyes, providing AR navigation for the visually impaired.

Achieving this was no small feat. Running is dynamic, and the ever-changing road conditions add complexity to the environment. The core technology of Project Guideline is a deep learning model based on the DeepLabV3+ framework, which can distinguish between different objects in an image, such as people, vehicles, and roads. In this project, it's used to identify guide lines and detect obstacles.

In addition to AI for blind individuals, the project also incorporates Google's ARCore for spatial awareness and positioning to assist with navigation, topped off with a low-latency voice system to inform the visually impaired runner.

Still Limited by Hardware and Environment

"I felt free, like I was effortlessly running through the clouds," Thomas Panek remarked after completing a run independently with the help of Project Guideline. He later participated in a 5K run in New York's Central Park. In 2022, Project Guideline also enabled a group of legally blind runners in Japan to participate in a virtual relay race, where six runners completed the 42-kilometer race independently in 4 hours and 29 minutes.

Thomas Panek is using Project Guideline with a mobile phone and headphones. (Source: Google)

While Project Guideline has proven its viability and seems ideal, it still faces several challenges:

1. It requires pre-marked paths, limiting its use to specific locations.
2. There are technical and hardware requirements; the development of 'Guideline' has been focused on Google's own Pixel phones and headphones. Although it's now open-source, porting it to other brands will take time and further optimization.
3. Wearing a smartphone at the waist for ARCore to view the path may not be the most comfortable or convenient method.

Despite these limitations, the ability to run independently is no longer just a dream for individuals who are legally blind. According to statistics collected in 2021, there are 60,000 people with visual impairments in Taiwan, but less than 50 guide dogs, making this project a blessing for a region with scarce resources for the visually impaired. Looking ahead, we hope the project's technology will continue to expand. For instance, integrating smart glasses to replace the smartphone camera could be a more ideal scenario, especially for those who are accustomed to wearing sunglasses or those with low vision who require glasses. We look forward to this project democratizing technology, allowing everyone to enjoy a life of freedom and ease.