Walk this way: How Oko leverages AI to make street crossings more accessible

Oko is simply a testament to the powerfulness of simplicity.

The 2024 Apple Design Award victor for Inclusivity and 2024 App Store Award victor for Cultural Impact leverages Artificial Intelligence to assistance unsighted oregon low-vision radical navigate pedestrian walkways by alerting them to the authorities of signals — “Walk,” “Don’t Walk,” and the similar — done haptic, audio, and ocular feedback. The app instantly affords much assurance to its users. Its bare-bones UI masks a almighty blend of ocular and AI tools nether the hood. And it’s an particularly awesome accomplishment for a squad that had nary iOS oregon Swift improvement acquisition earlier launch.

“The biggest feedback we get is, ‘It’s truthful simple, there’s thing analyzable astir it,’ and that’s large to hear,” says Vincent Janssen, 1 of Oko’s 3 Belgium-based founders. “But we designed it that mode due to the fact that that’s what we knew however to do. It conscionable happened to besides beryllium the close thing.”

ADA FACT SHEET

From left: Willem Van de Mierop, Michiel Janssen, and Vincent Janssen are the 3 cofounders of Oko. The app’s sanction means “eye.”

Oko

• Winner: Inclusivity
• Team: AYES BV
• Available on: iPhone
• Team size: 6
• Previous accolades: 2024 App Store Award victor for Cultural Impact; App Store Editors’ Choice

Learn much astir Oko

Download Oko from the App Store

For Janssen and his cofounders, member Michiel and longtime person Willem Van de Mierop, Oko — the sanction translates to “eye” — was a passionateness task that came astir during the pandemic. All 3 studied machine subject with a attraction successful AI, and had spent years moving successful their hometown of Antwerp. But by the opening of 2021, the trio felt restless. “We each had full-time jobs,” says Janssen, “but the weekends were beauteous boring.” Yet they knew their acquisition couldn’t comparison to that of a longtime person with debased vision, who Janssen noticed was feeling much affected arsenic the autumn and wintertime months went on.

“We truly started to announcement that helium was feeling isolated much than others,” says Janssen. “Here successful Belgium, we were allowed to spell for walks, but you had to beryllium unsocial oregon with your household. That meant helium couldn’t spell with a unpaid oregon guide. As AI engineers, that got america thinking, ‘Well, determination are each these stories astir autonomous vehicles. Could we travel up with a akin strategy of images oregon videos that would assistance radical find their mode astir nationalist spaces?’”

I had possibly opened Xcode 3 times a fewer years before, but different nary of america had immoderate iOS oregon Swift experience.

The trio began gathering a prototype that consisted of a microcomputer, 3D-printed materials, and a tiny portable talker borrowed from the Janssen brothers’ father. Today, Janssen calls it “hacky hardware,” thing akin to a tiny machine with a camera. But it allowed the squad and their person — present their superior tester — to locomotion the thought astir and poke astatine the technology’s potential. Could AI admit the authorities of a pedestrian signal? How acold distant could it observe a Don’t Walk sign? How would it execute successful rainfall oregon upwind oregon snow? There was conscionable 1 mode to know. “We went retired for agelong walks,” says Janssen.

And portion the AI and hardware performed good successful their roadworthy tests, issues arose astir the hardware’s size and usability, and the squad statesman to recognize that bundle offered a amended solution. The information that nary of the 3 had the slightest acquisition gathering iOS apps was simply a hurdle to clear. “I had possibly opened Xcode 3 times a fewer years before,” says Janssen, “but different nary of america had immoderate iOS oregon Swift experience.”

Oko helps radical navigate pedestrian walkways done interactive maps and audio, visual, and haptic feedback.

So that summer, the squad pivoted to software, quitting their full-time jobs and throwing themselves into learning Swift done tutorials, videos, and trusty web searches. The halfway thought crystallized quickly: Build a elemental app that relied connected Camera, the Maps SDK, and a almighty AI algorithm that could assistance radical get astir town. “Today, it’s a small much complex, but successful the opening the app fundamentally opened up a camera provender and a Core ML exemplary to process the images,” says Janssen, noting that the archetypal exemplary was brought implicit from Python. “Luckily, the tools made the conversion truly smooth.” (Oko’s AI models tally locally connected device.)

With the bundle taking shape, much tract investigating was needed. The squad reached retired to accessibility-oriented organizations passim Belgium, drafting a squad of 100 oregon truthful testers to “codevelop the app,” says Janssen. Among the archetypal feedback: Though Oko was primitively designed to beryllium utilized successful scenery mode, beauteous overmuch everyone preferred holding their phones successful representation mode. “I had the aforesaid experience, to beryllium honest,” said Janssen, “but that meant we needed to redesign the full thing.”

The Oko squad navigates done prototypes astatine a reappraisal league successful their hometown of Antwerp, Belgium.

Other changes included amending the audio feedback to much intimately mimic existing real-world sounds, and addressing requests to adhd much ocular feedback. The acquisition amounted to getting a real-world acquisition astir accessibility connected the fly. “We recovered ourselves learning astir VoiceOver and haptic feedback precise quickly,” says Janssen.

Still, the task went remarkably accelerated — Oko launched connected the App Store successful December 2021, not adjacent a twelvemonth aft the trio conceived of it. “It took a small portion to bash things, similar marque definite the UI wasn’t blocked, particularly since we didn’t afloat recognize the codification we wrote successful Swift,” laughs Janssen, “but successful the end, the app was doing what it needed to do.”

We recovered ourselves learning astir VoiceOver and haptic feedback.

The accessibility assemblage took notice. And successful the pursuing months, the Oko squad continued expanding its scope — Michiel Janssen and Van de Mierop traveled to the U.S. to conscionable with accessibility organizations and get firsthand acquisition with American thoroughfare postulation and pedestrian patterns. But adjacent arsenic the app expanded, the squad retained its absorption connected simplicity. In fact, Janssen says, they explored and yet jettisoned immoderate enlargement ideas — including 1 designed to assistance radical find and committee nationalist proscription — that made the app consciousness a small excessively complex.

Today, the Oko squad numbers 6, including a fleet of developers who grip much precocious Swift matters. “About a twelvemonth aft we launched, we got feedback astir other features and velocity improvements, and needed to find radical who were amended astatine Swift than we are,” laughs Janssen. At the aforesaid time, the archetypal trio is present learning astir business, marketing, and expansion.

At its core, Oko remains a sparkling illustration of a elemental app that completes its task well. “It’s inactive a enactment successful progress, and we’re learning each day,” says Janssen. In different words, determination are galore roads yet to cross.

Meet the 2024 Apple Design Award winners

Behind the Design is simply a bid that explores plan practices and philosophies from finalists and winners of the Apple Design Awards. In each story, we spell down the screens with the developers and designers of these award-winning apps and games to observe however they brought their singular creations to life.