An immersive app doesn't only look and feel great — it has to sound
incredible, too. During WWDC, we spoke with four Apple Design Award
finalists about the sensational sounds of their apps and games. Come
with us as we travel on a musical journey through cool jazz, Spatial
Audio soundscapes, and even original album-length compositions from
artists like Arcade Fire, St. Vincent, and Madlib.
Please, Touch the Artwork: A jazz thing Thomas Waterzooi's elegant
puzzle game Please, Touch the Artwork is inspired by Dutch painter
Piet Mondrian, who, with his iconic style of lines, squares, and
primary colors, is considered a pioneer of 20th century abstract art.
And that kind of guy? He needs the right kind of music.
"I tried to imagine what Mondrian would listen to while painting in
his workshop," says Waterzooi, the game's Brussels-based developer.
"Some kind of jazz. And since the game is designed to be relaxing, it
would have to be a calm, dynamic kind of jazz."
That mix of timeless art and cool soundtrack creates a classy vibe in
Please, Touch the Artwork, whose puzzles are based on three of
Mondrian's most famous works: Composition with Red, Blue, and Yellow;
Broadway Boogie Woogie; and New York City I.
As the puzzles grow and change, so does the music, which was created
by composer Lars Burgwal. The music for the New York City section
begins with only bass — and as you progress through each puzzle, the
piano, saxophone, and vibraphone all come to play. (Waterzooi also
added a little drum flourish whenever you tap a painting.) "With
puzzle games, the music has to be relaxing," says Waterzooi. "It can't
annoy you at any point."
So not so much Broadway boogie woogie for Broadway Boogie Woogie? "It
would be too fast!" laughs Waterzooi. "We couldn't swing nearly that
much." He has, however, worked in a little nod to the style.
"The goal of that game is to join characters named Boogie and Woogie,"
he says, "and when you do, there's a little completion animation with
a musical accent. It's not much — just three or four notes — but it's
based on some boogie-woogie right-hand piano schemes."
Download Please, Touch the Artwork on the App Store
Odio: Absolutely Spatial Audio apps don't get much more immersive than
Odio. The Apple Design Award-winning 3D audio app employs a
mesmerizing mix of Spatial Audio and head tracking to conjure up its
AR soundscapes.
While you might flow between a rushing waterfall, the deep sea, and
even a world of calming digital ambience, you're no passive listener
in these realistic realms: Each soundscape can be manipulated through
a clever system of arcing sliders that reposition each sonic element —
a rushing river, dreamy whalesong, or wash of digital static — around
your head in 360 degrees.
Max Frimout is the app's audio engineer, and though his work is heavy
on synthetic, otherworldly digital elements, his audio career started
with something considerably more analog. "I was originally a harpist,"
he says. "One day I opened the ES1 Synthesizer in Logic Pro, and now
I'm here!"
Odio originally focused on nature sounds, but after a few months of
development, the Netherlands-based team at Volst wanted more. "'What
if we have musicians compose their own environments?'" says Roger
Kemp, co-founder and designer at Volst. "That's when it all clicked."
Frimout is also one of the app's five composers. A musician and DJ by
trade, he began creating his Odio soundscapes with lines of melody,
then layered in effects and flourishes with names like "synthetic
water," "moving chords," and "filtered drone." Soundscapes are built
in Logic Pro and tested with AirPods Max. "That's how I look around to
hear how it feels," he says.
Most of Frimout's compositions are the result of sonic
experimentation, but the soundscape called "Wow!" followed a more
organic path. "I started with a series of melodies that basically all
came to me in the same evening," he says. "I think that shows how you
can have all this equipment and all these concepts but still be
incredibly inspired by a single event."
And yes, it contains harp: That's Frimout playing on the loop called
Heartbreak — though you might not recognize the sound as strings.
"It's just three chordal structures," he says with a laugh, "but
they've been processed and processed and processed."
Download Odio on the App Store
A Musical Story: That '70s game A Musical Story is inspired by a very
groovy time: "It's all about the freedom of '70s music," says Charles
Bardin, the French composer/developer who created the game with art
director Alexandre Rey, composer Valentin Ducloux, and developer
Maxime Constantinian. "Mostly, we were inspired by the sense that,
back then, anything could happen."
Conceived in 2017 and launched in March 2022, A Musical Story is a
harmonious mix of song, narrative, and art. The story follows an
up-and-coming band trying to break into the business, replete with
vintage guitars, outfits, and hairstyles. To move the narrative along,
you tap your screen to the beat, creating some great soul- and
R&B-inflected music in the process.
But the game is mostly wordless, driven by the primal, powerful
connection between music and memory. It's an ideal playground for
Bardin, who studied at the Conservatoire de Musique de Lyon and who's
been creating and covering game music for more than a decade.
As it happens, the development process didn't begin with the music —
Bardin and Rey started by establishing the circular tap-along play
mechanic. "In most games, the notes come down on the screen and you
play them when they arrive," says Bardin. "I love that, but it's also
something you can play without any sound. I wanted a game that really
relies on listening."
Once the team landed on the mechanic, it was time to tune into the
songs themselves. "We knew we wanted short sequences of music to
unlock the story," says Bardin, "but a lot of musical games rely on
electronic or techno music, where the beat is very clear. We wanted to
prove that we could make more organic music — something that wasn't
quite so thump-thump-thump-thump."
He also made sure the music drove the story along. "I wrote a song
called Her for a scene in which the character goes to a pub, sees a
girl playing music, and instantly falls in love with her," says
Bardin. "It begins with just a Rhodes piano and some bass and drums,
but as you move closer to the stage, you hear more and more of the
music. When you get close enough, you discover her face and her
voice." It's the only time vocals appear in the game itself aside from
the credits. "We wanted this moment to be powerful," Bardin says.
"This is the voice of the most important character in the game."
Download A Musical Story on the App Store
Headspace: The music of mindfulness Over the past few years, the
meditation and mindfulness app Headspace has partnered with A-list
musical artists to help people concentrate, relax, lock in, or nod
off. With Focus Music (found, appropriately, in the Focus tab), the
app has amassed an array of original music and playlists from artists
like Arcade Fire, St. Vincent, Erykah Badu, Madlib, and even film
composer Hans Zimmer.
Focus Music was designed in part by John Legend, the app's chief music
officer. "There's so much possibility right here on our phones," says
Legend. "It can be a scary thing for some artists; it's not what we're
used to. But if we take advantage of the possibilities, there are all
these different ways to reach people."
The singer-songwriter Aluna trained in reflexology, transcendental
meditation, and tai chi — all skills she wove into her hour-long
Headspace composition. To create it, she designed six-minute blocks of
sound, grounded in specific spaces like a crackling campfire, bustling
park in late afternoon, or dripping cave.
Strictly speaking, it was not her usual approach. "Normally when you
write a song, you're doing wordplay and you want dynamics," she says.
"It's completely different from music that for an hour has no start
and no finish." (It's also more complicated than it sounds — there's a
lot of difference between the sound of water dripping from a cave and
dripping from your faucet.)
The science at the intersection of music and mindfulness is clear,
says UC Berkeley cognitive neuroscience professor Sahar Yousef, who
partnered with Headspace on Focus Music. "We know that when we play
music in rehab facilities, people improve quicker," Yousef says.
Here's the (extremely abridged) explanation of what's going on when
you listen: Your brain forges connections via neural networks, the
little zaps of electricity that constitute all your thoughts. The good
news is that these networks can be manipulated, and you're probably
doing it right now. You can train yourself to think that the aroma of
coffee means it's time to wake up, and you can train your brain to
recognize the music designed to chill you out.
In other words, these soundscapes serve as little life hacks. "Michael
Phelps listened to Eminem before every race," says Yousef. "This is
the same thing."
Download Headspace on the App Store
