Welcome to the new era. Whether you know it or not, Virtual Reality is here and it’s awesome. While most people using VR would be considered early-adopters, the industry is already attempting to make the jump from “The Next Big Thing” to mainstream, day-to-day utility. If you’re like us, you’re probably wondering, “What will that sound like?”
With all of the excitement around the new technology, we’re happy to report that audio is front-and-center on the minds of both content producers and technical engineers in the field. Not only do you need excellent sound design and mix in order to sell a VR experience, you also need it to respond to real-time head movements, just as the visuals do. So if you hear a monster behind you and turn your head to see the monster – it’s going to feel weird if it still sounds like it’s behind you.
First-person video games have been dealing with this issue for a long time, evolving since the days of Battlezone (arguably the first “virtual reality” game) and Wolfenstein 3D (the first first-person shooter). They assign sounds to objects and use programming to determine what triggers will initiate that sound; volume and panning can also be controlled and be made to depend on the player’s location and head position. Moving past a waterfall in a game world can sound just like walking along the footbridge at Multnomah Falls. Accelerating a race car can make your living room rumble. Gaming has become an extremely immersive experience and Virtual Reality is a natural progression.
VR experiences are in strict sense dependent on the user being able to move around within and interact with the virtual space. With 360 video, a more limited format, the user gets to control the viewpoint by moving her head but is not able to move around or interact with the environment. In many ways this is a hybrid of traditional video and virtual reality. Content creators are scrambling to figure out the strengths and limitations of this new format.
And the audience is growing, rapidly. This is still a new technology so it’s primarily being used by early adopters (don’t expect to walk in on your grandma with an Oculus Touch strapped to her face anytime soon), but there are so many potential applications that it seems inevitable that VR will quickly move into the mainstream and 360 videos will be right there as well.
Even in the time we’ve been using this very new technology it has already improved greatly (browser-based spatial audio was announced for the first time in late July). As VR continues to evolve, expect spatial audio to become an expected feature of most videos. Viewers will notice if it is missing.
We’ve been working with spatial audio and we are loving the new challenges and opportunities it has given us to craft truly immersive experiences. 3D mixes have to be robust enough that they work regardless of where a person’s head is turned. In essence, every mix made using spatial audio has the potential to become an infinite number of mixes during playback.
So, what can we show you so far? Put on your headphones (and VR goggles if you have them) and check out this abstract interpretation of how Puma’s evoTouch shoes are made, masterminded by the folks at Juliet Zulu. *Google Chrome is the only browser that supports spatial audio so far.
We also added our brand of audio flavor to a 360 video riffing on a monster familiar to fans of the show Grimm: Mishipeshu. This video really takes advantage of the localization and panning of spatial audio. If you lose sight of the monster, just follow your ears!
Any time a new technology emerges in our industry, which happens often, we confront the question as to whether it will stick around or end up gathering dust in the storage area wedged between a rack of cassette decks and stacks of old computer keyboards. In this case spatial audio seems quite useful, even necessary, for some videos. The real question is: will VR become ubiquitous? Will Millennials complain about Gen Z always staring into their Oculus the way Baby-boomers grumble about the iPhone? So long as it means that we get to keep designing sound for those punk kids and their wacky new goggles, I hope the answer is yes.
(If you want to fall down a very pleasant rabbit hole, head over to YouTube’s growing collection of 360 videos with spatial audio. It’s good way to get acquainted with what’s already been developed and get inspiration for what could be done.)