by Joel Beckerman, PromaxBDA UK speaker and founder of ManMadeMusic
Virtual reality has piqued the interest and obsession of innovators throughout the world with its promising and undiscovered opportunities to deliver amazing experiences.
The first completely new storytelling medium in our lifetime, virtual reality can best be described as scalable theater where you are one of the actors—and the creative opportunities are truly unlimited.
But in the fledgling days of this transformative medium, we’re seeing the building of a new dramatic language from the ground up, as traditional processes for entertainment are flipped on their head.
But one thing is clear. Without cuts, dissolves or camera moves, there is really only one way to guide and drive narrative attention and move a story forward— sound.
As with exploring any new frontier, there are plenty of hurdles for virtual reality to explore and overcome
No matter what technology advancements come forward, the key to any compelling virtual reality story or experience will be meaningful 3D spatialised audio.
“As with exploring any new frontier, there are plenty of hurdles for virtual reality to explore and overcome”
From the perspective of developing content, the greatest struggle has been redefining the creative process. In the film and TV industry, sound and music are normally thought of as icing on the cake, components to tack on in post-production to create the desired polished finish.
However, those diving into the medium are quickly discovering that virtual reality necessitates sound to be brought to the table from the beginning for key narrative development.
Given the challenges and strengths of this growing medium, sound now plays a pivotal role in the narrative storytelling capability for each activation.
Driving focus and attention
Without direct control of the user’s point of view, directors for virtual reality must rely on sound to drive focus and attention with their audience. Not only is sound driving narrative, it’s also giving a sense of authenticity and specificity to each virtual reality activation.
Through on location recording, spatialised audio is able to fully immerse the viewer in the world, triggering strong sense memories and also inspiring their imagination through the authentic sound bites, fully completing the desired illusion.
If this illusion isn’t seamless, virtual reality can leave viewers feeling confused or even nauseous. These aversions can often be traced back to an incompatibility between what the body is seeing, and what the body is hearing.
The brain processes sound faster than any of the other senses, making it the organiser for all of our other sensory input. This results in a feeling of disassociation when the sound becomes incongruous with our visual surroundings, which is often the case in virtual reality experiences when sound is not carefully considered.
It’s much akin to the early days of 3D graphics, where proposed state of the art explosive images felt gratuitous rather than intentional and left the audience feeling visually assaulted.
While the technology for high resolution graphics in virtual reality are still under development, the technology behind 3D spatialised audio is already in place.
Full-feature VR and sound
While the full-feature virtual reality user base is currently small, VR is already shown to be beneficial as a tool for personal experiences, with merely a phone and earbuds. As great virtual reality videogame content comes to market, watch and see virtual reality headsets become the must-have Christmas gift this year.
This more immersive variant is expected to have explosive growth in the next six months, driven by lower prices and YouTube, Facebook and Hulu services now ready for massive content distribution across the internet.
The success of these smaller activations can be attributed to the spatialised quality of 3D audio. Typically, in the older virtual reality generation, when you turned your head to the left or the right, the sound and music followed you, instantly signaling to your brain an inconsistency in the projected reality.
But now, as you whip your head back and forth, the origin points for the sound stay the same.
Allowing you to forget you’re wearing headphones, and completing the illusion within your brain that you are moving through a new space, where objects and sounds have true permanency.
When your brain is able to make sense of the world around it through meaningful sound cues, our sensory input has synced up and the audience no longer feels disoriented.
The very next wave is going to be the true ‘community’ experience—in amusement parks, museums, shopping malls, theaters, concerts and stadiums where this revolutionary technology is set to raise the bar on immersive experiences.
As with exploring any new frontier, there are plenty of hurdles for virtual reality to explore and overcome as it continues to develop as the unique medium it is. But this is precisely why 3D spatialised audio is so vital for driving both storytelling and experience in virtual reality.
Hans Zimmer once said: “If you talk to any director, they’ll say music is 50% of the movie” but in the world of virtual reality, sound is closer to 65% of the experience.
We often see sound and music as the ‘glue’ that holds an attraction experience together. It has to seamlessly tie every stage of the experience together, completing the illusion. You can’t risk such a vital tool playing second fiddle in your experience.
Think sound first, and the rest will come.