Meibei Liu, Media Rich Paper –
Filmmakers continue to test the limits of how they can improve picture, coming up with innovations such as IMAX cameras, 3D capture and 4K resolution. However, they are also working just as hard to improve film sound, aiming to make the moviegoing experience more real and impactful. Filmmakers agree that the most effective way to do this is with three-dimensional sound.
Surround sound uses sets of speakers to create a multidimensional sonic atmosphere. Beginning in June 2012, Dolby introduced the more complex Atmos system, hoping to create a 3D aural experience in theaters. The biggest difference between this new system and traditional surround sound is the extra speakers on the ceiling, which help to create a better spacial sound atmosphere. Films like The Life of Pi, The Hobbit: The Battle of The Five Armies, Gravity,The Martian and many others all used this new sound system. Dolby’s official website introduces the new features of the Atmos system and shows some examples of how it sounds.
Many people call Atmos a “3D sound experience”. But if one googles “3D sound”, the similar terms will appear in the results. This actually refers to another technology called “Binaural Recording”. This is a sound recording technology that came out over 100 years ago that was used to recreate the experience of watching a play or concert for people who could not actually be there in the theater. By recording the sound on set and then transferring the sound files to the clients, Binaural Recording aims to recreate the exact same sound experience for human ears. While it sounds complicated, the concept behind the recording technology is rather simple. One builds two microphones inside two fake ears and puts them on a model head. With this set of microphones, the slight difference in the waveforms created by the sound sources are captured by the left and right ears. This stimulates greatly a sense of distance and direction for listeners.
While it sounds complicated, the concept behind the recording technology is rather simple. One builds two microphones inside two fake ears and puts them on a model head. With this set of microphones, the slight difference in the waveforms created by the sound sources are captured by the left and right ears. This stimulates greatly a sense of distance and direction for listeners.
Here’s the example of what it sounds like:
Videos shows it is recorded.
As innovative as this technology is, it was only used for a few experimental art forms and the recreation of some plays and concerts before it faded out. It is coming back, mostly as an element of VR and video games, but still is a rather minor component.
If Binaural Recording tends to create effects similar to those created by surround sound and Atmos sound, why are they treated so differently? Why was Binaural Recording never used in any theatrical feature films not even 3D or IMAX films? Imagine watching Life of Pi and hearing the fish come from different directions towards you. How realistic and impactful would that experience be. Here are the reasons of why has Binaural Recording technology been deemed not suitable for a story-emphasized film.
First, the Binaural Recording system costs more money and time while surround sound is easier to record and mix. Second, this technology is only effective when the user is wearing head phones, which leads to a series of issues, including financial issues for the theaters and social downside for the audience. It is hard for a film to find a balance way to approach within a scene or even a shot, because of its listener-centered perspective. Finally Binaural Recording is a better fit for VR and video games as they have similar special approaches.
Before filmmakers decide to adapt binaural recording, they’ll likely ask themselves the following questions: “Is this easy to do on set?” “Will this cost more money and time?” “Is this controllable during pre-production, production and post-production?” If using binaural recording, the filmmakers would need to choose a perspective, considering what to record and where to put the mic. The sound technician would have to abandon their training and experience to come up with a brand new process. The sound picked up in a binaural microphone is different; it is not traditional sound that can be edited and mixed during post, but sound that has perspective and special 3D sound effects built in. They would need to think more delicately in terms of storytelling while recording the sound on set, as the Binaural Recording captures the distance and space so well. For example, imagine a simple scene with a character walking from a door to a sofa. In traditional sound recording, this would be recorded with a boom microphone. However, with binaural recording, the sound recorder would need to think about who is watching him walking inside the scene and when the perspective changes. This would require lots of additional planning before the shoot. If the director decides to makes a last-minute change of to how to approach the scene, the plan would need to be adjusted again. And during the post sound process, technicians will find that the binaural recordings are more difficult to adjust than traditional sound. Is binaural recording really worth the effort? By comparison, surround sound and Atmos sound offer much more control during post-production; technicians can simply pan a sound clip right or left to shift the direction, and then change the volume to manipulate the distance.
This video is of a sound designer talking about sound effects and sound mixing on Interstellar. From this video, one can see a sound element as a component could be recorded in an easy way and then mixed to create a desired atmosphere.
The limitation of “3D sound” is that it works better with a set of headphones than speakers. Research shows that with a speaker, human ears are tracking the source of the sound, but not the virtual sound source. This means one would need headphones in order to truly experience film sound in 3D.
One might argue that theaters could simply install headphones. But let us consider: why do people go to the theater? They probably own Gravity on a Blu-Ray DVD, along with a large screen and a pair of good headsets. But the theater offers a different and unique experience. At the theater, people can enjoy Life of Pi on a larger-than-life screen and be immersed in high-quality surround sound, all while sharing the experience with your friends, your family and all the other strangers in the theater. Imagine watching a exciting action film in a theater with a headset. Wearing headphones puts you in your own world. It separates you from other people nearby and therefore takes away from the shared movie theater experience. The movie theater connects people. Many years ago, theaters were a place for people to meet and socialize. But going to a theater and watching a film with a headset makes theatergoing an individual activity rather than a social activity. Practically, the adaptation of a binaural sound system would likely involve a re-design and reconstruction of movie theaters, which would be financial burden for theaters. Furthermore, a change of sound disc to sound film playback system would require many years and discussions among studios to complete the transition. It’s unlikely the entertainment industry will make the change to binaural sound, due to its impracticalities.
Despite the financial, social, technical issues, binaural sound recording would also affect story telling in film. Unlike a VR film, in which the audience is part of the story and exploring the plot by themselves, a theatrical film usually has characters and is not interactive. Characters drive story and, therefore, the audience’s primary focus is on characters. Even though filmmakers sometimes use 3D to make the films come to life, the audience is still just an “outsider”. Surround sound helps the audience become more immersed in story. But 3D sound is more objective, and can impose limits on storytelling.
Example: WWI scene using the 3D sound.
In this clip, the viewer is part of the story. 3D sound helps the viewer feel as though he is in the story.
There are some art films that uses 3D sound as well. It is used effectively. However, this film isn’t a traditional narrative.
3D binaural sound strongly focuses on the listener’s point of view, creating a strong sense of “I” for the audience. If one wishes to put the 3D sound into a narrative film, then you have to think about what perspective you want to approach from, and it might be difficult and confusing. A simple scene like a conversation between three people recorded from the point of view from three different characters could be very confusing, and that’s before one considers additional sound effects. One needs to consider who is listening at each moment, who is the next center of the sound and how do they transit? Here’s an example:
This is a simple narrative piece about using the 3D binaural sound technology. And you will find the weakness of using it. The strong feeling of sound direction and the virtual location from where we are listening ruin the subjectivity of the audience while watching the video.
3D sound along with the Binaural Recording technology is more suitable for VR. When an audience member hears sound coming from behind them, they will want to turn around. With a narrative traditional film, the screen is fixed, the vision is limited to the rectangle area. Audiences won’t turn around or looks at the area that is outside the screen because there are nothing showing. But with VR, audiences actually can turn and face any direction. 3D sound can be used here to prompt and trick audiences to move a certain way and discover more information, at the same time getting better control of what they want the audience to focus. The same may apply to video games as well. While surround sound creates a rather subjective effect, it allows the audience to enjoy and experience the film and focus on the story without paying much attention to the sound.
In conclusion, 3D sound with Binaural Recording is an innovative technology that will no doubt be important in the future of entertainment, but it is more likely to be used in virtual reality than in movie theaters.