Can AI Really Create

So We’ve seen the trailer of film Morgan edited by AI, which look pretty professional.

But have you seen this?

 

Sunspring- A sci-fi short wrote by AI “Benjamin”. Benjamin even wrote the theme song for this short.

AI is like VR right now. People are still trying to figure out what they can really do with it. But it will be a sure thing that AI will be totally being able to take over a lot of the human works.

But can AI really create?

Although “Benjamin” was able to analysis as many Sci-fi films as possible. What it really did was still searching in its database, finding a line and putting into the script either randomly or by percentage. In the short, it was very obvious that the script doesn’t make any sense. It was more of a combination of different elements from different films. But it does not really have a heart. Same to the trailer edited by AI, It was all about data analyzing.

Film is a very complicated media. It’s a combination or evolvement from many other medias. It’s visual and verbal. It’s a lot about rhythm, emotion. It is hard to calculate the data of how a film is made. There is not a definition of what is scary, what is sad. In the film No Country For Old Man. The killer made you feel threatened even when he is sitting on the sofa. If we just analysis the visual, the color, the script. It won’t be a typical scene that an AI can pick up. But it works efficiently.

AI might be good at executing instructions. But in terms of making the right choice for a certain situation creatively, it’s more of a challenge. Thus why people still need supervision to control the creative part, no matter if it writing novels or making trailers. With out it, the outcome might not be as good.

Advertisements

From “Tangerine” to the style of independent film

When I was at an interview, I said when I did my undergrad, the style of films we appreciated were very different from what USC likes. USC is more industrial with a Hollywood style. But at the time, we liked independent style better. Then the interviewer asked me with a questioning look: Can you describe what is the independent film style you mentioned? I was so nervous after hearing the question, thinking I said something wrong.

I do think there is a style of independent films. Sometime the style came without a choice, but sometimes they were choices made by the director. After watching Tangerine last week, I confirmed this thought.

Independent films, most of the time will be compared to Hollywood film by the audience. It normally means films that are not produced by major Hollywood studios, sometimes art films, experimental films.

With the appearances of cheaper digital cameras, non-liner editing softwares, sound mixing software and PC. Making a film becomes more and more easier.

Using non-stars, telling stories of normal life or social issues,showing the mundane details of daily life with low budget, independent films has its unique brand. Films this year like Manchester By The Sea is an example.

This slideshow requires JavaScript.

As a result of less budget, independent films tend to use natural light, hand held camera or steady long shots, long dialogue or very limited dialogue.

Because of the usage of smaller cameras and limited lighting. Indie filmmakers are able to show what a real life looks like, bringing the film closer to the audience. Just like what Tangerine did. It shows the real Los Angeles which is very different from the glory DTLA in other films, thanks to the small iPhone 5s provides more instant reaction from the filmmaker with a smaller film set.

This slideshow requires JavaScript.

I believe the difference of the style was determined by the different usage of film technologies at the first place.

What kind of films really needs 3D?

“A 3D film can end up gaining 33% higher profit than 2D.”

According to this reason, studios are making more and more 3D films. Some of them were shot in 3D, and some of them were converted to 3D in post.

When go to the theatre audience normally gets to choose 2D or 3D version to watch. But according to this news, audience in China did not have a choice when watching Jason Bourne (2016).

http://www.bbc.com/news/technology-37212239

“Out of 149 cinemas in Beijing, only eight are currently showing the 2D version, according to local media. In Shanghai, it’s said to be only nine out of 174.” As a result, Universal earned $11.8 million and set a new single-day record on the opening night in China.

As a film shot in 2D, Jason Bourne obviously is not suitable for 3D due to its shooting and editing style, compared to the two films we saw at the SCA IMAX theater. There are too many cuts; shots are too close to the object, audience’s eyes needs to move too frequently to track the object in different cuts and the hand-held camera made the experience worse. When screened in 3D, these movements could not be adjust by human brains correctly, which lead to Chinese audience’s nausea during the film.

While Born To Be Wild and Hubble almost kept the same object scale, had long shots, slow cuts, which make the audience dive into the amazing 3D effects.

To make a 3D film, the technology is not the only aspect the filmmakers need to consider about. They need to adjust other film aspects to make 3D work, such as Editing, Camera position and movements, Shot scales and so on.

What’s interesting is that after the complain from Chinese audience, Universal decided to add more 2D screenings. Which proves that the market can determine what consumers need, and consumers can determine the market as well. It’s a two way relationship.

The Difference Between Atmos Sound and 3D Binaural Sound and Why Films Won’t use Binaural Recording Technology

Meibei Liu, Media Rich Paper –

Filmmakers continue to test the limits of how they can improve picture, coming up with innovations such as IMAX cameras, 3D capture and 4K resolution. However, they are also working just as hard to improve film sound, aiming to make the moviegoing experience more real and impactful. Filmmakers agree that the most effective way to do this is with three-dimensional sound.

dolby-atmos-and-auro-3d-fig5-sm

Surround sound uses sets of speakers to create a multidimensional sonic atmosphere. Beginning in June 2012, Dolby introduced the more complex Atmos system, hoping to create a 3D aural experience in theaters. The biggest difference between this new system and traditional surround sound is the extra speakers on the ceiling, which help to create a better spacial sound atmosphere. Films like The Life of Pi, The Hobbit: The Battle of The Five Armies, Gravity,The Martian and many others all used this new sound system. Dolby’s official website introduces the new features of the Atmos system and shows some examples of how it sounds.

https://www.dolby.com/us/en/brands/dolby-atmos.html


Many people call Atmos a “3D sound experience”. But if one googles “3D sound”, the similar terms will appear in the results. This actually refers to another technology called “Binaural Recording”. This is a sound recording technology that came out over 100 years ago that was used to recreate the experience of watching a play or concert for people who could not actually be there in the theater. By recording the sound on set and then transferring the sound files to the clients, Binaural Recording aims to recreate the exact same sound experience for human ears. While it sounds complicated, the concept behind the recording technology is rather simple. One builds two microphones inside two fake ears and puts them on a model head. With this set of microphones, the slight difference in the waveforms created by the sound sources are captured by the left and right ears. This stimulates greatly a sense of distance and direction for listeners.

Screen Shot 2017-03-05 at 11.33.22 PM               Screen Shot 2017-03-06 at 10.39.27 PM

While it sounds complicated, the concept behind the recording technology is rather simple. One builds two microphones inside two fake ears and puts them on a model head. With this set of microphones, the slight difference in the waveforms created by the sound sources are captured by the left and right ears. This stimulates greatly a sense of distance and direction for listeners.

Here’s the example of what it sounds like:

Videos shows it is recorded. 


As innovative as this technology is, it was only used for a few experimental art forms and the recreation of some plays and concerts before it faded out. It is coming back, mostly as an element of VR and video games, but still is a rather minor component.

If Binaural Recording tends to create effects similar to those created by surround sound and Atmos sound, why are they treated so differently? Why was Binaural Recording never used in any theatrical feature films not even 3D or IMAX films? Imagine watching Life of Pi and hearing the fish come from different directions towards you. How realistic and impactful would that experience be. Here are the reasons of why has Binaural Recording technology been deemed not suitable for a story-emphasized film.

First, the Binaural Recording system costs more money and time while surround sound is easier to record and mix. Second, this technology is only effective when the user is wearing head phones, which leads to a series of issues, including financial issues for the theaters and social downside for the audience. It is hard for a film to find a balance way to approach within a scene or even a shot,  because of its listener-centered perspective. Finally Binaural Recording is a better fit for VR and video games as they have similar special approaches.


Before filmmakers decide to adapt binaural recording, they’ll likely ask themselves the following questions: “Is this easy to do on set?” “Will this cost more money and time?” “Is this controllable during pre-production, production and post-production?” If using binaural recording, the filmmakers would need to choose a perspective, considering what to record and where to put the mic. The sound technician would have to abandon their training and experience to come up with a brand new process. The sound picked up in a binaural microphone is different; it is not traditional sound that can be edited and mixed during post, but sound that has perspective and special 3D sound effects built in. They would need to think more delicately in terms of storytelling while recording the sound on set, as the Binaural Recording captures the distance and space so well. For example, imagine a simple scene with a character walking from a door to a sofa. In traditional sound recording, this would be recorded with a boom microphone. However, with binaural recording, the sound recorder would need to think about who is watching him walking inside the scene and when the perspective changes. This would require lots of additional planning before the shoot. If the director decides to makes a last-minute change of to how to approach the scene, the plan would need to be adjusted again. And during the post sound process, technicians will find that the binaural recordings are more difficult to adjust than traditional sound. Is binaural recording really worth the effort? By comparison, surround sound and Atmos sound offer much more control during post-production; technicians can simply pan a sound clip right or left to shift the direction, and then change the volume to manipulate the distance.

This video is of a sound designer talking about sound effects and sound mixing on Interstellar. From this video, one can see a sound element as a component could be recorded in an easy way and then mixed to create a desired atmosphere.


The limitation of “3D sound” is that it works better with a set of headphones than speakers. Research shows that with a speaker, human ears are tracking the source of the sound, but not the virtual sound source. This means one would need headphones in order to truly experience film sound in 3D.

Screen Shot 2017-03-06 at 11.25.46 PMScreen Shot 2017-03-05 at 11.10.10 PM

One might argue that theaters could simply install headphones. But let us consider: why do people go to the theater? They probably own Gravity on a Blu-Ray DVD, along with a large screen and a pair of good headsets. But the theater offers a different and unique experience. At the theater, people can enjoy Life of Pi on a larger-than-life screen and be immersed in high-quality surround sound, all while sharing the experience with your friends, your family and all the other strangers in the theater. Imagine watching a exciting action film in a theater with a headset. Wearing headphones puts you in your own world. It separates you from other people nearby and therefore takes away from the shared movie theater experience. The movie theater connects people. Many years ago, theaters were a place for people to meet and socialize. But going to a theater and watching a film with a headset makes theatergoing an individual activity rather than a social activity. Practically, the adaptation of a binaural sound system would likely involve a re-design and reconstruction of movie theaters, which would be financial burden for theaters. Furthermore, a change of sound disc to sound film playback system would require many years and discussions among studios to complete the transition. It’s unlikely the entertainment industry will make the change to binaural sound, due to its impracticalities.


Despite the financial, social, technical issues, binaural sound recording would also affect story telling in film. Unlike a VR film, in which the audience is part of the story and exploring the plot by themselves, a theatrical film usually has characters and is not interactive. Characters drive story and, therefore, the audience’s primary focus is on characters. Even though filmmakers sometimes use 3D to make the films come to life, the audience is still just an “outsider”. Surround sound helps the audience become more immersed in story. But 3D sound is more objective, and can impose limits on storytelling.

Example: WWI scene using the 3D sound.

In this clip, the viewer is part of the story. 3D sound helps the viewer feel as though he is in the story.

There are some art films that uses 3D sound as well. It is used effectively. However, this film isn’t a traditional narrative.

3D binaural sound strongly focuses on the listener’s point of view, creating a strong sense of “I” for the audience. If one wishes to put the 3D sound into a narrative film, then you have to think about what perspective you want to approach from, and it might be difficult and confusing. A simple scene like a conversation between three people recorded from the point of view from three different characters could be very confusing, and that’s before one considers additional sound effects. One needs to consider who is listening at each moment, who is the next center of the sound and how do they transit? Here’s an example:

This is a simple narrative piece about using the 3D binaural sound technology. And you will find the weakness of using it. The strong feeling of sound direction and the virtual location from where we are listening ruin the subjectivity of the audience while watching the video.


3D sound along with the Binaural Recording technology is more suitable for VR. When an audience member hears sound coming from behind them, they will want to turn around. With a narrative traditional film, the screen is fixed, the vision is limited to the rectangle area. Audiences won’t turn around or looks at the area that is outside the screen because there are nothing showing. But with VR, audiences actually can turn and face any direction. 3D sound can be used here to prompt and trick audiences to move a certain way and discover more information, at the same time getting better control of what they want the audience to focus. The same may apply to video games as well. While surround sound creates a rather subjective effect, it allows the audience to enjoy and experience the film and focus on the story without paying much attention to the sound.

In conclusion, 3D sound with Binaural Recording is an innovative technology that will no doubt be important in the future of entertainment, but it is more likely to be used in virtual reality than in movie theaters.


References: