IN 2016, Sven Martin told us about the work of Pixomondo on Game of Thrones. He then worked on many shows such as Carnival Row, Midway and The Mandalorian.
How did you and Pixomondo get involved on this show?
The movie is produced by Paramount Pictures, with whom Pixomondo has worked before on shows like Star Trek – Into Darkness and Super 8. Most of the shoot took place in Germany, and PXO has offices in Stuttgart and Frankfurt. Paramount was also looking for a show supervisor, so after some calls with the show’s executive producer, Paramount’s EVP of visual effects and finally the director, we were setup.
How was the collaboration with Director Stefano Sollima?
I really enjoyed working with him. He is very honest and direct. Right from the beginning during our first call, he was clear about his general approach to do as much practical as possible and only use VFX where really needed. I share his view of not ‘wasting’ money on something what can be achieved for real. The great thing about a practical approach is the unexpected and naturalistic creation of side effects. With these elements we always add in visual effects to avoid a clean and perfect CG look.
What was his approach and expectations about the visual effects?
This film is clearly not a visual effects driven movie, so the main mission was to support the action without drawing any attention. This gets tricky when dealing with an airplane crash, with the audience being aware of the fact that it can’t be shot for real.
I had seen Stefanos previous work, so I had a principal understanding of his style and what he was aiming for. To give Stefano the freedom he needs on set required a lot of ‘thinking ahead’ in close collaboration with the art department and the special effects team.
How did you organize the work with your VFX Producer?
Yasamin Ismaili joined the production during pre-production in Berlin Babelsberg. The first weeks went into planning and budgeting, while the script got refined and sequences got developed. This was a very creative process and many scenes in the final movie came to be during this period. For example one day the director presented his idea of the interrogation in a burning car. While I was in a creative exchange with the other departments to discuss different approaches, Yas took care of tracking and budgeting of the new idea and making sure we’d stay within the given budget. Yas was fantastic, she was always on set with us and pushing hard to get all the information we needed. During post production, with Yasamin back in the States and myself in Germany, we had the perfect setup to handle the unexpected COVID situation, which hit us shortly after principal photography wrapped. The director and the film’s editor, Matthew Newman (‘Drive’), were in Europe and the Studio and DoP were in the US, so we could stay in close contact with everyone more or less around the clock.
How did you split the work amongst the Pixomondo offices?
During pre-production, our L.A. previs team started to layout the plane crash sequence. Due to a lucky coincidence, we got that award while I was in L.A. for a presentation at Siggraph. So we could actually do the kick-off in person and discuss ideas first hand.
The visual effects work got divided between our German offices in Frankfurt and Stuttgart, with Mark Spindler and Adam Figielski supervising the teams. Mona Mohr helmed the show as Pixomondo VFX producer. While the Frankfurt team concentrated on Aleppo, plane extensions, underwater sinking shots, airports and the Washington D.C. sequences, the Stuttgart team focused on Murmansk battle, jet fighter attack, the plane crash, diving and open water shots, skinny Kelly and the cemetery.
Can you elaborate about the environment work?
The main environments in the movie are Aleppo in Syria, Murmansk in Russia and the open water around the plane crash.
As a basis for Aleppo, the production decided to shoot on an abandoned Chemical plant outside of Berlin. This location was also used in movies like ‘The Hunger Games’ before. The art department built one street set in the center of this complex, one story high with us extending up and in all directions.
The harbor city of Murmask was shot on the Babelsberg Studio backlot. The art department re-dressed the ‘Old Berlin’ style streets to a more Russian look. The battle takes place both within the streets and on the rooftop. We had to create extensions for the adjacent rooftops and expand the city.
We photogrammetry scanned all environments for the CG extensions and to extract hires textures.
With me on set was Pixomondo’s set-supervisor Christoph Schmidt who took care of all scanning and second unit plate photography, e.g. shooting stabilized water plates from a boat. The ocean environment was a combination of real open water, a wave pool in a Search & Rescue training facility in northern Germany and the water tank at Studio Babelsberg.
Which location was the most complicated to create and why?
Aleppo was probably the environment we spent most R & D on. We wanted to push our City Builder further, a tool we started to develop for a previous Star Trek show. Thomas Kutschera had developed a script-based solution for building creation and city layouts based on original map information. To tear down these houses, the team came up with Houdini setups to destroy selected areas interactively, creating piles of rubble and additional detailing of the broken edges. Texturing was mainly procedural and we wrote an exporter to transfer the city to other software packages as well. Additional trees, lamps and wire poles could be distributed within the streets for detail. Cars and pedestrians added life to the city.
Not knowing how many shots we’d have to insert a digital Aleppo in later, and from which angles they’d be seen from, we decided to stay as flexible as possible.
Did you use any LED techniques for the background on the driving shots?
Overall I am a big fan of using screen techniques to achieve driving shots. The interactive light and captured reflections you get automatically is the key to a perfect illusion. Unfortunately, we could only shoot the driving backgrounds at the very end of the shooting schedule, when we were filming at the original location of Washington D.C. Therefore we had to do traditional greenscreen photography, but at least we knew the exact angles for the BG plates.
Can you explain in detail about the planes creation?
Our CG plane was a detailed copy of a real A320 that the production acquired and shipped to the studio. To make the plane fit into the water tank later, it got cut into multiple pieces, each roughly eight meters long. The interior and the cockpit were used in multiple setups, both during the flight and underwater scenes.
Christoph scanned the plane and various individual parts like the engine, so we could rebuild them to use as later crash debris. To extend the plane beyond the given eight meters in length, artists created a highres replica with galley, seats and overhead compartments.
A second plane shot in an open hangar at Leipzig airport was used for exterior shots during boarding, and became the outside reference with a new livery added.
Can you elaborate about your work on the intense plane crash seen from the inside?
The producers knew the plane crash would be the most challenging scene and therefore asked us to start this previs first. After multiple iterations with the director and DoP, everyone had a good understanding of how this sequence would look in the end.
The special effects team supervised by Gerd Nefzer had developed multiple setups within the water tank to make the plane break, roll-over and sink.
After a missile hits the plane, the descending plane was shot on a gimbal rig to get realistic rumble movements and shaking the actors and camera around. A steadycam followed our protagonist Kelly to the pilots where the cockpit hit the ocean surface and exploded through the windows. This element was shot in a dump tank, releasing gallons of water over a slide on the front section of the plane.
The rising water level was accomplished with a third setup, sinking another plane cutout into the water tank. When Kelly’s team starts to get out on the ripped-off cockpit side, Kelly decides to rescue the gear in the rear of the plane. The fourth rig could rotate the plane section around its axis, turning it upside down with the seats hanging from the ceiling. This setup was able to submerge completely for the upcoming diving scenes. A fifth rig setup was used to break an eight-meter section of the fuselage ‘in half’ and making the tail end sink.
To make the audience believe a full plane was sinking and rolling over, we extended one side with our CG airplane interior, while adding a CG ocean environment outside the windows and the sheared-off cockpit.
After the first take, the director loved the idea of creating a seamless and continuous shot, starting from the missile hit to the fuselage rolling over. We tried to carefully match camera framings at the start and end positions in the various setups. I prepared quick proof-of-concepts using the video playouts on my Laptop right after every shot. It worked out pretty well and the director got more and more excited with every new element I added. At the end we had a roughly two and a half minute long, continuous steady cam shot following the actors in a sinking and rotating plane.
How did you create the various FX for the plane attack and crash?
Quite late in the editing, the studio decided to add more views from the outside on the plane crashing and sliding over the ocean surface. We created a couple of suggestions as rough previs shots and Mat and Stefano could test them in the cut immediately. These new shots required an exploding and burning engine, teared-off wings and lots of debris and water spray. The plane comes to a stop with the nose digging deep into the ocean and ripping the cockpit open.
We found a real water crash clip in which the whole plane gets covered completely in white water. This became a main reference, but we decided to tone down the amount of water spray to show more of the plane itself. The FX team lead by Pixomondo’s Marc Joos, started to setup Houdini simulations for fire and smoke, rigid body and water simulations based on refined animations. At the same time, the existing A320 model got reworked to appear as a destroyed version. Multiple simulations had to run and interact with each other and it became quite a complex setup.
In the last shot, the plane slides from far back towards close to camera, so the FX artists split the water container into different resolutions based on the distance to save simulation time. The closed container, creating the front wake, required some sensitive tuning. The wake had to be large enough so that the audience could see inside the open fuselage but still hit the camera.
How did you work with the SFX and stunts team especially for the plane crash?
As mentioned earlier, this was one of the best experiences I’ve had in my career. Right from the start we sat together, talked through different options and decided which department could best solve it. For example, the special effects team shared their rig files, so we could check camera angles and develop greenscreen setups, which was then handed over to the art department. There was a lot of back and forth to achieve the best configurations for every department.
The stunt team, supervised by Doug Coleman, had their training space right beside our VFX office, so we worked closely from the beginning. They shared their stunt tests and we could influence the way in which safety wires and stunt rigs would not cause extensive paintouts. That was especially important with the plane crash sequence because a lot of precautions had to be taken to keep actors and stunt guys safe.
Can you elaborate on the underwater shots?
All underwater shots were shot for real; we did not use any dry-for-wet. Michael B. Jordan did intensive dive training prior to and during the shoot so he could hold his breath for multiple minutes – while acting and diving! It was very impressive as in those scene – it was all him!
When we see Kelly diving down into the tail section, those scenes had to be shot horizontally in the tank. On both open ends we did CG extend the plane according to Kelly’s position in the story. Meaning, the real plane section was used for the first class, economy class and the galley part of the A320.
To create the impression of a vertically sinking plane, we added lots of debris passing by Kelly and did bubble simulations heading towards the open side. The large debris was keyframe animation first to define the overall layout and then there were simulations on top. A rotomation of Michael B. Jordan and the scan of the interior made the bubbles react in physically correct way.
Additional wide shots showing the fuselage sinking down were done in full CG. One shot also utilized a Kelly digital double, which is based on a scan of Michael B. Jordan. We scanned all actors in case we had to add them later or replace stunt people if it was too dangerous.
The look dev of these wide underwater shots were actually the trickiest part. The scene took place at night, out on the open ocean somewhere in the Barents Sea. We did not want to play a Hollywood moon light situation, so the only actual light source was coming from the burning debris pieces floating on the surface and a few small position lights on the plane. We did a lot of iterations to find the right balance of giving enough visual information to the audience without losing the connection to reality.
How did you create the various explosions in the movie?
Nearly all explosions in the film are creations by the SFX team. But to enhance it further, we sometimes added more debris or additional fire blasts. Most wall hits are SFX, but to amp up the action we added additional dust hits close to the actor or to support the action in the edit better.
The big Aleppo explosion, for example, is based on a real pyro gag done on location, but it got extended with a Houdini simulation for more size and interaction with the added digital set. Additionally, the plate explosion was covered by a large industrial chimney in the foreground, which was not wanted in the shot. The burning car at the airport was also based on real fire and got enhanced and filled with visual effects. First we removed the gas pipes and SFX gear and then filled the gaps and brought the fire closer to the actors. But again I have to point out that Michael B. Jordan really stepped through the fire and got in and out of the car all by himself – and there were a lot of flames burning around him!
Which sequence or shot was the most challenging?
The most challenging was the late addition of the full CG crash shots. I think it was very helpful, that Stefano, Mat, Yas and I had spent a lot of time together during pre-production, shoot and postproduction, so there was a deep trust in us. I had a good understanding of Stefano’s vision and style, so we could pass the creative steps quite quickly and could concentrate fully on the technical challenges due to the given time.
Is there something specific that gives you some really short nights?
Yes, the shoot itself! I think we had more night shoots than day shoots, so I remember lots of cold nights outside. The good thing was we did not have to add any cold breath in post!
What is your favorite shot or sequence?
Without any doubt the airplane sequence. It involved so much planning, previs, techvis and different techniques used in collaboration with the other departments, and I’m very proud of the way it turned out. The audience is likely not aware of what that sequence entailed, but I love that we could use every trick in the book to make it look the way it did.
What is your best memory on this show?
The great team spirit on the show throughout, both during the shoot and during VFX production.
What I love about movie making is the fusion of creativity and craftsmanship – when all those talents join forces, the fun begins.
How long have you worked on this show?
From August 2019 through the pandemic until end of October 2020.
What’s the VFX shots count?
We worked on about 542 shots in total.
What was the size of your team?
Roughly 110 artist worked on the show.
A big thanks for your time.
WANT TO KNOW MORE?
Amazon Prime: You can now watch Tom Clancy’s Without Remorse on Amazon Prime.
© Vincent Frei – The Art of VFX – 2021
