In 2014, Russell Earl had explained to us the work of ILM on CAPTAIN AMERICA: THE WINTER SOLDIER. He then worked on another Marvel movie, ANT-MAN. Today, he talks about his new collaboration with the Russo brothers on CAPTAIN AMERICA: CIVIL WAR.
What was your feeling to work again with the Russo Brothers and VFX Supervisor Dan Deleeuw?
It was great to get the chance to work with Dan and Joe and Anthony, and the entire Marvel team again. Having worked on WINTER SOLDIER together, we had a great foundation. We know what their aesthetic was so it gave us a shorthand in discussing shots and what they should be.
This new Captain America has a bigger scale. Does that change the approach to the VFX?
The approach was similar to the approach on WINTER SOLDIER. A good mix of practical and digital effects. In the end, the effects work we do should take a backseat to the character development and storytelling. As always we try to base everything possible in reality so it feels real and grounded.
Can you describe to us one of your typical day during the pre-prod, on-set and then on post?
Our production team at ILM did a great job of keeping us on task. Typical day in pre-production would be meeting with the team to go over concept art, and discuss each of the sequences and our plan of attack. Taking the time to think everything through as best we could gave us a solid foundation to build upon so that ultimately all the work we were doing would show on screen.
In the heat of production we would have dailies first thing in the morning, where we go into one of our theaters and review the day’s shots. At our busiest that was about 170 shots to review, which meant we’d get out of dailies at lunchtime. We’d have a quick bite, usually at my desk, and then we’d squeeze in whatever meetings needed to happen before a studio review. In the studio reviews we’d look at the work with the team at Marvel, make sure things were on the right track and working towards final. Straight out of that review we’d roll into nightlies. Nightlies lasted another couple hours in the theater reviewing more shots. Then it was back to the office to do more desktop reviews and run through the sequences prepping for the next day and for daily studio reviews. Oh, and I left out the stretch breaks… every 30 minutes or so we’d encourage everyone to stand up and take a quick stretch!
What sequences were contributed by ILM?
The massive airport fight was our hallmark sequence. We also worked on the prison raft exteriors, including the Iron Man suit-up shot.
What was your approach about the airport fight showcasing all the heroes?
The previz was carefully choreographed and well thought out. It was the basis for principal photography and thus the basis for ILM’s work. Working from the initial editorial turnover, Tim Dobbert, our layout supervisor went through and figured out where all the action would play out and then worked tweaking composition of the individual shots. Tim and his team were working using the proxy airport set that we had built based off the point cloud data we received. The initial layout pass positioned our heroes around the airport, and then we did additional passes to add in hero vehicles, or move things around to dress to camera.
Can you tell us more about the filming of this fight?
The sequence was shot on a huge green screen in Atlanta, Georgia. The stunt team took the previz and then went off and choreographed all the beats of the clash and the fight. When it came time to shoot, we were referencing the previs and the fight vis to shoot all the pieces we would need in post. The fight was so long and complex it was shot in many smaller pieces over the course of a few weeks. There were only a few days where we had everyone there (for the shots where they are all running together for example). The rest of the days we would focus on the smaller beats of the larger fight.
We had stunt performers standing in for Black Panther, Iron Man, War Machine and Spider-Man so we could get the physical interaction we needed, and allow the DP to frame up shots. In post we rendered those characters all CG. We also rendered 99% of the airport, planes and vehicles to fill out the shots.
How did you created the digital doubles?
We started with scans and photographs of all the actors in costume and sculpted from there. We spent the most time on Spider-Man and Black Panther, since the designs changed and evolved as we were building them. There was a lot of back in forth with the whole gang to dial in every last detail to get just the right look. We always wanted to be sure that we kept the photo-real look of the characters.
ILM created most of the heroes in the previous films. How did you enhance them especially Iron Man?
We were fortunate to have Bruce Holcomb supervising every aspect of asset builds on the film. Iron Man was a complete rebuild for this film, the Mark46 suit. Bruce knows Iron Man inside and out, which was invaluable. The Mark46 is a more streamlined version of the previous Iron Man suits, and it is as close to the “bleeding edge” armor of the comics, as it has ever been. One of the fun things we got to do on this film was show off the inside of the suit, in a great little exchange with Ant-Man.
Can you tell us more about the challenges with the render of the metallic look of the Iron Man and War Machine armor?
We were rendering in Vray for Katana. We worked extensively with our texture painters to get the wear and tear just right and with our TD team to make sure we dialed in each of the metallic surfaces – raw metal, anodized metal, painted metal, etc. – to appear as photographically realistic as possible.
Can you tell us more about the previs process for this sequence?
Production VFX Supervisor Dan Deleeuw spent a lot of time thinking this sequence through and planning it out with Joe and Anthony. They had a little miniature set built complete with action figures for all the characters so that it made it easier to keep track of who was where, and how the action would all play out. This was what we started from.
Can you explain step by step about the creation of a massive sequence like this?
It all starts with our crew. The entire team had lots of experience working on big shows like this people like Francois Lambert leading up the compositing department, and Pat Conran our CG Supervisor. Working with these guys makes my job so much easier, they are amazing problem solvers and have a great aesthetic sense, so it comes down to planning. Our plan here was to try to keep everything moving along simultaneously, which meant lots of moving pieces and lots of tools that would allow us to upgrade assets as they were being improved upon. Of course none of would have come together without our production team and tools tracking every element and artist at every step of production – They are a truly incredible team.
We started by making proxy versions of all the sets and assets that could easily be swapped for higher resolution, more highly detailed geometry as the assets progressed. Our lighting lead Jeremy Bloch came up with a concept we dubbed “the imaginary tool” (it was a running joke until the tool actually came into existence). This amazing tool would look at where a shot sat in the sequence, and determine what damage had occurred and make sure to swap the correct geometry and paint in.
Did you develop specific tools or process to help visualize so many CG characters and environment?
On a complex show like CIVIL WAR, there are always tools we need to develop. “The imaginary tool” was one of many specific tools that were written to complete the sequence. We had tools in Nuke that allowed us to populate the airport with smoke and other damage that again could be tweaked per shot. We were careful to work in specific ways so that everything, each character and vehicle, etc,. were painted and shaded in the same way so that they would all sit in the same world, no matter what lighting conditions or location we put them in.
How did you handle the challenges with the Giant Man?
Giant Man was a really fun part of the sequence. A bunch of the parameters in the look development that were set for Giant Man were driven by his scale, so, for example when he was a half-inch tall the amount of displacement or bump was different than when he was 50ft tall. We also had to go in and add much more detail in the paint and sculpt as we were so close up to him. In addition to the cloth simulation on his outfit, we were doing things like simulating the boots so that they didn’t just feel like big rubber boots.
The facial performance for Paul was shot using a universal capture setup similar to what we had used on Peyton Reed’s ANT-MAN. It essentially consisted of a multi-camera setup that framed his face. We took that footage, solved the track and re-projected back onto a 3D head for Giant Man, so that anytime you see the performance through his visor, it’s Paul.
This fight is featuring the new Spider-Man. Can you explain in detail about his design and creation?
Having a hand in the new Spider-Man was exciting for our whole team. The decision to go all CG with Spider-Man was made midway through shooting. Once the decision was made it meant that we had the freedom to go back in and redesign the suit. We worked very closely with the Brothers, and the team at Marvel, looking at all aspects of the suit, the materials, the proportions, the web lines, even how light would play off its surface, it was an ongoing discussion with lots of testing of different looks.
In addition to the suit material properties and design, we also talked a lot about the facial movement including eyes. Again, we did lots of animation and simulation tests with differing amounts of jaw movement, fabric stretch and slide, eye irising, etc., until we settled on a look that just felt right for the character.
Can you tell us more about his rigging and his animation?
With everything we were doing the mandate was to always see Tom in the character of Spidey. This was the guiding force in every decision. Animation supervisor, Steve Rawlins, and creature supervisor Aaron Grey worked very closely with their teams to make sure we had the tools we needed to get the character of the animation and level of detail just right. It started with Aaron and his team building a rig that would work with motion capture and keyframe animation, while animation was going they were busy sculpting the underlying muscles.
We shot Tom Holland delivering all his lines with a dual head camera setup and multiple reference cameras to record his body performance. The goal was to always have Spiderman feel like he was Tom, to keep the great performance he was giving. We also worked very hard to keep the awkward imperfections while at the same time still nailing the key comic book poses. Again, we always were striving to make what he was doing feel physically possible.
Once animation was complete an initial muscle simulation pass was done, and on top of that there was a fabric simulation pass. From there we would evaluate the simulations and decide if we needed more or less wrinkles, more or less stretch, additional levels of simulation, etc. It was a bit of a back and forth until we nailed just the right look.
How did you created the huge Leipzig airport environment?
The airport was a huge undertaking and our single largest challenge on the show. We did a two week reference shoot at the Leipzig location headed up by our environment supes Johan Thorngren and Shane Roberts where we photographed and measured all the key areas, where the fight would take place. From the initial previz we had seen we were fairly confident we were going to have to have a full virtual airport for the sequence. In addition to all the photographs, Rise FX scanned all the areas of the airport and delivered point cloud data back to us that we used as a basis to start our build. We built in modules as best we could for repeating things like the jetways, and then we had custom pieces as well. The airport you see in the movie is augmented from the actual location, with extra height and more gates.
The build of the airport also included all the vehicles, planes, cargo, etc. All in all, over 600 unique assets make up the airport. We see it from 1/2 inch Ant- Man scale to 50 ft Giant Man scale.
Can you tell us more about the creation of the helipad and the prison exteriors?
We had one initial concept for the raft prison, and from that came up with a story of how you would actually construct such a thing. We built off the assumption that it was essentially a big submarine. From there working with Dan we gathered a bunch of submarine reference and went from there.
Was there a shot or sequence that prevent you from sleep?
Thankfully we have a really talented group of artists and an equally talented production team at ILM. Partnering with Jeanie King and Katherine Farrar leading up the production side, we had a solid plan to get the show done. We were able to stay flexible and shift priorities as needed. I can honestly say that the only sleep prevention I got, was just thinking about all the moving pieces and how they would all fit together.
What do you take away from this experience?
It all comes down to the people. Joe and Anthony Russo are very collaborative guys, and they push to get the best images up on the screen. The great attitude and collaborative spirit starts there and trickles down through the entire crew. We had a great team here in San Francisco and in our Vancouver studio working on the show and I couldn’t ask for a more talented bunch to collaborate with.
How long have you worked on this film?
I think we started initial talks around the beginning of 2015, we shot over the summer and finished post the first week of April 2016, so just over a year in total.
How many shots have you done?
I think the total when all was said and done worked out to about 637 shots.
What was the size of your team?
The team varied in size throughout the project, but in total we had about 220 people working on the movie split between our San Francisco and Vancouver studios and an additional team at Base FX.
What is your next project?
Sorry. That one is top secret, but I hope to be back in the Marvel Universe soon!
A big thanks for your time.
// WANT TO KNOW MORE?
– Industrial Light & Magic: Dedicated page about CAPTAIN AMERICA – CIVIL WAR on ILM website.
© Vincent Frei – The Art of VFX – 2016
