In 2016, Russell Earl explained the work of ILM on CAPTAIN AMERICA: CIVIL WAR. He then worked on VIRAL and THE CLOVERFIELD PARADOX. Today, he returns to the MCU with AVENGERS: INFINITY WAR.
How did you get involved on this show?
We had worked with directors Joe and Anthony Russo and the Marvel team on the last two films, which were really building to this.
What was your feeling to be back in the MCU?
So great! This marks my fourth film in the MCU, and I never really left. One of the great things about it, is that the team at Marvel aren’t afraid of change or trying new things, which makes it an exciting place to be. It’s never boring and you’re never dealing with the same old thing. Nor do you necessarily know where you are going to go next, which makes it really exciting. This film is a great representation of that.
How was this new collaboration with the Russo brothers and VFX Supervisor Dan DeLeeuw?
Amazing. Joe, Anthony and Dan are truly collaborative, always asking questions and listening, and generally including everyone in the process. As Dan has said many times, “the best idea wins, no matter the source,” and its true. Everyone on the team is constantly striving to make things better and have the effects better serve story. In the end, story rules. Couldn’t ask for more.
Having had experience on two prior films with the team really helped. We have a great shorthand and it gave us the confidence to show shots and concepts early and often even if things weren’t quite ready for prime time. It was important to the show to keep everyone in the loop.
How did you organize the work with your VFX Producer and the ILM VFX Supervisors?
Like most VFX houses these days, ILM has a global presence with offices in San Francisco, London, Singapore and Vancouver. We ran this show out of our San Francisco office, where we had a great production team led by Katherine Farrar and Jeanie King.
With Katherine and Jeanie leading the charge, we looked through all the shots and and came up with logical breaks in locations and action so that we could keep beats together, trying to avoid splitting shots up too much between our studios. Even though we were splitting up beats we were sharing assets, tools and knowledge across each of the offices.
I worked closely with Associate VFX supervisor Robert Weaver, Animation Supervisor Kevin Martel, Compositing Supervisor Nelson Sepulveda and CG supe Doug Sutton. We would start our day with internal reviews with London in the mornings, and end our day with reviews with our Singapore team in the evenings.
Our London team was headed up by VFX Supe Dan Snape and Producer Danielle Legovich. The Singapore team was led by VFX Supervisor Jeff Capogreco and Producer Cabral Rock.
What are the sequences made by ILM?
The bulk of our work was the Wakanda battle. We worked on the sequences that take place in and around Wakanda. This included the flight into Wakanda, the work in Shuris lab, the dropships entering and crashing, as well as all of the battle on the savannah. It also included the Hulkbuster fight in the jungle and some of the shots of the characters blipping out as Thanos wreaks havoc.
Can you explain in detail about the creation of Wakanda during the arrival of the Avengers?
Having worked on the Airport battle in CAPTAIN AMERICA: CIVIL WAR, we had a good feeling going into this one that we would need to be able to create any shot fully digitally to be able to support the storytelling and realize the Russo Brother’s vision.
The sequence was shot on a horse farm outside of Atlanta. Production designer Charlie Wood designed and built a savanna set on the location with a 300 ft long river, which essentially was the starting point for our battle. Along with the river, there were a few African looking trees as well as grass that had been planted months before to get to the right height for shooting. There were a half dozen or so different areas on the farm that were set for the hill our heroes arrive on and the run, the battle, the jungle, etc.
We recreated the world from the ground up starting with Lidar from the location, aerial photography and lots of additional set photography. That coupled with Production artwork and general African reference became the basis for the world.
Our Vancouver office had worked on the city of Wakanda for BLACK PANTHER, so we had a great starting place for the city buildings. The biggest difference for us was that we had to add the savannah for our battle, an area that didn’t exist in BLACK PANTHER. Our Generalist supervisor Johan Thorngren worked closely with Shane Roberts and our generalist team to resurrect the city and create the surrounding hillsides and jungle.
We worked simultaneously in our core pipeline to develop the close up areas near camera that would have interaction with our characters. We built a lego kit of african trees grass and plants, as well as all the vegetation that existed on set. We modelled the grass, rocks, trees using maya and speedtree. We setdressed our sets with the slew of assets and variations we created and ingested using our proprietary set dressing tool, Metropolis. In conjunction with ILM’s procedural system, Dataflow, we covered the vast landscapes,with foliage like different grass variations, flowers, and wheat as curves. This gave us the fidelity to adjust things like density, length and color that affected the look of the meadows, depending on the shot and the camera’s perspective. Since we were generating them as procedural curves, we also were able to do things like crowd and hero collisions as they trampled through fields, and control wind to give it more life when they weren’t interaction with them. Instead of having a fully textured and lookdeved ground, we relied on ground projections in the lighting side to get a mix of rocky, gravel, and grassy look for our different shots. Our lighting team also had controls to the procedural grass system to get a more specific layout, in case we needed to see more dirt, sparser grass, or more percentage of a grass variation in different parts of the ground.
Hulkbuster is back in action. How did you update the model since Civil War?
Hulkbuster in INFINITY WAR is a completely new model we created based on art from Ryan Meinerding and the visual development department at Marvel. Working with our hard surface model supervisor Bruce Holcomb, we took the artwork and worked to bring it to life in three dimensions creating the complex character model. We worked off the overall proportions of the Hulkbuster suit from the last AVENGERS film, but in the end it was an entirely new build trying to showcase the next generation of suit.
How did you handle the challenge of the reflective aspect of the armor?
The challenge in reflective surfaces is that each and every curve and line is important as its hard to hide any imperfections and you don’t want to end up with oddly shaped pieces. It requires a good industrial design knowledge and attention to detail. When it comes to lighting reflective objects, so much of it is dependent on the environment it appears in, and we had to take care to build our environments and lighting rigs in ways that always made the suit look its best. We would then hero light each shot adding caustics from the water and big bounce / fill just as the DP would on set.
Can you explain in detail about the design and creation of the Black Order?
The Black Order were based on artwork from Marvel and maquettes that were created by Legacy FX based on the Marvel designs. The Order were shared by multiple facilities, so we were involved at different times on different characters. Cull was an ingest. Proxima was worked on by multiple studios and we ingested the final asset as well. When it came to Corvus we were very fortunate to have Lana Lan as our creature model supervisor. Lana worked closely with modeller Sven Jensen on bringing corvus to life. Simultaneously while working on the sculpt, Gareth Jensen our texture paint supervisor brought his face to life.
Can you tell us more about their rigging and animation?
Kevin Martel, our Animation Supervisor and and Associate Animation Supervisor Shawn Kelly, did a great job of keeping consistency throughout all our locations. For Corvus Glaive, we tried to make him controlled and deliberate in his actions. His fighting style was smoother and had more finesse than Proxima’s. We approached her as being more of a broadsword fighter, compared to Corvus who would be closer to a fencer in his approach to battle. We kept Corvus’ poses with clean lines of action, like a snake ready to strike. Cull Obsidian on the other hand is a power puncher in the truest sense. He uses his brute strength and power to crush his adversaries without as much calculation as the other members of the Black Order. This allowed Banner, while piloting Hulkbuster to get the upper-hand on Cull in their showdown.
We simulated most if not everything for each character. This included muscles, cloth and hair. ILM’s in house muscle system was used to preserve anatomy and musculature, our in house PhysBam solver to simulate cloth, and haricraft for Proxima’s hair. For the cloth it was important to achieve the desired material properties for the costumes of the black order, especially under different actions and movements. For Proxima’s hair It was important to get the behavior of her hair looking correct under different situations. When she was idle we wanted her hair to flow naturally in the wind and when she was in combat we wanted her hair to behave and move correctly. Corvus proved challenging because of his costume. His cape and hood had a lot of dangling bits that were simulated. Similar to Corvus, for Cull Obsidian we ran cloth and rigid sims for his garments, around his chest, shoulder and loin cloth.
Can you tell us more about their facial animation?
Proxima Midnight had some important dialogue to deliver to the heroes before and during battle so we utilized our state of the art facial motion capture technology for her. We tracked Carrie Coon’s body and facial performances and then remapped them to Proxima’s facial anatomy and character. It was very important to us to keep the integrity and accuracy of Carrie’s performance intact through the process. For other members of the Black Order we relied on a more traditional keyframe approach using combinations of actor and animator reference material.
There are many fights during this big battle. How did you manage so much choreography?
The Marvel stunt team in Atlanta created the basis for our main fight choreography. It was at that time that they began to develop the style in which all the characters would fight one another. We shot motion capture of all the main beats with them, and then mixed in keyframe animation for the more alien and animalistic way that the outriders would brawl. We also utilized our motion capture stage at ILM to continue to build out the choreography, creating additional motion vignettes that we could populate the scenes with. We knew that it would be important to choreograph larger groups fighting each other at one time so the battle scenes didn’t look like a bunch of one on one boxing matches. In most of our favorite big film battles that involve clashing armies, the fights are messy and characters cluster together and switch off fighting partners. We made sure we did that in the motion capture and animation as much as possible. Once we had built up our large library of animation clips Jim Van Allen and team then worked within our Houdini crowd tool to distribute and choreograph the overall shots. In the end not only are the outriders CG, the vast majority of bg wakandans are as well.
Can you explain in detail about the design and creation of the Outriders?
The design of the outriders came to us from Marvel design team. We ingested the model from Framestore and picked up where they left off. In terms of materials and anatomy we worked to make them as scary and believable as possible. We added a lot of variation to the skin and surfaces, adding the ability to reveal damage, raw wounds, based on maps generated from our FX team.
With their specific anatomy, how did you handle the rigging and animations?
We had a great team of rigging artists headed up by Creature Supervisors Abs Jahromi and Aaron Grey. The outriders anatomy is bipedal in nature, but we knew we didn’t want them to look like guys in costumes. Using their extra arms in a way that would give them an advantage on the battlefield seemed the most logical way to begin thinking about the way they could move to separate them from their human counterparts.
Preserving volume and achieving correct behavior in their musculature was very important. Aside from quality, the speed at which the muscles simulations ran was very important because of the amount of hero outriders we had at any given time. We successfully implemented a simulation setup that could be quickly iterated on and yet give us the quality we needed.
For shots that needed a lot of hero Outriders, we optimized our rigs to allow for many to be active in one shot for an animator to animate with. Furthermore for the purposes of animation, we matched the rigs of both the Outrider Soldier and Lieutenant and made sure that their skeleton and control rigs aligned. That was existing animation cycles could be reused.
We developed a toolset for animation that helped with limbs being torn off. An animator could simply choose a limb and be given the option to detach it from the rig and animate it independently. The attachment and detachment process then carried on through the pipeline and cached ready for Lighters to pick up.
When we first began to make them run as animals on all legs we looked at gorillas and apes since their anatomy was the most relatable in nature. This was a good start but the results didn’t feel scary enough for what we were after. These creatures were described to us as things from a horror film, and the softer loping run of an ape seemed too familiar. Basing their gait and locomotion on wolves garnered similar results. With Dan’s direction we started pushing them more towards insects in their posture when running on the ground. Lowering their bodies and keep their limbs bent at the elbows like insects, but still keeping their heads focused and predatorial like packs of wild dogs. Faster, more unpredictable motions made them feel more dangerous. They could attack low or raise up the their full height to take out their opponents. We played the extra limbs in battle as if they had an extra brain. Mirroring the limbs actions too much made for uninteresting silhouettes and didn’t give them the advantage in a fight Marvel wanted. In the end we opted to give each limb its own purpose which made for much more interesting poses and actions.
Did you received specific indications and references for their animations?
A combination of rabid dogs, insects and horror films inspired most of the outrider motion. We put together a reel of horror film clips to help set the mood we wanted to capture. We looked at films like THE GRUDGE, THE RING and many others. We knew we wanted them to be unsettling creatures so having those ideas anchor them was an important step in making them come to life.
Is there something specific that gave you sleepless nights?
I think it was all the moving pieces, both in the shot creation itself and editorially. I liken it to building a car as you are driving a hundred miles an hour down the highway. What could go wrong?
We always are striving to use the latest and greatest tools to produce the most believable images on screen, which means you’re never certain that things will work. Fortunately, we have a great team of talented artists that put it all out there to make it happen. Credit goes to our amazing artists and technology teams for all their hard work and dedication to the craft.
What is your favorite shot or sequence?
It’s hard to pick one, but while the overall sequence was a blast I think I’d go with the initial reveal / clash of our heroes and the baddies, Thor’s Arrival, and Hulkbuster / Cull fight.
What is your best memory on this show?
My best memory would be working with the talented team all around. Marvel has a real knack for hiring great people, and working in a collaborative way. It was great to work with Dan and Jen again as well as the whole team starting all the way at the top with Kevin Feige, Victoria Alonso and of course Joe and Anthony Russo.
How long have you worked on this show?
We had about 7 months of shot production coupled with about a year of various stages of visual development tests and pre-production.
What’s the VFX shots count?
In total we worked on about 600 shots, but as edits come together, not all of them ended up in the final film.
What was the size of your team?
Globally our crew size was close to 400 people.
What is your next project?
After a brief stint on ANT-MAN AND THE WASP, I am onto the next AVENGERS film with the team.
A big thanks for your time.
// WANT TO KNOW MORE?
Industrial Light & Magic: Dedicated page about AVENGERS: INFINITY WAR on ILM website.
© Vincent Frei – The Art of VFX – 2018
