Jeff White explained to us about the work of ILM in 2012 on THE AVENGERS. Then he worked on TRANSFORMERS: AGE OF EXTINCTION. He explains in detail today his work on WARCRAFT.
How did you and ILM get involved on this show?
WARCRAFT was probably the first project ILM has done that started with a connection on twitter between @ManMadeMoon and @ILMVFX. Jason Smith and I had just finished working on AVENGERS when we heard about Duncan Jones directing WARCRAFT. Not only were we huge fans of MOON and SOURCE CODE but we had grown up playing Blizzard games. We started out working on a piece for Comic-Con 2013 that ended up opening the film.
How did you work with director Duncan Jones and VFX Supervisor Bill Westenhofer?
Imagine making a huge fantasy film with your nerdiest best friends, that’s what the experience was like on this film. Duncan made the move from independent filmmaking to huge visual effects project seamlessly. He completely understands the technology and isn’t afraid to push the envelope of what’s possible but always in service of the story. He’s not only an incredible filmmaker but a genuine and wonderful person to work with. He’s also the first director I’ve ever worked with where we swapped stories about editing config.sys files to get games to run in DOS back in the day.
Bill Westenhofer was fresh off of LIFE OF PI and a subject matter expert on all things WARCRAFT. WARCRAFT was an incredibly complex, technical shoot and Bill orchestrated all of the motion capture setup and shoot methodology. He was also heavily involved in the design process of the characters with Blizzard. Bill was an excellent guide for how the world was physically laid out and all of the creatures in it. Bill’s experience with creating memorable creatures was invaluable and we had a great time trying to figure our way through the problems that a film of this scope presents.
On the ILM side, we had Hal Hickel as the Animation Supervisor. Jason Smith, Nigel Sumner and I worked as VFX Supervisors on the project.
What was their approach and expectations about the visual effects?
When Duncan talked to us about WARCRAFT, his idea for the film was that the story should be told equally from the human and orc perspective. Instead of the orcs being nameless monsters, Duncan wanted to present the orc characters as being complex and layered, with motivations that could be read as good or evil.
The audience needed to be able to engage with the orcs and care about Durotan and Draka’s plight. What helped us do that was that they were very human in look. Durotan was a reflection of his character, handsome and noble while Gul’dan was hunched and twisted from the effects of the Fel.
What was your feeling to enter the world of Warcraft?
Excited and yet daunted at the prospect of bringing to life a world that has such a passionate fan base. Blizzard has a long history of creating ground breaking cinematics for their games and have established an incredibly high bar. For WARCRAFT, we fully embraced the Blizzard aesthetic of vibrant color palettes, fantasy landscapes and exaggerated proportions. It was a true balancing act to make that look real and required extensive input from the ILM Art Department and our Asset department to mold that aesthetic into something that would work in a live action film.
We had a lot of fun peppering the visual effects work with Easter eggs and visual references that would let the fans of WARCRAFT know that this film was being made by a team who deeply cared for the source material. From the harvesters in Westfall to the summoning stone in Elwynn Forest, we wanted the world to feel authentic to the fans. A great example of this was Bill Westenhofer creating a Murloc in Blender for a single shot where the Callan rides across a bridge.
Can you describe to us one of your typical day during pre-prod, on-set and during the post?
Pre-production was a scramble planning for a huge and complex motion capture shoot. Duncan and Bill were committed to getting the orc performances on set instead of isolated in a motion capture stage. The teams at Profile Studios and Animatrik partnered to build massive motion capture volumes in environments ranging from Elwynn Forest to Gul’dan’s tent. Elywnn Forest was incredibly difficult because of the camera occlusion from the leaves, horses, armored knights and fire on set. They tucked mocap cameras into little cutouts in the trees and had other towers of cameras they could fly in to help get coverage in a particular area.
Up to nine orcs could be displayed in real time, superimposed over the actors on Duncan’s monitor and through the lens for the camera operators. This was incredibly useful for judging performances and making sure that the orcs weren’t standing inside of each other given how much wider they were than the human performers. Blackhand had massive skulls on his shoulders and we used the real time system to show Clancy Brown how much he needed to turn his shoulders in order to look left or right. The end result was a lot less post-manipulation to fix the data.
The moment when everyone began to realize the power of the technology was when we were shooting a scene inside Gul’dan’s tent. Duncan was able to shoot as many takes as he felt he needed to get the actors performance. Then we’d have the actors sit down and the camera operators could shoot coverage angles viewing the mocap performance playback through their viewfinder. They loved the speed at which they could try a variety of angles for a scene and we had perfect clean plates.
The movie is featuring lots of CG characters. How did you approach them?
Each character had their unique challenges, but there was a lot of overlap between them for the tool development that was required. That included building out our pipeline for facial capture solve, a brand new hair styling tool we call ‘ILM HairCraft’, look development in The Foundry’s Mari and many other smaller but no less important projects. We looked for reuse as much as possible but also put a significant amount of time into making sure our hero characters looked unique.
Can you explain step by step about the creation of Durotan, Gul’dan and the other Orcs?
Everything started with the concepts created at Blizzard by Wei Wang, he did an amazing job capturing the likeness of each of the hero orcs. ILM’s Art Director Christian Alzmann, then took those concepts and incorporated the likeness of each actor into the designs. He made sure that key facial lines were aligned between the orcs and humans playing them to get the best possible fidelity out of the facial capture.
The model build and facial library creation was a lengthy process. Durotan was modelled by Jung-Seung Hong and Sunny Wei and Gul’dan was modeled by Kris Costa. The hair grooms and costumes were particularly difficult. We spent a lot of time trying to figure out how to incorporate the Blizzard aesthetic but still allow the characters to move. For instance, Durotan has a huge waist belt that looks great when he’s standing up but doesn’t allow for much flexibility when sitting down or fighting. The costumes were so complex, we actually held separate dailies sessions just to look for costume and hair intersections because the full hair renders were so expensive computationally.
Can you explain in details about the performance capture and especially the facial work?
Duncan’s goal with the orcs was for the audience to stop thinking about them as CG creations and connect with them as characters. With the very human design of the orcs, we knew that the facial animation would be critical to that goal. All of the filmmakers were understandably nervous about how well the facial capture would work and if performances would translate to the orcs with their exaggerated proportions and huge tusks. To that end, we shot the Durotan and Draka tent scene very early in the production schedule and ILM’s Brian Cantwell and Paige Warner used it to architect our facial capture pipeline build for WARCRAFT. About two months into the shoot, they sent us the results and there was a collective sigh of relief. The actors were excited to see how well the system captured even the most subtle facial movement.
Our approach was letting the facial solve do most of the work so that the incredibly talented animators at ILM could spend their time finessing the performances as they were translated from human to orc. The system worked by using the stereo track of the dots on the face to drive the dialing of ILM’s Fez facial shape system. We then ran the footage through our SnapSolver which analyzes the imagery frame by frame and ensure the actor’s mesh was locked to the tracked dots, even if the performance was outside the bounds of the rig. We tracked the pupils for eye movement and then transferred everything to the orcs. The lips were solved to match the human actors and then run through a multipass simulation using a custom built tusk deformer inspired by a Pixar development for MONSTER’S UNIVERSITY to push the corners of the mouth out and have the appropriate tension on the lips.
We had over two hours of solved facial capture on the show that fed the hero and background orcs. The key to making this process work on the scope we were attempting was that Duncan fully committed to his actor’s performances. He treated it just like a performance that was shot with a live action actor and didn’t try to mix and match different takes on the face, which often makes them look unnatural and disconnected.
A great advantage to the system is that we can capture facial performances onset. This allows the environment to influence the performances. For instance, there’s a scene where Draka, played by Anna Galvin, jumps into a river with baby Thrall. Anna actually jumped in a very cold river in the middle of a Vancouver winter and winced slightly which came through the facial capture. That kind of detail is hard to achieve when capturing in a grey stage or key framing a performance after the fact.
Can you tell us more about the shaders and textures work?
Given how human the orcs skin was going to look, we re-evaluated our skin shading technology and re-achitected much of it. We had extensive reference capture sessions both in the Lightstage and various outdoor lighting environments. All of the orcs shared the same topology, which was critical as we worked on look development of the skin as we could easily share advancements between characters. The more orcs we added to the roster, the more this cross pollination happened.
After getting the base skin working, a shader tree was built by Vick Schutz and Damian Steele to apply tattoos, blood, dirt and dust. For the level of Fel on the skin, we had a fully procedural system built by Frankie Kwak that could take any orc from tan to full Fel with a single slider.
How did you handle their rigging?
Creature Supervisor Brian Paik used ILM’s rigging system, BlockParty 2 for all of our CG characters. It’s a volume-based rigging system that also allows compartmentalization of the rig into modular blocks. The rigs could be created from the creature geometry directly, using knowledge about where anatomy falls within the mesh of the character. The orc clans were built from a kit of parts that were all designed to be inter-changeable.
We created a certain number of hairstyles, teeth sculpts, weapons and clothing pieces and generated orc variations procedurally from that library. BlockParty 2 could deform the costumes to match the orc’s proportions which allowed us to use the same costume pieces for male and female orcs.
Every orc had a full muscle and skin setup for secondary dynamics and skin slide. The most complex aspect of the orcs was actually their costumes. Layers of cloth, fur lined leather, dangling bones and ropes all had to be setup for simulation, even the rings in their tusks. It was some of the most difficult creature work we’ve every done and we were constantly amazed at the level of complexity that Brian Paik and his team of character TD’s were able to pull off for this film.
Can you tell us more about the baby Orc?
Though he grows to be a fearless leader of the orcs, baby Thrall was designed for maximum cuteness. It was actually one of our most challenging models to get right. The baby was sculpted by Marco Di Lucca who absolutely captured the cuteness of a baby while making it just the right amount of orc. We looked at hundreds of pictures of human and primate babies to find the blend features that made Thrall distinctly orc.
Which character was the most complicated to created and why?
What was amazing about this show was how hard so many of the characters were. Durotan had numerous damage variations and a complex braid style for his hair. Gul’dan was covered in dangling skulls and ropes and had a cloak made of strips of cloth that wove through spikes on his back. King Magni had one of the most complex beards we created for the show that actually looped down through his belt creating a simulation nightmare. Blackhand transformed into a fused spine monster. The Gryphon was covered by thousands feathers generated using a hair on hair system. Each character came with its own unique set of technical and artistic challenges and there are so many more in the WORLD OF WARCRAFT that we hope we get a chance to create.
How did you work with the stunt teams for the fight?
The stunt team was led by Stunt Coordinator Tom Struthers. Every day they patiently donned the mocap suits, dots and helmets for all the battle and action scenes. Stunts went the extra mile for us, motion capturing wire pulls so the impact of the orc weapons had the appropriate impact on their foe. They also dedicated a considerable amount of time to filling out the crowd motion library for the big battle scenes.
Many shots involved big amount of Orcs. How did you created these crowd shots?
We knew that creating the hero creatures would take a significant amount of time and focus, so we partnered with Joseph Kasparian and his team at Hybride to provide the crowd simulations and renders. We built a pipeline to share our lighting, environments and cached animation of our first few rows of hero orcs. Hybride would simulate and render the crowds and deliver them back as elements. They could swap clans, weapons and actions on very tight turnarounds. Hybride’s crowd system could respond quickly to specific direction from Duncan for smaller shots while scaling to handle thousands of orc and human knights fighting each other.
The movie takes us to various locations. How did you create them?
ILM’s environments supervisor Jerome Platteaux is a huge WORLD OF WARCRAFT fan. That was useful because Duncan wanted to embrace a ‘small world’ idea where you could see several different locations from the game in one shot. It allowed us to have a lot of fun placing Redridge mountains in the background of an Elwynn Forest shot. Fans of the game will see that we were pretty accurate to the way the world is laid out.
What makes WARCRAFT fun is that each environment is so unique. We sent our photography team out to the closest real world equivalent we could find to gather material to use for backgrounds which is why Westfall looks suspiciously like the rolling hills of Marin County. The photography was useful for a base but in general we had to build almost all environments in 3D because of the complexity of the camera moves and necessity to be able to place the camera anywhere.
Which location was the most complicated to create and why?
Black Morass was by far the most complicated environment we had to create. We actually built it as one physical space where we could place our camera anywhere in the scene and get a background. We had scenes in the Frostwolf camp at night, Durotan and Orgrim talking on a ledge during the day and we followed King Llane and his soldiers all the way from the Swamp of Sorrows to the Portal at the other end of the camp. Each time it appears in the film, we had to redress based on the state of the portal construction and Gul’dan’s Fel magic eating away at the vegetation.
The magic is very present. Can you tell us more about its creation?
We started by finding the highest-level mage on our crew and screen capturing a variety of spells from WORLD OF WARCRAFT to draw inspiration from. Like the characters and environments, we wanted the WARCRAFT fan to be able to watch the movie and know exactly which spell was being cast, be it teleportation or arcane barrage.
WARCRAFT magic is unique in that it often requires multiple steps for the mages to cast them. To work out timing and key moments we built rigs for the animators to lay out the key events and movement to the spell to be able to iterate very quickly. We then passed that rig to the FX Technical Directors’s to simulate and our FX Supervisor, John Hansen drove the look of the Houdini simulations. The spells also include graphic elements like intricate line work and runes appearing in the air.
Did you receive specific indications and references from Duncan Jones about the magic?
Blizzard was a great help here as well in the design of the magic. They did numerous concept pieces for how the spells might appear and we worked on translating that into motion. The actors also played a big role. We had lengthy discussions with Ben Foster and Ben Schnetzer about what they were doing at each stage of the spell and designed around their actions. The props department also built LED rigs that were attached to their hands and gave us the right amount of interactive light as the spells began to form.
Have you improved some of the ILM tools such as Plume or Fracture for this show?
Our big development on the show was a hair grooming tool called HairCraft and was headed by Stephen Bowline. Durotan’s groom, done by Ryan Gillis, has tightly braided head hair, large braids down his back and front, loose hair, chest, nose, ear, eyebrow and knuckle hair, even peach fuzz on his cheeks that broke up the sheen on the skin in a very organic way.
We also built a tool called LightCraft that allowed us to take two HDRI’s shot on set and perform a stereo solve to determine each lights distance away from the characters. We were able to get much better first pass renders with this tool which allowed us to spend more time on creative lighting.
At a moment, Medivh transforms into a big demon. How did you created this transformation?
We started by creating a digital double of Medivh in his human form. This asset was used as the basis for a semi-demon form, using the same geometry. Most of the semi-demon form was created on screen using prosthetics on Ben Foster. Because Medivh, the semi-demon and the big demon all used the same creature mesh, we were able to use BlockParty 2 to generate rigs that we all identical aside from shape. During the transformation, we interpolated one rig to another, with a lot of help from the modeling team to help us make the transformation smooth.
How did you organized and split the work amongst the different ILM offices?
San Francisco, Vancouver and Singapore all worked on WARCRAFT and all had very hero orc sequences to complete.
How was your collaboration with the other studios and their VFX Supervisors?
We worked very closely with Hybride on WARCRAFT. Hybride handled all of our crowd simulation work as well as the entire Energy Chamber scene with the Golem and various other set extensions.
Rodeo FX work was supervised by Arnaud Brisebois and created all of the shots inside the Karazhan library as well as Medivh’s peak in Blackrock Valley and the Chamber of Air. Rodeo contributed heavily to the design of the magic with the push spell in the library.
El Ranchito handled all of wide shots of Stormwind Castle and Volta helped us out with the Funeral scene at the end of the film. All of the other studios were critical in the completion of the show. The supervisors and their crews were incredibly dedicated and collaborative.
What was the main challenge on this show and how did you achieve it?
It was all about the orcs on this show, we wanted the audience to feel that they were just as compelling to watch as their human counterparts.
What do you keep from this experience?
It’s an amazing experience doing a project like WARCRAFT. We went in with the best-laid plans and adapted to challenges along the way. Late in the project we had the Blizzard team of Chris Metzen, Michael Morhaime, Nick Carpenter and several others come up to ILM and speak to the crew. It was a unique and inspirational experience to interact with the original creators of the world and characters we were working to bring to the big screen. We have a tremendous amount of respect and admiration for Blizzard and have been fans of their work for twenty years.
Was there a shot or sequence that prevented you from sleep?
The scope of this project was so massive it quickly becomes one big snowball technical and artistic challenges. The complexity of the orcs, not to mention dwarves, gryphons, frostwolves, crowds, environments was plenty to cause some sleepless nights. It’s amazing to be at place like ILM, however, surrounded by some of the best artists, developers and production talent to work through those challenges.
How long have you worked on this film?
We created our first shots for the film for Comic-Con 2013 and completed our last shots for the film in April of 2016.
What was the size of your team?
At the peak we were well over 1’000 people worldwide.
What is your next project?
Hal and Nigel are on ROGUE ONE: A STAR WARS STORY, Jason Smith is on TRANSFORMERS: THE LAST KNIGHT and I’m in post-production on KONG: SKULL ISLAND.
A big thanks for your time.
// WANT TO KNOW MORE?
– Industrial Light & Magic: Dedicated page about WARCRAFT on ILM website.
© Vincent Frei – The Art of VFX – 2016
Directors cut, Directors cut. Troll Invasion. Thanks for the great work.