In 2016, Chad Wiebe explained the work of Method Studios on DOCTOR STRANGE. He then joins Industrial Light & Magic and returns to the Thor universe for THOR: RAGNAROK.
How did you and ILM get involved on this show?
ILM has had a long and successful relationship with Marvel and THOR: RAGNAROK was just the next step in that relationship.
How was the collaboration with director Taika Waititi and VFX Supervisor Jake Morrison?
It was really amazing. When I was presented with the opportunity to work with Jake again, I immediately said YES. We had worked together twice previously, most recently on THOR: THE DARK WORLD, and while that show was one of the most intense of my career, working with Jake was an extremely positive experience and I always admired his ability to stay calm and collected when things got tense.
It was my first time working with Taika and all I can say is he definitely keeps things light hearted and fun, as you would expect after seeing the unique brand of humor of THOR: RAGNAROK.
What was their approaches and expectations about the visual effects?
Well, having worked on the previous Thor film, I think it’s safe to say that you never want to replicate what’s already been done in a previous film…. even if it’s the same character or the same universe. That combined with the films unique aesthetic and Jack Kirby style used in much of the concept art and set design, things had a very different look than you might be used to.
What are the sequences made at ILM?
ILM worked on the Hulk vs Thor battle, the Hulk Warrior Suite, and the sequence where Thor Escapes to the Quinjet.
How did you organize the work at ILM?
The work was split between our Vancouver and San Francisco studios.
Can you explain in detail about the creation of Hulk?
The Hulk asset was originally created by ILM for the AVENGERS/IRON MAN/CAPTAIN AMERICA films. So we used that as a starting point and upgraded from there.
How did you enhance the Hulk asset?
Throughout the various times we’ve seen Hulk in other Marvel movies, he has always seen slight changes or updates to the asset. Just as each director has a vision for the film he wants to make, it seems like Hulk also falls into the category of never using the same Hulk twice. So previously we’ve seen subtle changes such as hair and color changes, and slight facial features updates. This time we take a pretty big step in the evolution of Hulk .
Ryan Meinerding (Head of Visual Development at Marvel) had a very specific look and direction he wanted us to take Hulk. Gone was the excessive brow ridge and sunken eyes, long upper lip and soft jaw. Instead we had sharp and streamlined features, a chiseled Jaw and forehead, a longer nose reducing the gap between nose and upper lip, adjusted eyes and a brand new haircut. His skin color was also amped back up to a more saturated green. There was a specific direction to make sure Hulk seemed intelligent and more in line with what you would expect from a superhero, and our modeling supervisor Lenny Lee scrutinized the smaller details to create a Hulk like we’ve never seen before.
Can you tell us more about the facial animation?
Since we see Hulk speaking full sentences and holding lengthy conversations, we created a complete set of FACs shapes which would allow him to not only emote expressions, but to articulate conversations with no limitations. We started with a Medusa capture of Mark Ruffalo, capturing a wide range of expressions. From there our modeling team would choose select expressions which were then processed, cleaned, and folded into our proprietary Fez facial animation rigs.
Mark’s in shot performance was captured using a Head Mounted Camera (HMC), with dots applied to his face. This data was then tracked and handed over to our team of FacCap artists led by Kate Lee, who re-target this data using ILM’s proprietary Fez animation system, generating curves which were transferred to the final character. This gives a one to one representation of Mark performance mapped onto his digital representation. Fez allows for more art directed expressions, secondary detail, and increased edibility in animation.
At that point, our animation team (lead by animation supervisor Steve Nichols) took over and we would have to decide how much of the original performance to keep, and how far to go in performance adjustment or augmentation. There is always a fine line between the performance shot on set, and the evolving nature of the digital characters performance. Especially with a character such as Hulk, much of the evolution of his performance happened after principle photography and well into the post phase. So our animators had the challenge of determining in each shot how to retain as much of Mark’s performance and nuances as possible, while at the same time incorporating any new creative direction for Hulks performance.
How was simulated his presence on-set?
Well that’s always a challenge when you have an actor who needs to fight and interact with a creature that isn’t actually there. But for each shot, various props were used to give Chris something to work with. There were various life size Hulk busts used for things like eyeline, interaction and size reference used in shot while Chris was acting. This included an 8’6” full upper-body version of the Hulk that was puppeteered by two SFX technicians.
Hulk is seen through various lighting conditions. How did you handle this aspect?
One of the greatest things about working on Marvel films is that they are all very FX driven, that you really are able to collect as much on set reference as needed. So lighting reference, HDRI’s, set and prop reference was extensively captured for every lighting set up for each shot and set. So creating proper light rigs started by ensuring artists had the reference material needed to re-create and match any in shot lighting setup.
Can you tell us more about the Hulk to Banner transformation?
Taika had a very specific look in mind for the transformation sequence. The 1980 film ALTERED STATES starring William Hurt was a key reference provided, and Taika really wanted to capture the essence of a key sequence in that movie where William Hurts character is essentially trying to bash himself back into his normal human state. For Hulk, since he had been living as Hulk for so long, being presented with footage of Black Widow really sets off an internal struggle. So really trying to capture the rage, emotion and internal turmoil was our goal and our animation team really put Hulk through a wide range of emotional states when working towards the final look.
The transformation moments themselves posed some unique challenged for our Create Dev and Groom artists. Typically, you don’t work with a character with scaling muscles and anatomy, let alone with transitions as abrupt as only a few frames. So many of the parameters for our muscle, cloth and hair sims had to be manually animated to allow for extreme transitions to prevent things from exploding during simulation.
The arena is a really huge environment. How did you design and create this location?
The Arena was a pretty massive model both in physical size as well as model detail, so there was definitely some logistical planning required to be able to deal with such a large asset. Coming it at over 1300ft wide and over 460ft high, it had a seating capacity of over 160,000 crowd agents and used a light rig consisting of around 934 lights. Our biggest shots could have up to 100,000 agents visible at a time so render times could easily become a problem and solutions were needed to make things manageable. We relied on slicing the arena into 24 pie shaped wedges, and 10 individual tiers which allowed us to manage smaller slices as needed and break the arena up into sections which were re-compiled in Nuke in extreme cases.
The floor of the arena posed a unique challenge. On set they had a partial floor build which consisted of small blue rubber pellets that resembled small gravel. But we weren’t quite sure what we should do with it after once we starting building out our arena. We felt like an entire arena floor of blue pebbles might create too much of a sea of blue, so we had to come up with ways to incorporate the pebbles into a larger concept. The idea we proposed was to add a large Sakaarian floor design to the pebbles, keeping it simple, but complex enough to give break up and variation to the floor. The idea from there was that since it was all made from these small pebbles, throughout the course of the fight, this pattern would be broken up, colors would mix and eventually you would end up with a history of the fight, visible though the red and blue floor design becoming progressively more messy. Our Environment supervisor Martijn Van Herk and his team created a procedural set up which would allow us to re-create the floor with the Sakaarian pattern from any angle with infinite detail, and add progressive stages of destruction. This combined with all the destruction elements being added during the fight, would amount to a pretty extreme amount of destruction by the end of the sequence.
Did you received specific references for the arena?
We were supplied various concept pieces which were used as a starting point, as well as the 1959 film BEN-HUR which had some really great wide expansive views from the Colosseum floor looking out over the crowds that were used as reference during some subsequent re-designing of the arena.
How did you handle the crowd creation and animation?
We did extensive motion capture shoots of a group of actors re-creating as many actions and reactions as possible to use throughout the fight. Crowd supervisor Geordie Martinez and his team choreographed all of the motion clips using proprietary tools developed by our London crowd team, which were then simmed in Houdini and then exported to our proprietary platform Zeno, where we would apply cloth sims for any agents close to camera.
How did you created the digital double for Thor?
Well, our Thor asset went through a bit of an evolution as well, due to some extremely close up shots needed of our digital Thor. So skin, textures, hair and costume details had to hold up as well as Hulk since they really did complete for screen time in many shots. Texture supervisor Jean Bolte and Look Dev TD Mike Halsted really went to extreme lengths detailing out Thors skin and costume and to ensure no limitations on how we could use digitial Thor.
Can you explain in details about the various FX work on the huge hologram of the Grandmaster?
For the Grandmaster hologram, the general idea was that we wanted it to seem like he was huge, over 20 stories tall, and wanted any camera work in shot to feel like you were filming something massive. The challenge with that is obviously if your filming a human with traditional filming equipment, its very difficult to emulate something that realistically you would need an aerial photography unit to capture at full scale. Jakes idea to get around this was to set up a 5 camera array to film Jeff Golblums performance, which would then be used to project and reconstruct, allowing us to create a digital Grandmaster and a virtual camera that we could give the impression it was shot using an aerial unit. It gave us much more freedom to really not be constrained by any limitations you would have if you’d tried the same thing using a single camera.
We also created a digital version of the Grandmaster which was used to generate the Hologram FX. We would rotomate the footage provided from the camera array, perform full cloth simulations, and from there lighters rendered the Grandmaster from each camera to get a full top to bottom, 360 degree representation of the Grandmaster. Once we had the reconstructed Grandmaster with all the color information, our FX team (led by George Kuruvilla) projected rays from CG drones onto the Grandmaster and calculated the hit points on the Grandmaster’s geometry, which would be used to distribute particles along the surface as well as inside the volume of the mesh. With the animated drones being in constant motion, as well as the Grandmaster moving throughout his performance, the particles would constant spawn and die off, with some residual motion and forces, to create a look of constantly changing, evolving and reforming points of light. These passes were passed to our comp team (led by compositing supervisors Ivan Busquets and Ben O’Brien) who would compile all the elements and run them through a Nuke set up which was created to give the particles and even more glitchy and holographic quality, almost creating moire patterns, which were a main factor in the overall look of the holograms.
What is your favorite shot or sequence?
I love the Hulk vs Thor battle, but I think my favorite sequence has to be Hulk in the warrior suite. No matter how many times I watched that sequence, I couldn’t help but laugh. Not to mentioned seeing Hulk talk was just epic!
What was the main challenge on this show and how did you achieve it?
The main challenge was probably the fact that anytime you have a live action actor performing against a non-existent opponent, there are bound to be challenges getting convincing interaction, collisions, and its just really difficult to sell a phantom punch. So while there were some great props being used and Chris’s performance was amazing, there’s only so far you can go with them before it starts to reveal limitations. So we heavily relied on our digital version of Thor anytime we needed to sell interaction, connect a punch, or overall alter part of a performance. So arm, torso or full replacements were extensively used to ensure no limitations on how far we could go to make a shot work.
What is your best memory on this show?
Being my first show with ILM, the best part was getting to know the team I was fortunate to be able to work with, in addition to working some some old friends who I haven’t worked with in a while. Some great time’s were had!
How long have you worked on this show?
My part on the show was just about 12 months.
What’s the VFX shots count?
Just over 400 shots between both ILM Vancouver and San Francisco.
What was the size of your team?
130 give or take a few.
What is your next project?
Too soon to tell, but some exciting possibilities!!
A big thanks for your time.
// WANT TO KNOW MORE?
Industrial Light & Magic: Dedicated page about THOR: RAGNAROK on ILM website.
© Vincent Frei – The Art of VFX – 2017
