In 2015, Vincent Papaix told us about the work of Digital Domain on Lost River. He then worked on Furious 7 and Deadpool and joined the ILM teams in 2016. There he worked on films like Aquaman, The Irishman, Terminator: Dark Fate and The Stand.
Mathew Cowie has been working in animation for over 20 years. He joined ILM in 2012. He has worked on many films such as Doctor Strange, Black Panther, Aladdin and Eternals.
How did you and ILM get involved on this show?
Vincent Papaix (VP) – The Battle of New York recreation was part of the script early on. ILM, having done the original scene in The Avengers, we knew we could help recreate that scene from another angle. Jeanie King, our VP of Production, was our Executive Visual Effects Producer on the show and was involved in the show really early on.
How was the collaboration with Directors Rhys Thomas, Bert and Bertie and VFX Supervisor Greg Steele?
VP – We collaborated closely with Greg Steele and David Bosco at Marvel. We would present our work to them and they in turn would present it to be reviewed in context by the filmmakers.
What were their expectations and approach about the visual effects?
VP – Hawkeye’s story is grounded in reality. We are in New York City at Christmas. It was important to Greg and the filmmakers to recreate Rockefeller Plaza as accurately and realistically as possible. Our work was mostly matching tiles photography and making sure the image had a cinematic and photoreal look.
How did you organize the work with your VFX Producer and split the work between the various ILM offices?
VP – Ali Greene was our VFX Producer at ILM and together we worked hard to establish strong communication so that we could be as efficient as possible with our resources. The show was originally planned to be done entirely at ILM’s Vancouver studio, but as the scope of the work grew, we were able to tap into some of our talent in San Francisco. Due to the nature of working from home we decided to run the two locations as one unit, which meant we were able to cast the right talent for the right work, including a few of our key supervisors. It’s decisions like this we made as a team that led to the success of the show.
What are the sequences made by ILM?
VP – In Episode 1, we recreated the New York Battle as seen in The Avengers.
In Episode 2, we did all the bluescreen extractions in the Bishop office.
In Episode 5, we helped recreating the red serum as seen in Black Widow.
In Episode 6, we did the final battle scene from when Yelena and Kate jump off of the 23rd floor until the end of the battle on the Rockefeller Center ice rink.
The series opens with an impressive new look on the iconic New York battle from The Avengers. What was your approach to tackle this?
VP – It was really important to recreate the spirit from The Avengers. We restored as many ILM assets as we could from that movie and upressed geometry as needed based on our new camera angles. For The Avengers, the ILM team did a really impressive and comprehensive photo survey of New York city from hundreds of points of view. On Hawkeye, we started by using those photos to see what we could get out of the box. We quickly realized that the camera angles would be too different from The Avengers. We decided to go for a full CG approach. Our brilliant Environments team supervised, by Matt Lee and led by Ryan L. Jones, organized and built the section of the city we needed for our shots.
The work was divided amongst several artists. The environment work was done in Autodesk 3dsMax and rendered in Chaos V-ray. The Chitauris were lit and rendered in our traditional Foundry Katana / Pixar RenderMan pipeline, supervised by John Iskandar.
Mathew Cowie (MC) – This sequence required us to seamlessly intercut from the original Avengers footage, specifically when Hawkeye leaps off the building. For the animation team, the most important part was to ensure that our Hawkeye digi-double holds up to Jeremy Renner’s original performance. Jeremy does a great job of sharply hitting his action poses, while fluidly firing his arrows. It’s second nature to him. One of our in-house motion capture performers, Nathan Camp, did a fantastic job of studying and mimicking his movements. This allowed our animator on the sequence, Derrick Carlin, to work from a really strong foundation and further push the performance where needed.
What were the main challenges with this sequence?
VP – It was really important to stay true to the look that was established in The Avengers, and yet, created a different emotion. This time around, we were seeing the Battle of New York through the eyes of little Kate. This is the moment she decided to follow in Hawkeyes’ footsteps and become her own version of a superhero. Building the tension and the dangers was key for this sequence, the camera work was really important. For the rest, the brilliant team at ILM had a lot of fun crafting the shots to a very high standard.
Can you elaborate about the Rockefeller Plaza environment creation?
VP – The onset team did a full scan and texture shoot of the Rockefeller Plaza in New York City. They also filmed a handful of shots there during principal photography. Later on, they built an Ice rink replica and the entrance of 30 Rock on a stage in Atlanta. Our Asset and Environment team led by Brian Giacoppo and John O’Connell carefully recreated the Rockefeller Plaza to be true to the real location. It was an insane amount of work to match every single detail. We had to build interior stores and offices and all of their contents to make sure we will get the proper parallax and light response. The work was mostly done in 3DSMax and rendered in V-ray. The street needed to be alive, so for this, we created CG crowds to walk around the plaza. Phil Rouse, our crowd supervisor, worked closely with Brian to create the hundreds of CG characters.
It was really fun to compare our reference tiles photography to our full CG build, and we realized that we had a pretty good match. Our compositing team, supervised by Ziad Shureih, did a great job at analyzing the tiles to polish the integration between our CG and the plate photography. Ultimately, the full CG approach proved valuable to give us the freedom to create any camera angles the filmmakers were after. I’m very proud of our Environment team to have received a VES nomination in the Outstanding Created Environment in an Episode, Commercial, or Real-Time Project category for their amazing work creating the Rockefeller Plaza.
Can you explain in detail about the iconic Rockefeller tree?
VP – The iconic Christmas tree was a beast! It has over 50,000 individual lights and millions of needles. We started the asset from a base model from Unity/IDV SpeedTree to match the characteristics of a Spruce tree. We created 3 different assets based on our needs. One for Close up, to have more control on matching the plate branches, and a second of the full treetanding, for the first part of the sequence, and the third for when it has fallen, for the final battle in the ice rink. To add to the difficulty, the tree needed to be simulated at small scale (ambient wind) but also at large scale (tree falling in the ice rink). We also had to match and extend some tree branches when Clint was interacting with it.
How did you create so many lights and needles?
VP – Our CG supervisor, Georg Kaltenbrunner, worked closely with our FX artist Himanshu Joshi to create an efficient workflow in Houdini to scatter the needles, cables and lights on the tree branches.
How did you handle the lighting challenges?
VP – The tree was proven challenging to render. Thankfully our brilliant team at ILM leveraged some pretty clever workflows integrating Pixar’s RenderMan to optimize the way the light will raytrace in that incredibly complex asset.
Can you elaborate about the trick arrow effects?
VP – Every trick arrow had to be unique. We received some fun designs from the onset team for some trick arrows. Additionally, our concept artist, Igor Staritsin, created more designs. The sequence has 12 unique trick arrows. It was a super fun process to come up with ideas and find a practical way to unleash the weapon.
MC – For animation, the needle arrow was a fun challenge to tackle. The arrow is shot downward into the ice and spins, releasing tranquilizer darts via centrifugal force into the enemies. Our animator, Marc Beaujeau, set up a complex constraint system using surface and traditional constraints to ensure the darts would release accurately from the arrow and pin themselves to the performing plate actors.
Which trick arrow effect is your favorite?
VP – The PYM arrow, when Kate shrinks down the truck. That whole beat is full CG, we had a lot of fun working with Greg and the team, coming up with framing ideas and details. Then the baby owl comes back to pick up the tiny truck and flies away! It was a great moment to bring to life.
Can you explain in detail about the creation and animation of baby owl “Rocky”?
VP -The owl (Rocky) asset was carefully crafted by our team. We focused mostly on the groom to match the real one. Our Groom artist Jesus Fernandez did a fantastic job in Houdini.
MC – The idea was to make Rocky as curious and innocent as possible while perched inside the tree as a contrast to her attack on the mini-truck later on. We chose to go as photoreal as possible with Rocky’s performance which meant incorporating a lot of nuanced behavior. A subtle head tilt, a slight adjustment of the wings, some micro pupil dilation. Our animator, Meg Grube, dug heavily into reference and was able to find and translate these small details throughout the Rocky sequence.
Is there something specific that gave you some sleepless nights?
VP – Having a newborn during the show was the main reason for my sleepless nights. 🙂
What is your favorite shot or sequence?
VP – We were lucky to work on a variety of VFX work. From recreating the Battle of New York in Episode 1 all the way to the Final Battle in Episode 6. I enjoyed the process from pre-production all the way to final delivery!
MC – I would say the mini-truck sequence. It provided a lot of laughs throughout production.
What is your best memory on this show?
VP – The highlight for me was to work with Greg Steele and the Marvel team. Our calls were always super collaborative and fun!
How long did you work on this series?
VP – One full year.
What was ILM VFX shot count?
VP – 570 shots for the final series.
What was the size of your team?
VP – I feel really fortunate to work with such a talented team at ILM. We were up to 110 artists, production and supervisors on the show.
MC – We had a really great team of 6 animators.
What is your next project?
VP – I can’t say just yet, but I’m looking forward to it!
A big thanks for your time.
// All of the Trick Arrows in Marvel Studios’ Hawkeye!
WANT TO KNOW MORE?
ILM: Dedicated page about Hawkeye on ILM website.
Disney+: You can watch Hawkeye on Disney+ now.
© Vincent Frei – The Art of VFX – 2022
