Back in 2017, Scott Stokdyk explained the visual effects work on Valerian and the City of a Thousand Planets. He then worked as a visual effects consultant on Peter Rabbit.
How did you get involved on this movie?
I was contacted by Technicolor to meet on this project & represent them – I have a long relationship with Joanna Capitano, and respect her opinion on projects & Visual Effects in general.
How was the collaboration with Director Miguel Sapochnik?
Miguel is an extremely thoughtful director and person – his trust is hard-earned, since he expects excellence in everyone he works with. I have an enormous amount of respect for his process, and his vision, and really hope to work with him again!
What was his approach and expectations about the visual effects and especially about Jeff?
My hope from the beginning was for Miguel to direct the movie as if Jeff was just another actor on set. In the VFX post-process Miguel was able to get a second chance at directing the character by working with our animation team – I think he really embraced this process, and helped sculpt and build on top of Caleb’s performance in Post.
For Miguel to Final a shot, we needed it to feel not like a VFX shot – he just wanted to believe that Jeff was in the shot, performing!
How did you organize with your VFX Producer?
Sophie Leclerc & I have a mutual trust & great relationship – we have been through a lot of visual effects challenges together, and always have a good dialog about our work! Many of the decisions on this movie were budget-driven, so we both had to compromise in areas & fight for what we believed was needed in other areas…
How did you choose the various vendors?
I singed on with the Mill (now part of Mr. X) as a primary vendor, knowing that we had the full resources of Technicolor behind them – we used MPC for Mocap expertise, as well as an amazing on-set Supervisor named Mike Woodhead. Later into the process, we added Rising Sun Pictures, who were phenomenal to work with… At first I thought they were overqualified for the shots they were given, but when shots got trickier, this paid off amazingly well for us.
We relied on Fin Design + Effects for our motion graphics – which needed to be clean & precise for the storytelling.
Can you tell us how you split the work amongst the vendors?
Jeff shots were split between a Mill team in Adelaide and a team in Montreal. Adelaide did a lot of the development of Jeff, but on the other hand, the team in Montreal helped during early development & did some subtle beautiful work!
We got lucky to have access to some animators at the Mill Montreal in preproduction, who helped animate to the actors doing casting reads! That is part of what made us realize how great Caleb was – seeing his auditions translated to an animated robot just instantly worked!
Where were the various exterior sequences filmed?
We filmed primarily in New Mexico (all over), but also shot some VFX plates in Toronto, Los Angeles, and San Francisco!
Can you elaborate about the creation of this post-apocalyptic world?
This world was supposed to be completely void of any green living things – we had a lot of help from the art department of physically removing plants in the foreground from a lot of our locations! Those that couldn’t be handled by the greens crew were digitally removed… Most of our VFX environment experience was trying to get as much as possible in camera, then enhancing/fixing if we needed more world-building in an area…
What kind of references and influences did you received for the post-apocalyptic world?
Miguel loved the work of Simon Stalenhag, and also photographer Masataka Nakano’s “Tokyo Nobody” – both artists explore bare landscapes that told a story of abandoned humans & human technology…
How does the specific lighting affect your work?
The cinematographer, Jo Willems, lit Caleb with the same attention to detail as Tom Hanks – from my experience, this is rare, and it paid off for us – I think our CG Jeff creation is lit equally as well as Tom Hanks – it may seem like a little thing, since VFX can always change the CG lighting in post, but it elevated everything in VFX to have crafted Jeff lighting on set!
Which environment was the most complicated to create and why?
Sant Francisco was probably the most difficult, since everyone is so familiar with it – we wanted the Bridge area aged, but believable – we got some amazing footage of the real San Francisco bridge on a rare clear day, but the push-and-pull of making it post-apocalyptic was a delicate line to ride.
Can you explain in detail about the creation and animation of the massive storm and the tornadoes?
At this point in my career, I’ve narrowed my focus more towards the artistic than the technical, so I purposely try to avoid getting into these kinds of details.
My involvement here was to point the Mill towards my favorite references…. I gathered a lot of storm-chasing reference on “Oz The Great and Powerful”, and I took the greatest hits from that footage for all the things I wanted to see – getting good video from inside tornados is elusive – it is mostly from security cameras & dedicated storm-chasers, but it is really harrowing to watch – I wanted to capture some of that feeling.
How did you handle the FX simulations challenges?
I had the Mill start early – there were a lot of artistic choices and choreography in each shot, but the lighting played such a big part that I was always pushing to evaluate the FX in lighting/comp.
From my experience, this kind of work is extremely slow in turnaround, so we just tried to kick it off early & keep pushing on it. The artists really went the extra mile to add more and more details to help with the feeling that you were inside a tornado.
Can you elaborate about the design and the creation of Jeff the robot?
In the movie, Jeff was created with the dog in mind. Every design detail on him is seen though the lens of the dog – Jeff’s hands are made from soft baseball gloves that can interact with him.. the shoes are from old Crocs like Finch wears to smell like Finch…
Other details are things that Finch would have found in his lab – soft padding that would be taken from the aerospace company Finch worked for, camera lenses from surveillance cameras, and even Jeff’s head is supposed to be repurposed from the top of a gas canister!
How did you work with the team of Legacy Effects?
Legacy Effects outdid themselves on this movie – under limited budget, they created the most complicated animatronic they had ever done with Dewey – it was animated with 4-6 puppeteers for each shot.
For the Jeff robot, Legacy built a full Jeff, that could be puppeteered – he had real working motors, and all the wiring was functional – it informed our final CG design to keep the real wires that actually did something, rather than make up our own.
Legacy was able to have their model in shots at the start of Jeff’s introduction, but as soon as Caleb showed up and started performing, we knew we had to use what Caleb was doing!
We still tried to get every bit we could out of the real Legacy robot – including motion-capturing a puppeteering session, to really study how the motors reacted & how the physics worked!
Can you tell us more about the animation work?
Even though we captured mocap with the Xsens MVN suit, It would be a disservice to the 60+ animators to oversell the mocap in our movie – millions of keyframe decisions were made to recreate the ideas of an AI from the year 2030. The mocap data was studied, and used sparingly – it was the best animation reference we could have though – and it was captured from Tom Hanks as well as Caleb…
Our trickiest shots involved rotomating Caleb’s hands to keep the live-action gloves in the shots when we could – because of framing & eyelines, we tried to keep Caleb’s mask position as the front of Jeff’s face… then, because of different proportions between Caleb & Jeff, the animators had to puzzle-out how to get a body performance that connected the head to the hands.
What was the main challenges with Jeff?
Jeff was supposed to be an AI from the year 2030, and it is always difficult to project forward to predict the future. We got inspired by machine learning work, SIGGRAPH research, etc… and we made an animation plan for the progression of Jeff’s AI. The result was a 20+ page “Animation Bible” that Miguel & I wrote which outlined what we wanted from the character in each part of the movie!
Did you have to replace the dog by a CG version in some shots?
Seamus was amazing – such a sweet dog & performer that we didn’t do much with him – just made a meaner expression in one shot, and added him into a shot or two that we hadn’t filmed him in.
Which sequence or shot was the most challenging?
The footage in the tornado sequence was challenging because of the weather variations when we shot – we had 60mph winds that brought in overcast clouds, then blew them away minutes later – ultimately , we had to artistically replace skies that partially covered areas with the supercell, and partially were clear to let hard-light in – it was dictated by the cut, but solvable with good compositing.
Did you want to reveal to us any other invisible effects?
We tried so hard to get everything in camera for the budget, but also the feel of the movie… it was mostly achieved, but some of the interior shots ended up being shot on a blue screen stage… in that case, instead of creating a CG environment, we went & shot driving plates in an area of Los Angeles that had recently been ravaged by fires… their tragedy made for useful VFX footage….
We tried to be thoughtful & resourceful about every thing we did. I ended up compositing about a dozen simple shots myself of the movie, to make sure most of our VFX resources went towards the CG Jeff shots!
Is there something specific that gives you some really short nights?
Interaction shots are always difficult – I tried to mitigate this by using Caleb’s real gloves in as many shots as possible, but Finch interacting with Jeff wasn’t easily solvable. The solution is a delicate balance of animation, FX, lighting & comp, but takes very skilled group of artists to pull off. Being a production-side supervisor, you just have to have faith that these shots are well-cast with the right artists who can pull off miracles!
What is your favorite shot or sequence?
From the start, I knew that the last scened between Jeff & Finch, where they hug would be the ultimate test of our VFX – would audiences feel the connection & care?
A year and a half after delivering the final shots, I was able to watch the scene more objectively, and it still made me get teary-eyed!
What is your best memory on this show?
At the end of shooting – being on the Golden Gate Bridge with beautiful weather – for our shoot, the bridge was intermittently shut down, and commuters & pedestrians were irritated, but no one knew we had Tom Hanks on the Bridge in a Jeff Displacement Suit!!! It was surreal & beautiful.
How long have you worked on this show?
I started (with Sophie Leclerc) in July 2018, and finished at home in quarantine in July 2020 – I would have never predicted where this movie would have taken me, but it was an amazing ride!
What’s the VFX shots count?
Final count was 877 VFX shots – seems small by today’s standards, but every one of these shots got a ton of love & attention.
What is your next project?
I am currently on Borderlands, directed by Eli Roth – very different movie than Finch, but equally full of fun challenges!!!
A big thanks for your time.
// Finch – First Look – Apple TV+
WANT TO KNOW MORE?
Fin Design + Effects: Dedicated page about Finch on Fin Design + Effects website.
Apple TV+: You can now watch Finch on Apple TV+.
© Vincent Frei – The Art of VFX – 2021
