In 2021, Nicolas Hernandez explained the visual effects work of Important Looking Pirates on the first season of Foundation. Then he worked on Obi-Wan Kenobi, Resident Evil and Willow.
How did you and ILP get involved on this show?
Initially the production VFX producer, Kathy Siegel, approached us because of ILP’s excellent relationship and collaboration with Lucasfilm on previous shows. As a fan of the franchise, an admirer of the director’s work, and having previously collaborated with Andrew Whitehurst on a couple of shows (Ex Machina, Annihilation), I wouldn’t have passed up the chance to be involved.
What was your feeling to be back in this iconic universe?
Personally, I felt incredibly fortunate to be part of the most famous archaeologist’s last adventure. Indiana Jones holds a special place among my favourite movies and evokes some fond childhood memories. The whole team at ILP, shared a common feeling of gratitude for the opportunity to contribute to such an iconic franchise.
How was the collaboration with the Director James Mangold and VFX Supervisor Andrew Whitehurst?
Collaborating with the production VFX Supervisor, Andrew Whitehurst, was a great experience. His constructive feedback, expertise, and open-mindedness to creative suggestions made our collaboration truly enjoyable and rewarding. While Andrew effectively conveyed the director’s vision, we were also fortunate enough to have a couple of reviews with James Mangold, the director himself.
What was their approach to the visual effects?
The director’s focus was for the VFX to serve storytelling. So, very early in the project, there was a strong desire to do previs or rough temp of shots in order to feed the edit as quickly as possible.The goal was to ensure that the sequences were working in context of the cut. Also both the director and VFX Supervisor fully supported and valued the intentions behind rough work in progress and welcomed suggestions, which was greatly appreciated.
How did you organise the work of your VFX Producer?
ILP’s production team, orchestrated by the Executive VFX Producer Måns Björklund, VFX Producer Emma Cummins and VFX Production Manager Bradley Jordan worked their magic ensuring the project and the team journey was as smooth as possible. Bradley and I collaborated closely with the skills supervisors for most of the show planning, and we diligently tried to plan, replan accordingly and stick to our defined internal milestones. Our priority was to establish key and hero shots that would define the sequence look and feel. This approach played a crucial role in planning.
What are the sequences made by ILP?
In total ILP worked on 8 sequences.
We designed the majority of shots above water as the protagonists sailed through the Aegean Sea. CG sky, sea, boat extensions, and wake FX were on the menu.
Their journey then took place underwater as they were looking for an artefact hidden in a sunken Roman trireme. The wreck dive demanded complex and intricate CG : environment, trireme wreck, digi doubles with diving equipment, underwater creatures like eels and fishes, and a wide range of underwater FX.
Back on the boat with the treasure, we helped the protagonist escape from the pursuing Nazis by doing CG sky, sea, boat extensions, wake FX, gun shots, smoke, embers, exploding dynamite and melting wax FX simulations.
The adventure continues to the cave of Dionysus, where we extended and enhanced the environment.
Finally as the protagonists ventured deeper within the cave, we added CG creepy crawlies such as centipedes, beetles, tarantulas and cobwebs to a tight passage leading to Archimedes’s sarcophagus dressed with a few more CG critters.
Can you explain in detail about the creation of the boats and the ocean?
Production provided a partial 3D scan of the boats, accompanied with reference photography, we also had 3D scans of the set build of the cabin / deck used on stage. The CG Boats assets were created for set extension (hull, props and bow) and for full CG kind of shots. Most props, rope, railing, diving platform, fenders … etc had to be replicated. Plates were filmed either in a harbour, using a real boat, or in a blue screen stage, on a practical set build section of the deck or cabin (depending on the boat). In both cases the CG sea was expertly crafted by Timo Von Wittken, Light Supervisor; Timo used ILP’s ocean toolset to achieve a realistic and controllable water surface and perfectly mimicked the look referenced from the reference plates. The area around the boat, especially the wake and foam, was additionally simulated and blended within the CG water. For continuity sake, we often had to replace the majority of the real water, as the natural waves in the harbour were relatively flat.
How did you create the digital doubles?
The creation of the digital doubles was a close collaboration between Build, Rig, Animation, Groom and CFX.
We received 3D scans, and generic shared assets from ILM of the stunt divers and the actors. Adrian Tsang and Nicolas Giraud, Build Supervisors, oversaw the creation of the 4 digital doubles and their exhaustive diving equipment. Special attention was given to the asset groom and lookdev to ensure they looked good once underwater (simulated and light absorption constraints).
All characters were wearing similar diving suits so Matteo Di Lena, Rig Lead, created a “diver” template that was used as a base to easily compose different bits and props on all of them. The divers were oxygenated and connected to the boat by breathing tubes. The tubes were procedurally built rigs with different lengths by assembling tileable pieces of geometry and attaching them to a tentacle based curve rig, which gave animation a fair amount of flexibility to achieve some complex underwater motion in certain shots.
Martin Eneroth, Animation Supervisor, and his team of animators keyframed the CG divers. Various underwater diving references and stunt divers footage helped us to set their performance. The breathing tube and the various props were a mix of animation and FX.
Nora Hänsenberger, FX lead, did a fantastic job simulating many parts of the characters hair, vest, props, mask straps, moving props and secondary motion to the breathing tube…
To mimic underwater behaviour and give buoyant, slowed down underwater feel to the divers, forces used to simulate the hair and parts of the diving suits had to be manipulated. Reduced gravity and volume advection with added turbulence were the main tools used to simulate buoyancy and current-like motion. We also generated some tension based attributes to add some subtle winkles when the diving suit was compressing and stretching.
Can you elaborate about the filming of the underwater sequences?
The underwater sequence was filmed using two different approaches : dry-for-wet and in an underwater stage.
For the Dry-for-wet the actors were filmed at high frame rate in a blue screen stage, while wearing a minimalist diving suit. We then had the challenge to matchmoving their actions, replacing and simulating their hair, adding visors to their goggles, breathing cables and making props and features appear as if they were underwater (such as mask straps and other objects). Dry-for-wet was predominantly utilised for closeup, as the actor’s performance played a crucial role in the storytelling.
Another method was to film in a tank of water at Pinewood studio with professional stunt divers. This was for close up and mid kind of shots. Art department built a small section of the wreck but there was a massive amount of enhancement work required. Adding breathing cables and face replacements was often required.
For both approaches we ended up adding simulated FX breathing bubbles, particulate, sediment, volumetric beams from the divers torchlights and obviously integrating them with our CG underwater environment and sea life.
Anything that couldn’t be filmed ended up being full CG.
How did you create the shipwreck and the massive underwater environment?
We collected and gathered an exhaustive amount of photographs and video references of Roman trireme, wrecks and all sorts of underwater elements : seabeds, rock formations, marine plants and algaes, sea life… We began by modelling the entire underwater floor as simplistic proxy geometry, as the geography had to be quite specific. The trireme wreck was modelled plank by plank and splitted in 2 : the prow section was suspended on the first ledge 30m deep, while the stern section, the largest portion of the wreck, rested in the deepest area, 80m deep down. A small section of the wreck built by the art department for the tank shots had to be integrated into our build.
We leveraged ILP’s environment pipeline which enabled us to easily mix and combine procedural approaches with more hand-placed workflows. Stefan Andersson, CG supervisor, ensured technical workflows were streamlined and optimal.
After ensuring that the geography complemented the storytelling, we proceeded to detail the environment and wreck geometry, heavy scattering of our library of underwater assets (including rocks, plants, sargassum, sand, and seagrass) and procedural shaders. Gustav Alexandersson, led the development and look of the underwater and wreck and ensured all scattered assets were able to react to moving objects, such as the divers, eels, and even the 3D camera.
We also had to populate the wreck with a large quantity of props : amphorae, Roman skeleton, armors, shields, weapons, wooden beams, plants… Most parts were animatable using either ptc or rigs, so we could adjust the layout and nail any specific framing or animate any object as required
Finally to add life to the environment fishes were scattered at shot by shot basis. We studied references of moray eels feeding on fishes, which showcased their aggressive and fascinating behaviours, and successfully proposed incorporating some of these interactions into a few shots.
Can you explain in detail about the eels creation and animations?
Niklas Wallén, concept artist, designed the entire eel family, including adults, teenagers, and eelvers, by combining features from the fangtooth moray and giant moray species. We modelled and sculpted the adult plus the baby eel and designed the textures to be as procedural as possible. Stripes, colour and overall detailing was done solely by combining procedural noises, giving us the ability to generate unlimited texture variations.
The eels rigs were built using ILP’s proprietary procedural rigging tools. The underlying rig structure was built using a mocap style pipeline in mind allowing crowd and sim cycles to be mixed and, or interpolated with hero animation. Animation wise we studied references extensively and wrote a versatile tool to ease and manage scenes with up to 50 animated eels. As the animation rigs were hooked up to a motion path like system and we had the ability to easily control many aspects of their swim. Speed, amplitude of their undulations, floppiness of their dorsal fin, their banking when turning … individual eel performance could be easily art directed. Martin and the team of animators truly did a wonderful job.
Eels were animated for shots requiring hero FG / MG performance and for the majority of the shots we diligently mixed animation and simulated Eels.
Indeed, Corbin Mayne, FX Supervisor, created an intelligent particle system to solve the motion path of the eels, using a combination of goal seeking, flocking and proactive collision avoidance. The eels use their size as a hierarchy system, so the smaller eels will flock together more and scatter to avoid the larger ones, who are far more independent and goal-focused. Then their body and facial motion and skinning is procedurally generated using curves and a volumetric awareness of the available space. Finally, a skin simulation is applied on top to add dynamics to the motion. Tension information was also computed and used at render time to introduce dynamic wrinkling based on compression and stretching of their body.
What was the main challenges with the eels?
Managing the sheer number of eels and ensuring readability and continuity of their action were the main challenges with the sequence. Often, a single note addressed in one shot could have repercussions on the surrounding shots.
How does the underwater lighting affects your work?
Johan Gabrielsson, Light Supervisor, did a tremendous job setting up a controllable and accurate underwater lighting template. After a lot of evaluation and testing we decided to replicate the underwater lighting directly in 3D instead of going into full deep renders and achieve the look in comp. In fact all lights created, from the sun and sky to flashlights, behaved like they do underwater. They start off with the full-colour spectrum, then the different colours get absorbed based on the distance they travel until only blue is left. This made it possible for us to get an accurate look based on the depth where we placed the cameras. We validated colours using colour charts and we also implemented artistic overrides for depth and distance from the camera. This workflow was pivotal to the project success and helped us get quicker iterations and more accurate preview. Volumetrics, minutiae, breathing bubbles, eels, fishes were separated and rendered with some deep alpha information for maximum control in comp.
Viktor Andersson, Compositing Supervisor, and his team also did a fantastic job. As the dry-to-wet and water tank source material was filmed in a controlled environment, there was a lot of fine tuning and colour grading required to mimic the colour wavelength absorption relatively to the depth level. Volumetrics and murkiness required a lot of balancing and refinement in Comp. Storytelling and visibility was key so we had to find the best compromise to keep the shots as realistic as possible without damaging the action clarity.
Additional effects such as animated caustics (synchronised with the 3D sunlight volumetrics) and underwater lens aberrations were key to blend the divers into the CG environment seamlessly and realistically.
Which sequence or shot was the most challenging?
The wreck dive sequence presented significant challenges. Many shots required custom layout, extensive animation, and intricate FX; And to complicate the task all disciplines were interdependent.
The FX team lead by Corbin Mayne, FX Supervisor, faced the task of maintaining motion congruity across all simulations, which is crucial underwater as everything is affected. From hair and cloth to plants, bubbles, dust and particulates, all elements were driven by an underlying “water current” simulation based on the character’s motion. Recreating the diver breathing bubbles, in particular, proved to be quite tricky due to their distinct scale-specific behaviour. Large bubbles wobble and bend almost like cloth, and as they break down into smaller bubbles they become increasingly more rigid in their spherical shape.
Is there something specific that gives you some really short nights?
Several of the underwater full CG shots presented significant challenges. The balance of underwater visibility versus photorealism, also choreographing the action for storytelling necessitated making a few tricky compromises along the way.
What is your favourite shot or sequence?
It’s challenging to pick just one, but the wreck dive, particularly the part where the eels’ nest is disturbed, is in my humble opinion a really exciting sequence. It is also a funny callback paying homage of Indi’s deepest fears of snakes.
What is your best memory on this show?
Watching the trailer for the first time! Seeing a few ILP shots with John Williams’ iconic score was a very special moment. Also receiving feedback from Steven Spielberg was a nice surprise.
How long have you worked on this show?
The journey started mid December 2021, and finished toward the end of March 202, so over 15 months in total.
What’s the VFX shots count?
Approximately 250 shots.
What is your next project?
I’m afraid I can’t disclose any information, but it’s going to be a fun one for sure.
A big thanks for your time.
// Indiana Jones and the Dial of Destiny – VFX Breakdown – Important Looking Pirates
© Vincent Frei – The Art of VFX – 2024
