Julian Foddy started his career in visual effects at DNEG in 2006. He then joined ILM in 2015. He has worked on a number of films such as The Dark Knight Rises, Transformers: The Last Knight, Spider-Man: Far from Home and The Mandalorian.
Florian Witzel has been working at ILM for over 12 years. He has worked on films such as Rango, Pacific Rim, Avengers: Infinity War and Terminator: Dark Fate.
How did you and ILM get involved on this show?
Julian Foddy // Myself and Janek Sirrs (Production VFX Supe) had kept in contact after our collaboration on Spider Man: Far From Home. Sometime around mid summer 2019 Janek asked if we’d be interested in getting involved in Doctor Strange’s return to the big screen and I jumped at the chance. ILM started out on the project with some concept artwork and FX tests in late 2019. There are some aspects from those tests that remain in the movie despite the script being very different back then.
Florian Witzel // Working on the first Doctor Strange I was really hoping to get a chance to work on the second one as well. I personally love the type of creative work where you have to come up with a lot of new ideas and develop new looks. At first it didn’t look like there was a chance to work on it since it was a London-exclusive ILM show. But luckily there was so much work that in 2021 ILM SF was asked to help out with a few shots and our workload increased steadily from there.
What was your feeling about being back in the Doctor Strange universe?
Julian Foddy // Excited! The first Doctor Strange is one of my favourite MCU films and the effects work is stunning, so I was incredibly keen to pick up where that film left off.
Florian Witzel // I was ready, getting my fractal cook book out and again applying some quantum physics to those NYC buildings. The work evolved a bit differently this time around and I was super excited to be part of the movie again.
How was the collaboration with Director Sam Raimi and Production VFX Supervisor Janek Sirrs?
Julian Foddy // Sam gave very clear briefs in the turnover sessions we were lucky enough to attend (virtually). Janek is a great client to collaborate with, as he’s extremely trusting so we were given creative freedom to suggest ideas and develop looks.
Florian Witzel // Janek is an amazing VFX Supervisor to be working for. I’m a big fan of his work and it was exciting to be working with him on this movie. He really understands every minute detail of the work and knows what it means if he asks for something. It was immensely helpful to be working with such an experienced VFX Supervisor on the client side.
What were their expectations and approach about the visual effects?
Julian Foddy // As always the expectation of quality and photorealism is there, as with any Marvel movie, but approach-wise we needed to be flexible and strategic to allow us to deliver a 300 shot sequence like the Illuminati battle with creative changes and plate turnovers coming quite late into the process, after additional photography.
Florian Witzel // When ILM SF came on later in the show, it was mainly about volume and quality of work. And being able to deliver on time with no problems. There was so much work and the Marvel team was so busy that the client needed trusted partners to get the complex work done on time without any fuss.
How did you organize the work with your VFX Producer?
Julian Foddy // I worked hand in hand with Danielle Legovich, my VFX producer, throughout the project. Danielle was great, particularly in keeping me in touch with the ILM team while I was seconded to Marvel during the shoot. One key strategy which I think worked well was to make sure that while we worked up looks and effects on ‘hero’ shots, we periodically rolled out the work to all the other available shots in the sequence, which helped the filmmakers visualise how things were coming together as a cut.
Florian Witzel // Dan Cortez and I were constantly evaluating what kind of work and how much more of it we can fit in with the limited resources and team we had. Our main goal was to deliver the best possible quality on time without over stretching any of our valued artists and engineers. It was quite a puzzle and Dan Cortez was amazing in planning and collaborating to put it all together.
How was the work split between the ILM offices?
Julian Foddy // ILM London started on the project in earnest in late 2020, with build commencing for the Illuminati headquarters digital environment and digi-doubles, as scan data was received from the shoot. The work carried out by the London team encompassed pretty much the entire section of the movie in the Illuminati HQ, (Strange and America in the bio-monitoring room, the march to the tribunal room, Strange vs Mordo, Wanda’s arrival and takedown of the sub-ultron sentries, and the crescendo – Wanda’s gruesome battle against the Illuminati members in the huge atrium space, and the subsequent chase through the underground tunnels). Later in the project, ILM London also took on adding ‘other universe’ touches such as foliage and signage to New York’s streets (along with CG pizza balls!), and bringing ‘Zombie Strange’ back to life.
Florian Witzel // ILM SF came on towards the end of the show to take on some 911 work which Marvel was trying to get done. Our workload grew from there but did not overlap with London. We decided to run the show almost completely independently. We shared some assets and looks but since our workload was so different each studio ran very independently.
Can you elaborate about the creation of the really cool Illuminati sequence?
Julian Foddy // We started out with extensive previs for the whole sequence, and originally it was planned to shoot the ‘Illuminati Atrium Fight’ sequence in the actual British Museum. However, there would have been so much SFX and rigging equipment in there pretty much everything would have been replaced in post, so the decision was made to go with a fully CG environment and shoot on a green screen stage at Longcross studios. We used an Apple iPad with an augmented reality-version of the atrium during the shoot to visualise the walls of the museum and ensure camera positions were correct for continuity. Due to covid restrictions, not all Illuminati cast members were able to join the London shoot, meaning a lot of shots featured either lighting or stunt doubles, requiring digital replacement of either the face or full body.
In the gruesome moment where Wanda dispatches Black Bolt and Reed Richards in quick succession, some shots were a combination of multiple plates as John Krasinski’s elements were also shot separately. It’s worth pointing out that in this beat, both Black Bolt and Reed Richards (Mister Fantastic) are wearing 100% CG costumes developed by ILM, even in extreme close ups. The FX simulation work in these character deaths was also a fun challenge: For Black Bolt we wanted the ‘sonic blast’ inside his own head to feel physically plausible, so we looked at slow motion reference of objects shattering, and human faces in wind tunnels or skydiving. The complex rippling of the skin etc would have been very hard to simulate at normal frame rate so we stretched the whole shot out, simmed at 120fps using cowl, skin, and skull geometry all interacting with each other, then time compressed the result. There’s something in the sub frame motion blur that really helps sell this effect, along with the very cool ‘squelch’ sound effect brought by Skywalker Sound!
The complex shredding of Reed Richards into ‘spaghetti’ was also a real challenge, an area that really took some work was to make it feel like the geometry was being torn apart rather than emitting arbitrary ‘sausages’. The trick here was to transfer UVs from the body mesh onto the FX geometry so areas of the costume can be seen tearing away.
Of course, after this beat, things get really crazy- Wanda sees off Captain Carter and heads into a showdown with Captain Marvel. Again, a huge challenge for all departments as this beat is almost 100% CG apart from facial close ups. Just the choreography of the ‘cyclone’, and planning where FX debris would land was a huge challenge if we wanted to maintain continuity. The complexities of rendering 2 separate lots of magic / powers casting flickering interactive light and shadow onto fully CG characters and environment , and atmos, proved a challenge for the FX, Lighting , Generalist and Comp teams, but the end results look pretty spectacular.
How did you create the digital doubles?
Julian Foddy // In some cases ILM built the entire character + costume + hair, in others we ingested a body / face model and built sequence specific costumes and skin textures etc. The Wanda digi-double had four costume variants to cover her shots from completely clean to covered in robot oil and dirt.
Can you tell us more about the digital face work?
Julian Foddy // We ended up doing quite a lot of face replacement shots, for which the original plan had been to use 2D patches or CG digi double face, however having shot extensive training material with the key cast, which I was lucky enough to ‘direct’ as part of my on set role, we decided to leverage AI techniques into the face replacement mix. We came up with a technique using the training footage and corresponding renders of the digi-double doing a facial workout in various lighting conditions. This allowed us to train the real actors face onto the CG version of themselves, which just added another layer of realism and ‘believability’. Most of our hero digi-double and fully CG shots also got a boost of photorealism and performance plausibity from having a ‘faceswap’ pass added to augment the digi doubles performance. There’s also some more traditional face replacement work in the Strange / Mordo fight-both 2D and 3D. We managed to use a 2D technique for a couple of Mordo face patches, and in the punch-up in the trench, an almost full frame CG Strange face replacement which I’d defy anyone to spot – the reason it’s so successful is that we actually retained the stunt performers neck and mouth, so there’s a lot of visible tendon and jaw tension that helps sell the digital disguise above it.
How did you create the various FX work?
Julian Foddy // In some cases, we had great ‘jumping off’ points for the look we needed – whilst we’re in another universe, Wanda’s magic and Captain Marvel’s powers needed to basically mimic what we’d seen in other movies, but of course each scenario always brings new challenges. The destruction setup for the atrium was incredibly complex – we have two lots of powers / magic and all the associated volumetric and lighting components, and also lots of geometry ‘ingredients’ – floor tiles that get ripped up and bring under-floor concrete screed with them. Marvel gets blasted through the atrium wall needing a physically realistic setup incorporating inner wall construction materials, steel joists etc. We complete the beat with a huge plasma explosion that shatters the entire ceiling raining hundreds of thousands of shards of glass into the atrium. This required very careful choreography across the shots to get the correct storytelling result, not to mention some serious render time to ensure clean, noise free refractions and highlights on the glass granules. When we head down into the tunnels we encounter new FX challenges – there are some new magic effects Strange deploys that we pretty much concepted ‘in shot’, and a rather spectacular water simulation when the ceiling collapses. Koen Hofmeester (FX lead) and the whole FX team did a superb job.
What were the main challenges with the Illuminati sequence?
Julian Foddy // I described the Illuminati deaths etc earlier, but another creative and visual storytelling challenge was the clash of powers between Wanda and Marvel, culminating in the huge spherical blast that shattered the atrium roof. With fast-moving action and quick cuts, we had to see visually that Marvel starts out by blasting Wanda back, and has the upper hand, then Wanda’s chaos magic envelops the Marvel photobeam, ultimately concentrating the energy into a singularity which then explodes. It basically came down to ensuring a change over time in the prominent colour, so we start out yellow / orange and finish almost totally red. I spent a lot of time squinting at the screen to blur my vision while watching the edit to ensure the ‘flow’ was correct!
How did you capture the data for the environments?
Julian Foddy // The keen-eyed will of course notice that, despite being in Central Park, the Illuminati HQ is based on the British Museum (at least on the inside!). The original plan was to shoot the sequence on location in the museum atrium but practicality soon got in the way – the amount of SFX and Stunt rig equipment would have been impossible to shoot around, so the sequence
was shot on a predominantly green stage at Longcross Studios, with just a small set build for the columns and entrance Wanda walks through. We were however, able to gain unfettered access to the British Museum (which was closed due to covid), for scanning and photography. We ended up with a highly-detailed LiDAR scan of the whole space and a huge data set of texture / panorama photography under day and night lighting conditions. This allowed us to build a high detail, like-for-like model of the atrium in CG. There are some minor design changes in our version of course, the large opening to the bio-monitoring room for example, and the various statues placed around the atrium, including the ‘warrior princess’ statue that ultimately crushes Maria Rambeau’s Captain Marvel. We took the unusual approach in modelling to build the atrium as per real construction techniques – each stone block is a separate object, as is each floor tile. Inside the walls there are steel joists, cinder block inner walls, plasterboard etc – this meant that in moments of impact or destruction, there was not only plenty of complexity but also extreme realism and plausibility to what we see.
The New York exterior was actually all shot in the UK – at what must have been a huge cost, a two block section of Greenwich Village was built on the backlot at Longcross. This can be seen in the opening sequence of the film when Strange fights ‘Gargantos’ (the cyclopic tentacle monster). In our ‘alternate universe walk-and-talk’ sequence we were tasked with redressing and augmenting the amazing NYC set to look even more ‘other worldly’. Firstly, we had to extended buildings vertically as the set was only 2 stories high, so that gave us some scope to inject futuristic architecture, etc., but the main way to achieve the brief was the abundance of CG foliage that we added – every flat surface is covered in ivy or creeping plants, often with unusual hues not familiar on earth 616. This plant life also behaves correctly: bigger branches droop under gravity, and obey the rules of phototropism ( plants try to grow toward the sun). There are also lots of subtle visual clues to an alternate universe – signage, road markings, car number plates, traffic lights, etc., – all are not what we’re used to in the real world.
Can you elaborate about the design and creation of Benedict’s Third Eye?
Florian Witzel // In the very beginning there were a lot of different ideas and concepts of what the eye could be. It wasn’t decided yet how much of an independent character or what the relation to Doctor Strange would be. There were multiple ideas all the way from evil monster looks to animal-like shapes to a perfect human eye. Eventually the client settled on a human-like look which was actually the hardest version for us. Not only is it hard to create a perfect human eye and have it right next to his real eyes, but we also needed to figure out how it would sit naturally and anatomically believable on his forehead. Questions like is it a left, right or symmetrical eye, does it have eyelashes, where is the eye brow (which is very important to convey emotion) and how could the eye socket be positioned in his skull. Also the Third Eye can open and close at will, so we needed to figure out how it can do that on his forehead. At the end of the day, the eye needed to convey Doctor Strange’s emotions in a believable way. Any irregularity would send a different message. It was a very artistic and detailed process. Part of our dailies were called “eyelies” where we would only talk about human eyes. It took all the people involved and their careful attention to miniscule details to integrate the Third Eye believably into Doctor Strange’s face.
What kind of references and influences did you receive for the Third Eye?
Florian Witzel // At first we received a variety of concepts from Marvel. Different studies of animal eyes, evil monster eyes, as well as modified human-looking eyes. On our end, our Model Supervisor Bruce Holcomb made a collection of all kinds of “Third Eyes” being created in previous movies. It was a great collection of Cyclops, Monsters, and devine eyes from movie history. David Bocquillon also worked on various concept art of how an eye on Doctor Strange’s forehead could be shaped and convey emotions. All the studies and references really helped to develop a clear idea and also showed us what didn’t work and what we were trying to stay away from. Having done that research helped us communicate much better with Marvel and Janek Sirrs about where we are going and helped us stay focused getting the shots done without going in too many different directions.
Can you tell us more about Zombie Strange?
Julian Foddy // Zombie Strange is, at least in the ILM shots, pretty much all ‘in camera’ prosthetics and make-up on Mr Cumberbatch. We gave him a digital cheek / mouth interior with hanging strips of rotting flesh, and a CG hand to allow us to erode a few fingers where the flesh has fallen away! The shots where he comes back to life were great fun to put together – we timed CG lightning to match on set flashes for both timing and direction, and of course added a fully CG New York BG. Its subtle, but there’s also a zombie-esque, jerky, broken up feel to the sling ring portal he conjures, which echoes his physical exertions.
Which shot or sequence was the most challenging?
Julian Foddy // For the London team, undoubtedly the biggest shot, and challenge, was the large holo-display view of Wanda arriving and killing off the sub-Ultron sentries. We see the action through a total of 17 sub-ultron POV cameras as well the wall mounted CCTV feed. Normally a fully CG shot only needs to look good through one camera, but here our action had to be watertight from every conceivable angle, same for FX simulations, smoke, etc. The rendering and pipeline complexities of making 17 shots into one were a huge organisational challenge, with credit to Tom Baskaya and John Seru, the comp and generalist leads for the shot.
What is your favorite shot or sequence?
Julian Foddy // I think the climatic ‘cyclone’ at the end of Wanda versus Marvel might be my favourite section!
Florian Witzel // I’m trying not to be biased, but the Illuminati Sequence was definitely one of my favorites. Also following along all the work which went into its creation by ILM London and seeing it on the big screen was really amazing.
What is your best memory on this show?
Julian Foddy // I got to direct a splinter unit with a bridge closure and a cast of extras one night in central London, so that’s quite a special memory!
Florian Witzel // Talking with the team about Eyes for hours in dailies. It was just cool to be laser focused on one small section which had a big impact.
How long have you worked on this show?
Julian Foddy // For me personally, around 2.5 years, as I was involved in the early tests in 2019, then joined the shoot as splinter unit vFX supervisor in early 2021, and oversaw the post at ILM until April 2022.
Florian Witzel // ILM SF came in to help out during the last 6 month of the show.
What’s the VFX shots count?
Julian Foddy // Just over 400 shots across both sites.
What was the size of your team?
Julian Foddy // Around 130 in London, along with a smaller team in SF.
A big thanks for your time.
WANT TO KNOW MORE?
ILM: Dedicated page about Doctor Strange in the Multiverse of Madness on ILM website.
Janek Sirrs: Here is my interview of Production VFX Supervisor Janek Sirrs about Doctor Strange in the Multiverse of Madness.
Disney+: You can watch Doctor Strange in the Multiverse of Madness on Disney+.
© Vincent Frei – The Art of VFX – 2022
