Back in 2019, Olivier Dumont had explained to us the work of visual effects on Ford v Ferrari. He then worked on The New Mutants and Shang-Chi and the Legend of the Ten Rings.
Michael Perdew began his career in visual effects over 12 years ago at Luma Pictures. He has worked on films such as Guardians of the Galaxy, Thor: Ragnarok, Spider-Man: Far From Home and The Tomorrow War.
What is your background?
Mike Perdew // I’ve been with Luma for about 13 years now, working my way up from a coordinator to a VFX producer, focused primarily on feature films. I’ve completed over 60 projects with Luma over the years.
What was your feeling about being back into the Doctor Strange universe?
Olivier Dumont // A Doctor Strange movie is always exciting to work on since you know it will be very intense creatively. For the first movie, when I was still at Method Studios, I was happy to work on the design of the magic mystery tour sequence, where Strange discovers the multiverse—which was a nice gateway to this sequel. For Doctor Strange in the Multiverse of Madness, while the work was supposed to be more grounded, it certainly came with its load of visual craziness, starting with this one-eyed giant octopus chasing America Chavez through the streets of New York, and the battle with Strange and Wong that would follow.
Mike Perdew // It was a dream come true! Ever since the first film was announced I had dreamed that it would live up to the visual craziness of the comics and it didn’t disappoint. I was beyond excited when I discovered that we’d be working on one of my favorite character designs from the comics in the Gargantos battle sequence, and even more overjoyed when we got to take part in the Illuminati Tribunal Room scene that introduced a slew of my favorite characters.
How was the collaboration with Director Sam Raimi and Production VFX Supervisor Janek Sirrs?
Olivier Dumont // The movie is definitely a Sam Raimi movie, and you can really track his style all along, which is great. The focus of the director was to make sure we get the right story and the right action that fits his vision. We had to try different things to see what was the most adapted for the moment, and this is where Janek’s role was so important as he was the gatekeeper of the vision. For instance, he led the charge for the creature and Strange’s cloak behavior. Janek definitely has some of the best eyes in the profession, and his understanding of the storytelling is just outstanding. Janek’s approach was to try to stay as grounded as possible, knowing that it wouldn’t be simple to completely get rid of the cartoonish style of the creature’s design due to that big eye. He also kept a close eye on the animation to find the sweet spot between real and fun, and he was a great help in the general making of the shots. Janek Sirrs and Cyndi Ochs (client side VFX producer) have been amazingly supportive throughout the production of the show to get us to the delivery line.
How did you organize the work with your VFX Producer?
Olivier Dumont // While the simplified role definition is commonly “supervisor takes care of the visuals while the producer organizes it”, the line was definitely more blurred here because I wanted to tap into Michael’s knowledge as a big Doctor Strange fan. Very often, I asked him what he thought about some developments. In the end, this was very much a big team effort that included everybody due to the complexity of the tasks. We really try at Luma to have people from all departments sharing ideas as much as possible to make sure we are getting the best on screen.
How was the work split between the Luma Pictures offices?
Mike Perdew // Luma generally splits the work up 50/50, delegating the right tasks to the right artists. Since production occurred largely over a global lockdown, we had people all over the world. In general, animation occurred evenly across continents, while the LA office handled much of the design and conceptual work, before splitting the shot execution evenly.
What are the sequences in this film made by Luma Pictures?
Olivier Dumont // We worked on the Gargantos chase, part of the Illuminati tribunal room sequence including the death of 617 doctor Strange, the Clea tag at the end, and some extension work when 616 Doctor Strange, Wong, and America bury the future zombie Doctor Strange on the NY rooftop.
How did you work with the art department to design and create the Gargantos?
Olivier Dumont // We received an initial design of Gargantos from Marvel that gave a general direction and traits of the creature. We took it from there to make it three dimensional and add all the additional elements as seen in the movie. We had our own art department involved to gather ideas and visual elements that would help us define the look and texture of the creature’s different parts. Later, the same department would provide some quick visual alterations for discussions with Janek to find the right look.
Can you elaborate about the creation of this creature?
Olivier Dumont // We started by modeling Gargantos with few details, adding some simple shaders and textures. That allowed us to start a conversation with our character supervisor, Mathieu Aerni, about the nature of the different parts of the creature. For instance, defining all the parts that would be either solid like the plates on the top of the creature or semi-soft and fleshy like the underneath of the creature. We would also talk about the colors and the extra elements like the nostrils, the extra eye membranes, and the suction cup claws that weren’t in the original design but eventually gave more personality to the creature. After our rigging team did a first pass at what would be an awesome job of rigging an eight-legged and semi-soft creature, it was given to our animation director, Raphael Pimentel, and his team to start playing with it. This step was very important as it would define the final scale, proportions, and the behavior of Gargantos (animal vs. intelligent creature). Look and animation went through multiple revisions to make sure we ended up with the best look.
Can you tell us more about the creation of his skin and shaders?
Olivier Dumont // The details were added in Zbrush, going as far as detailing all the little bumps that you can see on the creature’s skin. We started by making it very symmetrical and then added the differences to make it less perfect. For instance, all tentacles were getting the same treatment at first for time’s sake, but were all reworked to give the feel of uniqueness for the parts we were seeing in the sequence. The head was also a good example, as we made sure to break the symmetry in this area to get a more organic look.
The shaders and textures went through different color schemes and materials. We started with a more brownish color with some splashes of reds to eventually go more toward the green shades to match the famous look of this type of creature in the comic book.
The look of the material was a compromise to get the creature look wet but still colorful at the same time under an overcast lighting. We used a lot of subsurface, especially for the fleshy looking parts, and it is always a challenge to do so on a giant creature to get the right look without losing the scale.
How did you manage his huge eye and also avoid a cartoonish look?
Olivier Dumont // We knew from the beginning that this would be a challenge. Between the fact this would be the focus in most of the shots and the intrinsic comedic look of the proportions, we put our main attention to that part. Gathering references from humans and animals, the eye like the rest went through a lot of iterations to find the right one with the Marvel team. It eventually was based on a human eye, but adding enough details to it to make it work at such a big scale. Our animation team has also done a great job to help reduce the cartoonish look while bringing the creature to life.
What were the main challenges with Gargantos?
Olivier Dumont // The animation and rigging of the creature was an enormous task, but thanks to the talented artists at Luma, it worked out well. I would say that the biggest challenge was to set up the continuity throughout the chase. Meaning, keeping track of what the creature was doing and the implication on the environment and keeping enough continuity so that you don’t question it. So, instead of a perfect continuity that would eventually force us to replace the practical plates with full CG because of the amount of destruction, we were carefully selecting the elements that would carry along from shot to shot, making sure that it wouldn’t disrupt the sequence. That was done closely with Janek to try to keep the important story points. It was very tedious work as you have to do this while the cut is still changing. We managed this task with Stephen Borneman, our CG supervisor, and with the animation team.
How did you recreate a huge part of New York?
Olivier Dumont // The practical set included about three blocks of two-story set buildings around a main street. We had to extend the set vertically and horizontally. Sometimes, for interaction purposes, we also had to replace the practical buildings. Marvel’s art department gave us a document referring to all the vertical set extensions and this would become our base to model these. We just had to make a few modifications to accommodate some story points but it was already very thorough.
For the horizontal extensions, we started by using a real time engine to speed up the layout process and quickly get some good background visuals to replace the postvis that were in the cut. The layout, once approved, went back to our pipeline to use the highness assets and get the final renders. To help us with all of it, Janek and Cyndi organized a shoot in New York to get plates and pictures. This was completed with matte paintings for the very far background of the streets.
Can you elaborate about the lighting work on Gargantos and New York?
Olivier Dumont // Most of the plates were overcast, and it is usually harder to integrate CG with this type of lighting. We got some shots that were sunny, but this could have become a problem as well with the absence of the building extra stories that would normally cast shadows on the practical set and actors. Fortunately, the sunny shots weren’t that many, thank you London, and they were shot in a way that we could work on them with minimum alteration.
The overcast look pushed us to define the creature look even more as everything could look very flat very quickly. We used every ounce of contrast we could to get the maximum value range on the creature. The DI did a great job at making the sequence pop even more.
The general methodology was this: We started to light the shot using the HDRis taken during the shoot to get an overall matching look to actors and practical set, then we enhanced the creature lighting to make it more interesting when it was looking too flat. The thinking was to imagine what a DP would do if this creature was shot on set.
Can you tell us more about the various FX elements such as magic and destructions?
Olivier Dumont // That’s another part where Janek’s role was important: the creation of the visuals for Strange’s powers. Janek looked at the old Doctor Strange comic books and found visuals for the powers that were different from the first movie. It led us to build “photo real” versions of iconic powers not seen before, like the Fang of Farallah, the Demons of Denak, the Flames of the Faltine, or the Chains of Krakkan. I’ll let you find these in the sequence.
We started with some concepts done by our art department and modeled the approved versions to be passed on to our FX team led by Wayne Hollingsworth. This required a lot of creativity from the FX team as the translation from concept to three dimensional and animated visuals wasn’t an easy task.
We had to plan to destroy a lot of different elements in the creature’s path and mayhem—cars, bus, street props, buildings, and so on. Some were small, others were big, the different materials reacting in different ways added to the complexity of the tasks. As usual, when you have to destroy something, the models need to be prepped for it, so the objects were meticulously chosen to get this extra pass of modeling. Shaders and textures also had to be enhanced to add the scratches and broken windows for cars, or the aftermath of the building destruction for instance.
Doctor Strange and Wong are using portals. Can you tell us more about the preparation of these shots?
Olivier Dumont // These were probably the most straightforward shots as we had a ton of references from the previous movies. That being said, the FX task itself has many different passes that are not necessarily visible, but are used to make it feel very organic. The other tricky part is what you see through the portal and when you have to join the two worlds with a camera move that goes from one to the other. We had to do a lot of tracking and projection to make it work.
How did you create their digital doubles?
Olivier Dumont // All our digital doubles were shared assets from other vendors. We still had to ingest, calibrate the shaders and textures, and recreate the hair as well as the rigging. As much as we can, we are trying to match the approved references sent by the other vendors, usually by asking for their turntable light rigs. We can then compare them in the same environment. You usually have to tweak the look to make it work in your own sequence.
Which shot or sequence was the most challenging?
Olivier Dumont // It wouldn’t be one shot for sure. The Gargantos chase was definitely challenging for all the points above where each shot had its own little challenge.
Is there something specific that gives you some really short nights?
Olivier Dumont // Let’s say the Gargantos sequence finishing stretch wasn’t the most restful moment…
What is your favorite shot or sequence?
Olivier Dumont // The Gargantos sequence was definitely a fun one. While it was a hard one to work on, this was the most rewarding.
What is your best memory on this show?
Olivier Dumont // That wouldn’t necessarily be a memory but more of a nice feeling: when you work with talented people like the whole Luma team that pushed hard until the end, or with the Marvel VFX team that supported us throughout the project.
Mike Perdew // One favorite moment was when Spider-Man: No Way Home came out because so many people watched that movie and saw the Multiverse of Madness trailer at the end of it. Seeing the buzz about that and about Gargantos got us all motivated to finish out the film and make it really great.
How long did you work on this show?
Olivier Dumont // About a year, but I still would have loved more time to push the visuals even more.
What is the VFX shot count?
Mike Perdew // Luma completed 266 shots across multiple sequences.
What was the size of your team?
Mike Perdew // We had a VFX crew of roughly 200 people over the duration of the project.
What is your next project?
Olivier Dumont // Can’t tell for now, but hopefully it will be as fun as this one.
What are the four movies that gave you the passion for cinema?
Mike Perdew // Primer by Shane Carruth, Audition, Velvet Goldmine, and Sam Raimi’s Spider-Man.
A big thanks for your time.
WANT TO KNOW MORE?
Janek Sirrs: Here is my interview of Production VFX Supervisor Janek Sirrs about Doctor Strange in the Multiverse of Madness.
Disney+: You can watch Doctor Strange in the Multiverse of Madness on Disney+.
© Vincent Frei – The Art of VFX – 2022
