In 2021, Dominik Zimmerle explained the work done by the Trixter teams on Shang-Chi and the Legend of the Ten Rings. Now he’s back to talk about his work on The Marvels.
Ernest Dios began his career 20 years ago, working in a number of studios including Automatik VFX, RISE and Trixter. His filmography includes many shows such as Babylon Berlin, Black Panther, Captain Marvel and Shang-Chi and the Legend of the Ten Rings.
How did you and Trixter get involved in this show?
Dominik: As we created Goose the cat for the original Captain Marvel and she is part of the cast of “The Marvels”, we were asked to bring her to life once again. This time, a new cat named “Tango” played Goose, so we needed to redo most aspects of the digital double regarding look and proportions.
Ernest: Besides the cat, they also had a few other sequences for us in mind involving complex environment builds and FX we were already familiar with, like the Jump Point, the Hard Light from Miss Marvel, Carol’s Binary, etc. where we were a natural fit.
How was the collaboration with director Nia DaCosta and VFX Supervisor Tara de Marco and Additional VFX Supervisor Sarah Eim?
Dominik: We only worked with Tara and Sarah. The collaboration was great. They were really interested in getting the best possible output for every shot. This dedication carries you a long way.
Since they were also on WandaVision, Tara and Sarah had a special connection to Monica Rambeau and her powers. It was extremely helpful that they shared their creative process for Wanda Vision and how Monica should develop in the Marvels.
What were their expectations and approach to the visual effects?
Dominik: The goal was realism and understated effects that feel believable, physical, and natural. The team wanted to stay true to the MCU references but without going into a too flashy or magical direction. Even when we did, for example, the Concept Art for some energy tubes that were considered a rather secondary item, the Marvel team thought about what this energy could actually be – so we did some very interesting research regarding plasmas. We really liked this approach to ground things in reality and then maybe pushing it a bit for dramaturgical reasons, instead of going wild from the beginning.
Ernest: For Goose, there was again a strong focus on making her believable, from the anatomical correctness and a credible cat-like behaviour on the performance side. Having all the reference footage shot on-location was perfect to match not only “a cat” but “our cat” with its different nuances, and be sure it will cut flawlessly with the live-action footage of her. When the cat goes crazy and shoots tentacles out of its mouth, it proved to be the cherry on the cake of the challenge.
Was this show different from previous Marvel Movies you worked on?
Dominik: Yes, it was. It was by far the longest show that we ever worked on. It was also a project that started in the late Covid times and ended in the first post-pandemic year. So the whole show felt like a journey – professionally and personally for everyone. We also worked with two supervisors internally. That was a great experience and helped us develop workflows to absorb bigger shows in the future.
How did you both organise the work amongst yourself and with the VFX Producer?
Ernest: The collaboration was very easy and organic, as we have known each other for quite some time and already worked together on two MCU shows – Captain Marvel and Shang-Chi and the Legend of the Ten Rings.
Dominik: We organised our collaboration depending on the demand and subject. We kept each other in the loop at all times so we could cover every aspect of the show. It was also great to see the other’s workflows and creative approaches, and we think the project profited from this. VFX Producer Sara Martínez Ruiz led the production team together with Production Manager Angelique Krawczak.
Ernest: We followed the usual split that is common on pretty much every show, in which we were responsible for the artistic approaches and client communication covering all creative aspects, while Sara led our wonderful production crew and organised the work with James Alexander, the client VFX Producer.
What are the sequences made by Trixter?
Dominik: One of the exciting aspects of the show was that we covered quite an array of different VFX tasks in our sequences and the work was quite “entangled” as it happens at the beginning of the movie. So many aspects were dependent on each other.
Ernest: Our main sequences were the ones happening on the Dead Planet MB418, that included a full CG environment and the development of the Broken Jump Point, Kamala and the cat entangling into the Khans’ living room, Monica’s fight with the Kree in the same location, a nice space dialogue between Monica, Dag and Talia repairing the Spectrometer of the Saber Space Station and Monica’s way to the Residual Jump Point. Besides these main ones we had a few smaller sequences.
Can you explain in detail about the creation of Goose the cat? Was it different from your approach in the original Captain Marvel?
Dominik: We learned a lot from the first cat we did for Captain Marvel and our approach was overall more streamlined.
Ernest: We received a large amount of reference material that formed the foundation of our asset development. Especially the lidar scans and light stage reference photos proved to be extremely useful for building the correct proportions and for matching the fur patterns and anatomical details bit by bit.
Additionally, as we received the Lightstage material in several variants – cross polarised, polarised and unpolarized – this helped a lot with a controlled shader development.
Dominik: We started the cat with a model that was a first educated guess of how the cat would look without fur, using all lidar scans as a reference but translated into a riggable neutral pose. Unfortunately, you can never really know how much of the cat’s volume is the actual body and how much the fur contributes, unless you shave it, which was out of question, of course. So all you can do is guess and look for references (naked cats, x-rays) to find the ratio with the highest probability.
Ernest: We then did a first rig and checked if we could align our default pose model with the lidar scan references and do adjustments based on the outcome. Once we were happy we did a first groom pass and did another few rounds of evaluations to find the most probable balance between skin and fur volumes. Once this was mostly settled, we went into fine details in terms of fur groom patterns, anatomical fine detail refinements around the mouth, eyes, paws and moved towards a final asset. In the meantime our animators trained themselves on cat-like behaviour and tried to match various references with our asset.
Dominik: A big challenge was to have a matching fur color pattern on the cat. Once you use a texture painted on the skin and assign it as a color to the hair, the pattern is offset – depending on the groom, adjacent strands might even displace the original texture in different directions. That required some tool development and good old trial and error.
Ernest: To make sure the asset held up to the expected level, Tara and Sarah picked one sample shot to match at a 1:1 level with our cat. This was the ultimate test and the goal of the whole asset development: our Goose needed to be seamlessly intercut with the real Tango. The shot was simple, which made it very hard. Tango just sat there, looking around, moving the ears, no moving camera or fast movements. Once we achieved this, we confidently moved towards shot production.
Dominik: For the shots, we learned something from the original Captain Marvel: no matter how well you match your asset, a cat can always look a bit different in every sequence – just because of the time difference between the asset data acquisition and the actual shoot. To be ready for this, we focused on establishing shot-based methods to nudge the groom if required and do shot sculpting for anatomical and structural refinements. One aspect that was entirely new to us was the Flerken tentacles. Although we also created Goose for Captain Marvel, ILM added them to the asset. We received the original tentacles to implement into our Goose asset and to use in our shots.
Ernest: The tentacles and the mouth membrane required quite some rigging work so that the animators could art-direct them following Nia’s instructions. Also in our shots Goose swallows and “un-swallows” several things, from characters to armchairs. Having solid interactions between the tentacles and the items were a key to make the interaction believable. We went as far as we could with animation and then gave the result to our Creature FX department to add suction cups, a de-intersection pass and some refinements to the volumes to make them more muscular.
How did you handle Goose’s rigging and animation?
Dominik: The approach for Goose’s rig was inspired by requests coming from the Animators during the original Captain Marvel. That’s one of the great things about doing something twice. Besides giving our animation team all the controls they needed – and wanted – aligning the output of the animation rig and the Ziva rig was one of our team’s big tasks and challenges. This means: we wanted to make sure that the pre-simulation deformation of the animation rig was a very good preview of the post-simulation result. We had a lot of interaction shots, where volume accuracy is paramount, so this was extremely important. From the beginning, it was also clear that we wanted to generally improve the cat compared to the 2018 version. So a lot of time went into creating the Ziva rig with our Creature FX department to get as naturalistic deformation and surface characteristics as possible. There is a reason why some people claim that cats are essentially a fluid.
Ernest: Animation started off from the reference material we received from Marvel side and from tons of reference footage. The animation team learned how cats move, behave in certain situations, hiss, purr, play. “Captain Marvel” veteran Simone Kraus-Townsend was fortunately taking on the Animation Supervision again. We also received beautiful references of Tara’s own cat. She even staged her like in our sequences, which was amazingly useful and fun. As mentioned before, the tentacles were a project on their own for the riggers and animators. We never had anything like this in the house, so the rig was new and the goal was to have utmost control and as much automation as possible. Also the characteristics of the Flerken tentacles are special. They are bursting out of the cat’s mouth to swallow or to release objects. While some of the tentacles are doing the actual contextual performance, the rest are supposed to move around like whips which is part of the unique character of the whole Flerken. Interactions were carefully crafted manually by the animators to match exactly the performance requirements of the client.
Can you tell us more about her simulation aspects?
Dominik: We used Ziva again for our cat’s muscle- and skin simulations. The experiences we gained during Captain Marvel were extremely beneficial here and we think we improved quite a bit.
We aimed to mimic the fluid random movements from a cat’s loose skin as closely as possible.
As we had a lot of shots in which Goose gets lifted, the quality of the interaction was instrumental for a successful shot. One part is the skin deforming under the fingers. We solved this directly in Ziva and refined it in a bespoke sculpting pass in shot context. The other part is how the fur behaves when the fingers dig in. We mostly used Houdini for this aspect with a fully exported groom to get all the fine details of the hand interactions. With our experience and better tools available, the interactions were far more straightforward to achieve and we could do this in more shots within the same time frame.
What were the specific challenges bringing Goose to life?
Ernest: We think cats are amongst the hardest animals to bring to life. They have a very specific behavioural pattern. They seem static, but there is always something moving, as they are always aware of their surroundings: the ears go in one direction, the cat quickly checks another with the eyes, weight shifts, etc. People know them pretty well so any mistakes are easily spotted. And again we did not need only a cat but our cat, so we also needed to be sure the performance in terms of facial deformations would read like the live action character.
Dominik: Another thing was all the interaction with Goose. As cats are so flexible and have such loose skin, there is a substantial shape change happening when the cat is touched or held up – there are compressed skin rolls changing shape, loose skin reacting to the slightest movement and all is in motion all the time in a harmonic composition.
Ernest: Also when you add fur to all this complexity there is a lot of movement happening and we needed to control these changes to a degree as it easily looked erratic. Also fur wise we could not rely completely on working with groom guides but instead we needed to simulate every fur curve to really get technical correct and appealing interactions and dynamics.
Can you tell us more about the creation of the Dead Planet environment?
Dominik: The idea of the MB418, or the Dead Planet, was that it was a part of the former Kree Empire. This world doesn’t have any water or even atmosphere left. The positive side of a Dead Planet is, from a complexity point of view, that you don’t need to deal with plants or animals; the challenging part is that you need to achieve a moon-like look of a planet without an atmosphere that fits the photography and is readable in terms of vastness and feature scale. No atmosphere means no aerial perspective, so there is no real measurement for distance besides the few buildings. But they are alien and really could have any dimension. That’s why we worked intensively with texture gradients and larger scale variance that compressed into the BG to tell depth. To provide the CG departments a detailed guideline for the look, we did extensive concept art in various directions to nail the design ahead of the environment build.
Ernest: The shots on the Dead Planet were all happening in a rather large sandbox. We started off by distributing each shot to its most probable location according to the edit of every specific scene. We then populated the global scene with mountains, large landmarks and buildings in layout and checked how the sequence was working. We then started to design the assets based on the Concepts that Dom mentioned. We did the entire environment build in Clarisse because it is – or was – amazing in dealing with such a huge scene and polygon density, while staying flexible the entire time to adjust everything to new edits and shot count changes with fast layout and animation iterations. We also rendered directly from Clarisse to avoid any unnecessary exports and get faster turnarounds.
How did you approach the shots in space?
Dominik: The shots in space were filmed with the set surrounded by blue screens and the actors wearing their space gear without visors suspended on various rigs. Blue dressed people supported the movement.
Ernest: The main goal was to keep the performance of the actors as seen in the photography while still achieving the animation behaviour that you would expect in outer space. To do that we tracked the body performances of the actors and used the original photography of the faces reprojected on our animated meshes. We needed to provide refraction layers for the visors using these textures as well.
Dominik: For the look, we gathered references from spacewalks and boosted the key light, contrast and bounce light in Katana and Nuke to get this strong, unfiltered sunlight, no-atmosphere feeling.
How did you work with the other studios? Were there many aspects that were shared with other vendors?
Dominik: Due to the complexity of the show and the fact that a lot is depending on various other aspects, there was a lot of sharing required with other vendors. Also, several vendors co-developed some assets and adjusted them for their specific needs.
Ernest: Also since there were some suit changes for the main heroes sometimes there was some branching out from the main asset. Some of the assets produced by various vendors and shared between us were Captain Marvel, Kamala, Monica, the Jump Points (Intact and broken), the Kree weapons, the Entanglement, the Hard Light variants, etc., etc.
Dominik: As sharing assets is not unusual, some established guidelines exist amongst the studios, and it works smoothly – most of the time. Also, everybody we worked with was usually very accommodating and provided help wherever needed if something required fixing. And of course, we try to do the same.
Can you tell us more about the broken Jump Point and the Residual Jump Point that you developed at Trixter?
Ernest: Both assets are important story aspects and actually part of the story setup for the whole movie.
We developed both assets from scratch, taking inspiration from the intact Jump Point that was seen in many MCU movies. Luckily we had already developed the JP in Guardians of the Galaxy 2 so we were very familiar with it. The Residual JP is basically a not-entirely closing JP. Glitchy, undefined, dangerous and mysterious. We built it around a hexagon structure and took inspiration from broken TVs and light shows to create the undefined randomly accentuated patterns.
Dominik: The broken Jump Points are kind of “illegally” created – without giving away spoilers here. They are a wild and furious variant of the regular Jump Points. The development goal was to use the language of the regular Jump Point but make it twisted, broken, wild and untamed. The hexagon leaves are bent irregularly; there are power surges everywhere and no consistency in any aspect. We used the setup of the Residual Jump Point that Ernest mentioned for all the glitching and merged it with a re-developed distorted version of the original Jump Point.
Which sequence or shot was the most challenging?
Dominik: Every sequence has challenges, but everything related to the cat and the Dead Planet was generally hard. But also very rewarding.
Ernest: The cat sequence was hard just because of what we needed to do: creating a 1:1 version of a reference creature. It got even harder when we needed to do all the interaction with Kamala and achieve a cat-like performance with tentacles shooting out the mouth.
Dominik: The Dead Planet was a challenge because it was a vast environment that needed to merge with the foreground plate seamlessly and because of all the scale and distance issues inherent to a no-atmosphere planet.
Is there something specific that gives you some really short nights?
Ernest: Nothing in particular, but everything together. The show excelled in the amount of different things to do: Dead Planet, cat, space walks, Khan living room fight, Corridors in Dar Benn’s ship, etc. Keeping all of this together and coming up with solutions is often quite a challenge.
What is your favourite shot or sequence?
Dominik: The sequence in the Khan’s living room where Kamala arrives with Tango and the Kree as stomach passengers was challenging but still incredibly entertaining to work on.
Ernest: Agree. Everybody screams, the performances of the actors are just hilarious and we had a cat with tentacles. Good stuff.
What is your best memory on this show?
Dominik: As always, the team spirit was unique. Some people were with us on the show for over two years; others returned to it several times. But everybody was enthusiastic to make the best pictures possible until the very last moment.
How long have you both worked on this show?
Ernest: A bit over two years.
How does such a long project differ from normal ones?
Dominik: It is fascinating how the perception of the show changes over time, how you see things in new contexts and get new ideas from changing edits.
Ernest: Also you see a lot more work from other vendors. Which is very inspiring.
What’s the VFX shots count?
Dominik: We have 223 Final shots in the movie, but there are a lot more we touched.
What was the size of your team?
Ernest: Throughout the show close to 350 people worked with us.
What is your next project?
Dominik: We heard about some exciting things coming our way, but the next project will be holidays.
A big thanks for your time.
WANT TO KNOW MORE?
Trixter: Dedicated page about The Marvels on Trixter website.
Tara DeMarco: Here is my interview of Production VFX Supervisor Tara DeMarco.
© Vincent Frei – The Art of VFX – 2023
