Frankie Stellato has over 15 years of experience in animation. He joined the Digital Domain team in 2011 and has worked on a number of films including Ender’s Game, Deadpool, Black Widow and Spider-Man: No Way Home.
Ron Miller has worked in visual effects for over 20 years. He worked at Weta FX for over 7 years before joining Digital Domain in 2022.
What was your feeling entering to Ant-Man universe?
Frankie Stellato // This actually wasn’t my first time in the Ant-Man Universe. I worked on a small sequence for Ant-Man and the Wasp where Hank and Janet were in the quantum realm. So overall, I felt fairly confident, and I was extremely excited to work on a character like M.O.D.O.K..
Ron Miller // This was my first time, and I was quite excited to be working with VFX Supervisor Jesse James Chisholm and Director Peyton Reed. When I heard that we were going to be working on M.O.D.O.K., I was even more excited, and honestly a bit terrified at the same time. I mean it’s a 5-foot-tall floating head. No pressure…
How was the collaboration with Director Peyton Reed and the VFX Supervisor Jesse James Chisholm?
Frankie Stellato // It was absolutely fantastic. Both Peyton and Jessie were extremely collaborative, informative, supportive and trusting of our team. We were given a lot of freedom with enhancing M.O.D.O.K.’s body and facial performances, based on the already great and hilarious performances from Corey Stoll (who is also great to work with!).
Ron Miller // Collaboration is a great word to describe it. They were an absolute pleasure to work with and had a lot of faith in what we could accomplish together.
What were their expectations and approach about the visual effects?
Frankie Stellato // At the beginning of production, we had a bit of an issue with M.O.D.O.K. looking a little bit like Humpty Dumpty, so Digital Domain was tasked with helping to design the final look, as well as his overall personality. We also had to figure out how to essentially stretch Corey’s likeness from human sized to a 5-feet-tall and 3-feet-wide face, which was a major challenge in itself.
Like any show, things were done incrementally. Versions of him being angry, sad, happy, fighting etc. were worked on to help mold the M.O.D.O.K. that made it into the final film.
What are the sequences made by Digital Domain?
Frankie Stellato // Digital Domain worked on pretty much all the shots where M.O.D.O.K. is delivering any facial performance, with the exceptions of some shots where his mask was down and he was in battle mode. My personal favorite was his death scene, as it was just so silly and made me laugh even from the very first previs I saw.
Can you elaborate about the design and creation of M.O.D.O.K.?
Frankie Stellato // There were many iterations of M.O.D.O.K. throughout the production. We started with Corey Stoll’s face in what was essentially an early version of the M.O.D.O.K. armor, but that quickly went away, as it didn’t feel like M.O.D.O.K. from the comics. A few versions and iterations after that, we began to kind of blend out the look of Corey, leaning more towards a version where he looked a lot like the M.O.D.O.K. seen in the comics. But we then lost any essence of Corey. We eventually found a balance between the versions and ended with our final design. We also designed M.O.D.O.K. ’s weapons and how they deployed and retracted, which was a lot of fun!
Ron Miller // When researching M.O.D.O.K., it seemed like there were endless versions of the character. There was some back and forth looking at proportional wedges of the face, and also some tests with adding layers of scarring to the character. However, in the end, Peyton and Marvel’s vision of M.O.D.O.K. was to ensure that Corey’s likeness would still be as recognizable as possible. They really steered us away from things being too grotesque.
Additionally, at the beginning of development, M.O.D.O.K. looked more like a guy sitting in a chair rather than a person being a part of the chair. There was a lot of work put into how this fusion between man and machine should look. One day, maybe the different hairstyles we worked on will be released to the public, as it added a whole other level of hilarity to the character. Ultimately, the decision was made to keep him bald, as it seemed too far of a departure from the actor, and audiences struggled to connect the two.
How was his presence simulated on set?
Frankie Stellato // There was a M.O.D.O.K.-sized, gray-faced maquette that someone would pick up and carry around to help with the actor’s eyelines.
This character is basically a giant head with a tiny body. How did you avoid the cartoony aspect?
Frankie Stellato // It was a very fine balance between all departments, really. The largest factor was probably the model. As we adjusted the model to blend between Corey’s proportions and M.O.D.O.K. proportions from the comics, we’d notice a nose that was too big gave off a cartoony look, whereas a nose that was too small gave off an oddly uncanny valley look. Even the spacing between his eyes and nose in relation to how wide his mouth was would very much affect how cartoony or realistic he looked. I think things really clicked once we had that perfect blend of Corey and M.O.D.O.K. in the model, with textures that were also to the correct scale, and lookdev that reacted nicely to the lighting.
Ron Miller // This was a massive challenge, because as soon as you start warping the face, it becomes stylized. Peyton wanted to ensure that the actor’s likeness, even though distorted, would still be leveraged as much as possible. We scrutinized every shot and we would always try to figure out if things were breaking down on the asset and why. Sometimes we had to adjust the animation on the face to compensate, and sometimes we had to go and adjust the lookdev.
Because of the size of the face, the lookdev and textures had to have a lot of work put into them to make sure the asset would hold up to the camera coming within inches. We had scan data from Corey, but when this was transferred to M.O.D.O.K., the resolution was not what it needed to be. We had to up-res all of it and then re-enforce it with a XYZ microstructure to ensure the skin would feel real in all lighting situations.
How did you manage the creation of the skin shaders?
Ron Miller // For M.O.D.O.K., we decided to use V-Ray 5 for the random walk SSS. For a face this size, we wanted to make sure the forms and details would hold up from all distances without doing a lot of comp tricks. Random walk dealt with the curved areas of the face better, so the base forms are a lot clearer, as well as preserving the fidelity of the fine details on the face.
Can you describe the filming process to have the facial animations?
Ron Miller // We had a day where we went to Digital Domain’s mocap stage in LA and did a full body and HMC capture. Peyton would work with Corey running through his lines as if he was on set. This capture drives our machine learning pipeline Masquerade. This gives us the ability to take the motion directly from the face of the actor and drive our digital asset of the ensuring 1:1 likeness.
How did you use Masquerade?
Ron Miller // When using Masquerade, we always develop a range of motion for our machine learning that will cover the majority of the facial performance requirements. This worked great for the Corey asset, but it had to transfer over to M.O.D.O.K. as accurately as possible. The eyes, nose and mouth all match Corey, but due to the proportion changes, we had to develop a magnitude map system to deal with the proportional difference between the two assets. Once that was in place, we could really study the facial motion and all the relationships on the face. For example, the lips’ relationship to the teeth and gums and the iris’ relationship to the eyelids. It all may seem a bit overkill, but things can start feeling off if not paid attention to.
Can you tell us more about the animation work beside his face?
Frankie Stellato // He was a very fun character to work with. Marvel kind of gave us a rough idea of how they felt he should act, and asked us to come up with something to help gauge his performances. We tried versions where he was not very expressive at all with his body. Mainly, keeping his hands down and on his chair, with subtle motions and gestures from the elbow down to the wrist. His legs in this version were essentially dead weight. Then we had versions where we went really far with it and almost overacted. This had the best result in conjunction with the audio and facial performance once we toned it down a bit.
His overall motion would constantly have a bit of a hover cycle to him when giving subtle performances. When he was moving, however, we had to be very careful with making him look light as he was meant to weigh tonnes.
What kind of references did you receive for his flying animations?
Frankie Stellato // We didn’t really receive any reference for his flying animation but knew he had to have a bit of a constant stabilizing hover when talking and had to really showcase his mass when moving faster and slowing down.
The armor is really shiny. How does that affect your work?
Ron Miller // M.O.D.O.K.’s costume required us to develop something both functional and photoreal, from modeling every nut and bolt in the guns to the nightmare stone in his forehead. Peyton and Jesse really wanted M.O.D.O.K.’s costume to have a feeling of opulence to help reflect the character’s personality. It was a bit tricky to balance lighting the face and the costume in the same shots because of those different material qualities.
Which shot or sequence was the most challenging?
Frankie Stellato // In my opinion, one of the hardest shots was when he was smashed down into the ground by Cassie. It started as a P.O.V. shot and I think ultimately ended as an over the shoulder shot. It was mainly difficult due to the performance needing some adjusting due to the camera angle and M.O.D.O.K.’s rotation relative to the camera. It was just a really unflattering angle that we had to adjust animation, lighting and the camera to make work.
Ron Miller // I personally always find the shots where the character is first introduced to be the hardest. There is so much hinging on these moments because of the audience’s reaction to whether they can invest into this character or not. When M.O.D.O.K. was first introduced, we got really close to the character, and the asset had to hold up on all levels.
What is your favorite shot or sequence?
Frankie Stellato // The death sequence, hands down. Specifically, when he coughs up white blood and shoulders his way into being an Avenger. Makes me laugh just thinking about it.
Ron Miller // I love a lot of moments in the prison cell sequence for many different reasons. During development, this is where I could really see everything coming together and all the hard work the artists were putting in was paying off. Plus, there is the moment where Scott Lang tells M.O.D.O.K. he has baby legs. So amazing!
What is your best memory on this show?
Frankie Stellato // The overall lighthearted nature of the entire show. The team is a bunch of fun goofballs (myself HEAVILY included) and it just makes going to work every day so much fun.
Ron Miller // I agree with Frankie. The team on this show was just amazing and such a joy to work with. From the client side through all departments at DD, everyone was so collaborative and just wanting to create something special.
How long have you worked on this show?
Frankie Stellato // I came onto the show May 16, 2022, and finished on January 22, 2023.
Ron Miller // I was involved quite early on for doing a test, so March 2021 to January 2023.
What’s the VFX shots count?
Ron Miller // There were 261 shots total. However, we did more than just M.O.D.O.K. for the movie. We did some Kang helmet transformations, as well as developed the look of the prison cells.
What was the size of your team?
Frankie Stellato // The animation team size was 25.
Ron Miller // The total team size at its peak was between 150-175 people.
What is your next project?
Frankie Stellato // I don’t think I’m allowed to say.
Ron Miller // At the moment, I’m working on some internal development. But the next show that comes in that requires hero facial work, I’ll be on it.
A big thanks for your time.
WANT TO KNOW MORE?
Digital Domain: Dedicated page about Ant-Man and the Wasp: Quantumania on Digital Domain website.
Jesse James Chisholm: Here is my interview of Jesse James Chisholm, Production VFX Supervisor.
© Vincent Frei – The Art of VFX – 2023
