In 2014, Eric Saindon explained the work of Weta Digital on THE HOBBIT: THE DESOLATION OF SMAUG. He then took care of the effects of THE HOBBIT: THE BATTLE OF THE FIVE ARMIES, PETE’S DRAGON and A GHOST STORY.
Mike Cozens told us about his work on THE HOBBIT: THE BATTLE OF THE FIVE ARMIES in 2015. He then worked on MAZE RUNNER: THE SCORCH TRIALS.
ERIC SAINDON – VFX SUPERVISOR
How did you get involved on this show?
Back in 2005 I talked to Lightstorm Entertainment about working with them on a project. Jim was working on BATTLE ANGEL and AVATAR and we all know which way he went. Over 10 years later Joe Letteri came to me as I was finishing PETE’S DRAGON and said that BATTLE ANGEL had come back around and asked if I was interested. I had since read the first few books of the manga and said absolutely.
How was the collaboration with director Robert Rodriguez and Producer James Cameron?
Both Jim and Robert are visionary directors. They both push the medium to be the best it can be to tell a great story. They also set a very high bar which allows us to really refine our craft.
How did you organize the work with your VFX Producer?
It is really the other way around with this question. I had two really good VFX producers helping on the shoot, Kevin Sherwood and Rene Sekula and they organized me by always knowing where I needed to be and what was on the next day or next week. While filming, you tend to get in a daily mindset and only worry about what’s happening in the moment. Kevin and Rene were always working a week ahead making sure the cameras were all setup for the next day’s shoot, that all the crew were in the right places at the correct times and that all other departments were getting all the info they needed from VFX. The producers are as vital as the VFX supervisors on any live action shoot.
How did you split the tasks with the other Weta Digital VFX Supervisors?
We had 5 teams at Weta working on different sequences. We named them all after motorballers.
Team Stinger led by Mike Perry and Anders Langlands worked on motorball and the Zapan fight.
Team Mace led by Mark Gee worked on the ambush alley scene and most of the shots in Ido’s house.
Team Killer led by Nick Epstein worked on the Underworld and Kansas City Bar.
Team Wingman led by Charlie Tait did the flash back scenes.
Team Gangsta led by Simon Jung and Philip Leonhardt worked on the crashed warship and most of the city streets scenes.
How did you upgrade the performance capture techniques?
The performance capture is always being upgraded with new hardware and better cameras. The big change we made on this film was adding another camera to Rosa’s helmet to allow us to capture depth information for her facial performance. With this extra amount of information, we were able to get proper lip depth and pull and wrinkle information that we have always just made assumptions about in the past.
Can you tell us more about your work during the shooting?
During the shoot our main goal onset is to be as invisible as possible but still gather all the information and performance necessary to bring our characters to the screen. We always tried to hide the tech from Robert so he never had to worry about the capture interfering with the shoot or overwhelming the set with lots of crew behind computer screens. Along with the motion data, we always capture lighting information, 3D scans of all the sets and all the camera information for us to reference later when recreating everything digitally.
Can you tell us more about the skin work?
For the level of detail we needed for Alita’s skin, we started using a new technique. Using deep learning, we laid out pores on Alita’s skin rather than cloning patches. Adding this with actual imperfections in Alita’s face like scars and wrinkles enabled us to place the camera very close to her skin.
We discover a lot of various cyborgs with different shaders. Can you tell us more about the creation of these various materials?
Using real world reference for metals, plastics and fibres, we created a library of material types. We also created an interchangeable package of arm, leg and body parts that could be pieced together in many different ways. Using new geo and materials we were able to create a large variety of cyborgs.
Can you explain in detail about the creation of her eyes and hairs?
Alita’s hair is actually grown as individual hairs on her head. We used digital tools like brushes and scissors to groom all her different hair styles. Once groomed, we solve the individual hairs on her head on a shot by shot basis.
Alita’s eyes were built using a simulation of fibres to represent the dilator muscle responsible for the dilation of the pupil. By simulating the action of this, we were able to get very high-resolution geometry for the iris. The end result was 8.5 million polygons per eye compared to Gollum who had 50,000 polygons total for his entire character.
How did you manage the lighting challenge?
I think the lighting on the movie was quite straight forward. Cinematographer Bill Pope did a great job with the live action and we were able to go off his cues to get our characters to fit into the plates quite nicely. We always had a lighting double in shots dressed in the same wardrobe that Alita would wear so we knew what Bill’s intent was.
What was the real size of the various sets?
The biggest set was on the back lot of Rodriguez’s Trouble Maker Studios and represented four blocks of Iron City fully dressed and full of extras.
How did you populated the wide shots of Iron City?
We used our Massive crowd software to populate the city with extras and cyborgs.
Can you tell us how you choose the various VFX vendors?
Weta Digital did all the Alita character work for all shots across the film. DNEG did the street motorball and the Kansas City bar scenes. Framestore did the background scenes for the motorball try outs as well as the entire pro motorball scene.
Can you tell us more about your collaboration with their VFX supervisors?
We set up a very strong pipeline to allow us and the other vendors the ability to pass assets back and forth. As we updated animation and cameras it was easy to pass data to the other vendors .
The vendors are all around the world. How did you proceed to follow their work?
LEI’s Richard Hollander was the person who was making sure all the elements fit together. We received new updates to the edit every few days so we could see how our assets and other vendor’s assets were sitting in the plates.
Which sequence or shot was the most challenging to create?
The hardest sequences were the one on one monuments between Alita and another character. Alita and Hugo on the bridge was full of interaction and touching that required huge amounts of detail and work to feel like they were together. Another challenging scene was when Alita was eating the orange with Ido. The amount of detail in Alita’s face and body was higher fidelity then anything we have ever done before.
Is there something specific that gives you some really short nights?
The main thing that gave me short nights was Jon Landau does not sleep. His brain is always going and always thinking of ways to improve the movie. Jon tended to call at all hours of the day with new things to make the project better.
What is your favorite shot or sequence?
My favourite shot in the movie is when Alita bites the orange for the first time. When Rosa did it onset, she bit through the orange peel and her whole body tightened. It was a great moment when we finally got Alita to the same state and level of detail.
What is your best memory on this show?
My best memory of the show is seeing the first sequence we finished for an event in Barcelona. It was Alita waking for the first time and seeing her body and the new world. Watching this with the Rosa reference for the first time was a remarkable point in the film.
How long have you worked on this show?
We started filming this project in Austin about 3 years ago.
What’s the VFX shots count?
The total VFX shot count was 2,100 shots.
What was the size of your set team?
We had both an onset team of 5 crew which helped gather all the scans, camera data and lighting information. We also had a performance capture team of 10 with various tasks to help collect all the performance data from the actors.
What is your next project?
I have started on 2 projects at this stage. First is a smaller project I can’t reveal at this time and the second is a little James Cameron project he has going.
MIKE COZENS – ANIMATION SUPERVISOR
Can you tell us more about the previs and postvis process?
Solid previs allows directors to work hands-on with the scene’s construction and provides clarity on early development of story. Weta Digital worked with Robert to create previs for the big action sequences on the film. We started with Robert’s storyboards and script pages. Robert’s boards would be intercut with frames from Kishiro’s art adding energy and a specific flavour to the beats while grounding the work in the original manga. We would sit with Robert and he would describe what was important for shot design, action and narrative. From there we animated rough vignettes of the action, always looking to balance Robert and Jim’s narrative with the energy from Kishiro’s work. Our camera coverage of the beats is always based in real world camera work; we find that when a camera is doing something it cannot do in real life, it pulls the viewer out of the moment. Once we had rough shots, we pitched editorial blocking for Robert to work with. We also provided alternate camera coverage of the beats so there were cutting options once Robert was in editorial. All of our previs was lit and rendered using our in-house Gazebo software. Lighting is an important part of the visualization and we can silhouette and frame characters with lighting or play with the storytelling by revealing characters with shadow and light.
Can you explain in detail about the creation of Alita and especially her face?
Creating photo-real digital faces is probably the most important piece of work we do at Weta. This starts with scanning the actor moving through a variety of facial expressions and then mapping the individual muscle shapes that combine to form each expression onto a digital “puppet” that animators can work with. Up until Alita, most of the characters we have done tend to have more topography which will help indicate what the muscles are doing under the skin (think of wrinkles that define how the face is moving). Alita is a young woman with young skin whose facial performance is defined by how the light plays across the smooth surface of her face. We did not have obvious features to cue the audience to what is happening on the face. This meant that we had to dive deeper into our understanding of what was happening underneath the skin in order to catch the correct motion happening on the surface. Rosa gives a complex and layered performance that needed to be translated in an accurate and detailed way.
Rosa’s performance of Alita carries the film and there is a lot of dialogue and actors will hit dialogue with a variety of ranges but often it’s happening in a very subtle range around the mouth. We invested heavily in understanding and then building all of the structure around the mouth (from the inside out) in order to catch all the subtle detail as it was happening on Rosa. Again, this all played with light and shadow, but humans have an innate understanding of what is correct on a face, so we had to be physically accurate down to the minutiae. Alita also has Rosa’s eyes and mouth (a digital version of them) though they are larger and smaller respectively. Matching the mouth to the actor is something we also did for Neytiri/Zoe on Avatar and this unifies the digital character performance with the Actor.
How did you handle her rigging and animation?
Alita has many different forms during the film. The two main forms are the “Cyber Doll Body” and the “Berserker Body”. The Doll Body is a clockwork construction Dr. Ido created for his deceased daughter. This is a fully rigged internal mechanical structure with a semi-transparent shell and we see the under structure at various points in the film, often with light passing through the shell. Having a properly rigged internal mechanism is another level of detail that solidifies the believability of the character into the plate. The Berserker Body is her URM form and is closer to our typical muscle rig. Both forms include a rigged the manubrium and the sternum to let Alita breathe as breathing was such an integral part of Rosa’s performance. All rigs went through a heavy development process to create mechanically and physically accurate behaviour which was critical for Jim. We had to be ahead of this as we developed her character.
Did you received specific indications and references for the animation?
Rosa is the beating heart at the centre of the Alita character. Her performance is what drives Alita from the opening sequence to the final act. However, there were specific scenes and shots where we needed to take over the action as a key-frame performance. When this happens, we first look for other Rosa reference or stunt reference from Garrett’s team. Depending on what we are trying to enhance, animator’s will act out pieces of action to work through the physics in order to solve a specific problem. Robert is very collaborative and we could show him rough reference and talk through direction at an early stage.
How did you work with the stunt teams for the fights sequences?
In the manga, Alita uses the cyborg martial art of Panzer Kunst which is a zero gravity fight style where a smaller opponent can take down a very large opponent. We had to develop what this would mean on the screen. We worked in tandem with the stunt team developing concepts for fight choreography depending on the situation. We combined many different fight styles to create Panzer Kunst and had to heighten what was physically happening in any one moment while balancing what is physically possible. No matter what is happening at any point in the film, the character posing needs to be strong and clear. Once we had the concept worked out with Robert, we continued to push and augment physical posing to create explosive yet clear action.
Beside Alita which other character was the most complicated to create and why?
From an animation perspective there are a bunch of complex characters in the film and each one of them had their own challenges. Grewishka, who is played by Jackie Earl Haley, takes on multiple forms during the film but he is always a giant. Jackie is 5 ft 5 inches tall. Grew is over 10 feet tall and weighs as much as large truck. Grew’s facial design is also different from Jackie but as we developed the character’s face, we pushed more of Jackie into it: the position of the jaw, shape of the lips, the angle and shape of the brows, etc. Jackie delivers his lines with a tight upper lip and often out the side of his mouth. As we folded more of the Jackie design and mannerisms into the character, Jackie’s performance shone through. Performance-wise his physicality had to be augmented to bring the weight into this character. Weta works to bring a real-world believability to our characters so getting the mechanics and the broad weight back into this giant beast was really important to us and grounded the character into the shot.
Which sequence or shot was the most challenging to create?
Great question! There were many challenges in all shapes and sizes. Alita spends a lot of time eating during the film and I am always surprised at how your face moves when you are working on your lunch. As you chew your lips distort and deform around your jaw, teeth, gums, facial muscles, fat pads and the food in your mouth. Our facial puppet is built from the inside out and includes all the internal structure. Having this really allowed us to push the face around while retaining the correct facial structure. Overall the biggest and most interesting challenge was to create a believable, memorable digital character that the audience would get swept up by from start to finish. We used every tool in the box to get us there and I am really impressed by the team that helped bring her to life.
A big thanks for your time.
WANT TO KNOW MORE?
Weta Digital: Dedicated page about ALITA: BATTLE ANGEL on Weta Digital website.
© Vincent Frei – The Art of VFX – 2019
It was so interesting to read. I saw alita in IMAX 3d last week and it was amazing. I have learned new things about VFX and motion capture from this article.
Thanks, great insights!
Referring to the material, I would gladly hear an attempt to explain (the transfer of fiction to reality) the principle of Alita’s body. If she ate and breathed, was there something like the lungs and some digestive system? Looking at an effective looking body image, apart from the heart, you can see only mechanical elements that allow simulating the normal movement of a person. Like an advanced analog clock. Did Alita feel the touch and other stimuli? Was it only the berserker’s body that made it possible? In addition, the “skin” on the Alita core is something like the Terminator’s orgasmic artificial skin? Or rather something like gum? During the fight in the Underworld you can see that her face is sweating. Similarly with hair, it’s just a wig or some form of hair that can quickly grow to the desired length.
It is a pity that in the movie itself he did not attempt to explain such threads. At the same time emphasizing that, for example: Ido had to work hard to put together Alita. Alita skin, however, differs from Zapan’s skin or other forms (Nysianna, etc.). In the new Robocop, this technology appendix shows how it works.
It’s nice if something like this is created, because I would like to read about it.