Last year, Raymond Chen explained the work of DNEG on ALITA: BATTLE ANGEL. Today he talks to us about the Disney+ film, TOGO.
How did you and DNEG get involved in this show?
As part of the bidding process, a small team at DNEG had done a short test of a CGI wolf in a wintry landscape, to show the facility’s ability to do photo-realistic furry animals. The time frame was short and there were still issues to work out, but I believe that the quality of the test gave the production confidence that our CG animals would work for the film, and DNEG was awarded the project.
How was the collaboration with director Ericson Core?
I took over VFX supervisor duties from Michael Owens, who was the production side VFX supervisor in pre-production. I told Ericson that though I was not involved in a lot of the earlier meetings and decisions, I would do what I could to help him through principal photography and post. I was glad that I was able to offer solutions to a number of the problems that came up during the shoot, including when to use the actual sled dogs and when to leave them out for CG versions, or how to get the dogs to seem as if they were interacting with a CG puppy that would be added in later, for example. During post-production, while Ericson definitely had a specific vision of how he wanted the film to look, he also emphasized that he wanted a creative partner, and was receptive to a lot of the ideas we brought to him, and to our suggestions on how to improve the work.
What were his expectations and approach on the visual effects?
This project was initially visualized as being done with a small alpine unit. Even when the size of the production grew during the planning stage, there was still an impulse to keep things smaller and more limited. Because of this, we didn’t utilize much green screen, and many of the shots had to be shot with the idea that separation would need to rely on very heavy roto work. In terms of the VFX expectations, they were pretty straightforward. The work needed to look photorealistic, consistent, and since Ericson is a cinematographer, there was a lot of attention to composition and lighting.
How did you split the work amongst the DNEG offices?
We split the work between 4 different DNEG locations and 3 different DFX supervisors: Russell Bowen in Vancouver, Lee Sullivan in Montreal, and Karim Sahai overseeing Mumbai and Chennai. For Comp Supervision, we had Rich Grande in Vancouver, Benoit Legros in Montreal and Radost Ridlen in Mumbai. We originally tried to group the work to keep similar types of difficult FX work or hero animation in one location, and dividing by sequences, but there was actually a fair amount of overlap and sharing between the different sites was unavoidable. This required tighter coordination between the different offices than we have had on some previous shows, and more frequent conversations and communication among the supervisors. We adjusted the split of work throughout the schedule to try and utiltize teams efficiently.
The environments are stunning. Where were the various parts of the movie filmed?
Principal photography was in multiple locations around Southern and Central Alberta:
CL Ranch in Calgary, Fortress Mountain Resort in Kananaskis, Abraham Lake, Stoney Nakoda First Nation, Spray Lakes, were some of the locations used.
How did you enhance those environments?
Much of the environment work was adding or enhancing mountains. In some locations, such as the area of Seppala’s home, even though there were some mountains visible in the distance, they didn’t convey the immense scale and rugged nature of the Alaskan ranges. Environment lead Bruno Leveque used elevation data and photography, including some of his personal work from previous visits to the same locations, to create 3D ranges around the Seppala compound. We went through several iterations to adjust the heights and distances of the various ranges to ensure that all the various angles shot at this location would show the epic scale that Ericson wanted. Similarly, a large portion of the footage shot up on mountains in the Kananaskis range needed to be replaced, in order to get rid of trees and place the action at a higher alpine elevation above the tree line. We actually didn’t have enough snowfall in many exterior locations during our shoot, so snow clean-up and enhancement was extensive on many of the exterior shots.
Can you elaborate on the massive blizzard?
The main dilemma we found with creating a blizzard is that it can easily become obscuring and visually uninteresting if played at the heavy level that it would be at for a real-life storm. Ericson wanted the film to be about the epic and the intimate, where the vastness of the landscape was always present to counter the smallness of the people and dogs. To that end, even as we designed various looks and feels for the different stages of the storm, it was critical to have some visibility. We used many different elements for the storm, particle sims at sizes to replicate larger flakes, smaller icy dust, and other various snow types, volumetric passes of mist or fine particulate, and 2d elements. This method was used for most of the exterior snow shots. For the heavier sections of the storm, we needed to use lidar of the environment, matchmove geometry of dogs and roto-ed cards to create the proper depth and build-up of mist/atmosphere/snow. CG supervisor Douglas Tancredi, and FX supervisors Lucas Janin and Jeremy Dineen led the effort for the sequences with the heaviest snow interaction.
Can you tell us more about the recreation of Seattle and the village?
The Seattle shot was filmed at Heritage Park Historical Village in Calgary, where we were able to use many of their historic buildings for the immediate foreground. The mid-ground and deeper background were filled in with a matte painting of Seattle, based on photography of Seattle in 1925.
A massive sequence is taking place on a frozen bay. How did you create this environment?
This sequence was shot on Abraham Lake in Alberta. The location was selected because of the quality of the ice. Since the lake is a man-made reservoir, the ice was clear and blue, there were interesting bubble formations created by methane from decaying trees under the surface, and the surface is typically snow-free and smooth because of prevailing winds through the mountain passes.
Unfortunately, weather didn’t cooperate, and we only had a couple of days with usable ice before it got snow covered. During the couple of days when it was clear, we took hundreds of reference photos which we used to create a CGI version of the clear ice. For many of the plates of Seppala and the dogs running across the ice of the lake, even though it was filmed on actual ice, we ended up replacing the crusty or snow covered surface with the more idealized CG ice. As the action was meant to take place in the middle of the Norton Sound, miles from land, we needed to remove the mountains and any hint of land from the background. Reflections of the dogs and the sled were rendered into the ice from matchmoved geometry or cards projected with the original footage.
How did you work with the SFX and stunts teams for this sequence?
In preproduction it was decided that the footage of the dogs slipping on ice as broken chunks were rotating into the water would be attempted practically. We had constructed a sloped bank of snow at our Fortress Mountain location to try and get the angle needed to show danger and peril, but the dogs didn’t want to run the path that our sequence choreography needed, and always chose the straightest path to the flattest section. So it ended up that most of the action beats, especially with the sloped ice chunks were done as full CGI shots, with CG dogs, sled and Seppala in addition to the CG ice and water. We did use squibs buried in snow for footage of the first crossing of Norton Sound, in order to get the dogs to turn and look at what was to become ice cracks forming. These squib bursts were then enhanced by ice particle simulations.
During that sequence, the ice is cracking. Did you use procedural tools for this aspect?
The choreography of the ice breaking up in the Norton Sound was a combination of hero layout work and procedural methods. The blocking of the whole sequence was done as rough post-viz done by Proof, who were brought into production’s offices in Santa Monica at the beginning of the post schedule to rough in the sequence. DNEG’s layout team, lead by supervisor Digger Jensen, then fleshed out the pieces of ice and progression of the break-up. The ice was created though a combination of hero-modeling and sculpting for select foreground pieces and a procedural system for the majority. FX lead Nicholas Papworth and Environment Supervisor Daniel Rhein developed the system of taking the layout geometry and using additional procedural operations to add both internal geometry (such as fissures, cracks and bubbles) and edge and surface detail. The shots focused on active cracking were done as multiple Houdini simulations, for crack formation, ice shattering, and water/mist release from the breaking.
Can you tell us more about the FX work on the ice and water?
For the first crossing of Norton Sound, the first cracks appearing on the ice were pretty straightforward, as there wasn’t any water revealed and the surface was not meant to be moving, just the initial cracks appearing. On the return crossing with the serum, however, the complexity was greatly increased, as the ice was meant to be broken into individual pieces, which would sometimes break even further, moving on top of a stormy ocean surface. In order to simplify the process, we blocked out the movement of the ice pieces in our initial choreography pass, constrained their movement to a generic ocean swell, before switching out to hero geometry and more accurate water simulations. Further sims for foam, smaller ice chunks, and slush within the water were done as secondary passes. FX lead Menno Dijkstra did a lot of work for the water setups, as well as many of the secondary passes. Some of the water/ice work was not fully simulated, for compositional reasons. As Ericson wanted to deliberately art direct the placement and timing of the major ice bursts and water splashes, these were blocked out as cards to begin with to establish timing and size, and then replaced with particle or mesh caches generated by the effects team or 2D elements within the comp.
How did you create the CG dogs?
The CG version of Togo and of the various dog teams were created as replicas for the sled dogs that were used to film the non-VFX plates. The team dogs were from Snowy Owl Sled Dog Tours, and were working sled dogs. We used Clear Angle to do scans of all the dogs, which was a difficult process, as they didn’t like the multiple flashing lights and snapping shutters within the photobooth. The processed mesh however was only good for general size and proportions, and for non-fluffy areas like the head and legs. Fluffy fur doesn’t process well, so we approximated the body underneath the fur by measuring the thickness of the coats at different points on the body. Using this, our build team created anatomically correct skeleton, muscle, fascia and skin models for the creature department. There was a lot of back and forth in the next stages as grooming was introduced. To create hero versions of Togo (both as a puppy and as a 12 year old), we used photos and video references and tweaked the lookdev/groom until the side by side comparisons were as close to identical as we could get it.
Can you tell us more about their rigging and animation?
DNEG designed a new quadruped rig to handle the complex performance requirements of the dogs. As we had so many close-up shots of CG dogs galloping, we needed the rig to have a number of features to convey the most realism possible.
A new front leg locking system gave a more natural feeling to the scapula taking the weight when the torso lands on it during each footfall. This locking mechanism allowed the animator to trigger tension in the muscles and joints without the need to hand key every single frame as the weight passes over a leg. An adjusted spine setup let the animator decide whether they wanted the hips or torso to drive the motion. This was essential to achieve realism in the highly dynamic performances of the dogs.
What was the main challenge with their animation?
The project had two major challenges for animation. The first was how to get a CG dog to emote realistically in close up facial hero shots, intercut with live action shots of the same dog. Arna Diego, Animation Lead in charge of dog facial animation, studied video reference footage of our canine actors to understand husky faces, behavioural patterns and individual personality traits. DNEG developed a new FACS system for the dog’s facial performances, lead by Lucas Cuenca. Arna and Lucas broke down all of the muscles in a dog’s face, built comprehensive style guides to illustrate what muscles fire depending on the expression, and worked closely with our Lead Rigger, Mischa Kolbe, to provide the animators with a fast intuitive rig with a great overlay control system.
The second major challenge was completing the full CG shots in the Norton Sound sequence, where all 11 sled dogs have to react to a tilting ice piece while galloping across it. How does a 4 legged animal, driving forward quickly along a slippery surface, get affected by the ground suddenly tilting? Adding to the complexity, all the dogs were bound together by a rope harness system, and a heavy sled dragged the rear of the team downwards as it lost control. The mechanics and physics of the animation was based on interpretation, as we didn’t have any actual video footage of a dog sliding on solid ice at an angle. Much of the reference we found was of dogs slipping on flat ice, a completely different style of falling. We gathered clips that had interesting hip and leg slipping dynamics and built them into the performances, being sure to keep every dog slipping and falling in unique and non-repetitive ways.
How did you create and animate their fur?
For the creation portion of the fur, our grooming tools went through a lot of R&D to allow for the very high density of husky fur. The density of fur on a husky is higher than anything we have ever built and required extensive research and development to improve accuracy as well as efficiency. A major hurdle was memory and speed as we had 11 of these dogs to render per team. Our Groom Lead Laurent Maynard along with Lookdev artist Jan Schubert worked tirelessly to achieve every property the fur on the dogs displayed and painted texture and guide maps to an intricate level of detail. For the animation of the fur, we used a number of processes to try and replicate the movement of actual fur. Our CFX team took the base animation and did a muscle/fat pass in Ziva, a skin pass in Maya, and finally the actual fur simulation in Houdini. This again was no easy feat as the dogs also wear harnesses, which press down on the fur, and are all interconnected to the ropes pulling a Sled, which also needed to be simulated.
Which sequence or shot was the most challenging?
The Norton Sound crossing with the ice-breaking was definitely the most challenging sequence, both from a technical and an aesthetic standpoint. One issue, like with the animation of dogs on sloped ice, was that it was difficult to find relevant reference for the look. We combined elements from footage found of river ice jams/breaks, icebreaker ships crossing arctic ice, and plain old stormy ocean waves (both open water and shoreline), but it was more subject to interpretation than something with a concrete well-understood look. Another issue was that while the dogs and sled were supposed to be on unstable ice, our plates were shot on the perfectly level static ice of Abraham Lake. Without the freedom in animating the dogs that we had with the full CG shots, we had to do a lot of work to introduce similar movement into shots based off of the lake plates. Some of the methods we used were separating the camera track of the dogs from the world space, so it would look like the piece they were on was rolling or pitching on the waves, and adding additional rotation into the overall shot, as if the film camera were also on an unstable ice piece.
What is your favourite shot or sequence?
There are a handful of shots of the puppy Togo being put onto the gangline next to Ilsa, who is supposed to be a fierce dog. In the script, Togo is supposed to snarl even more fiercely when Ilsa growls at him, and establish his dominance. While we were shooting the plates, Ericson decided to have Togo lick Ilsa, showing that he wasn’t intimidated, but instead winning her over with his friendly, happy puppy demeanour. The practical snarl rig wasn’t working very well, and the dog playing Ilsa, chosen for being docile enough to endure the snarl rig, certainly did not look fierce. The solution we proposed to Ericson was to use a full CG head replacement for Ilsa instead of using the plate and just painting out the snarl rig. This created a great opportunity for DNEG to utilize the latest developments for facial animation in these very close-up shots. Ilsa’s nuanced transition from snarling and fierce to resigned to enduring Togo’s licking was a challenging exercise for our rigging/animation/creature FX teams. It was rewarding to see the animated performance carry the story effectively and maintain the narrative of the sequence better than the original plates.
How long did you work on this show?
I started on the project soon after principal photography commenced in October of 2018, and was on for a year afterwards.
What is the VFX shot count?
DNEG completed 778 VFX shots for the film.
What was the size of your team?
We had 872 people over the course of the project, in four different DNEG locations.
A big thanks for your time.
WANT TO KNOW MORE?
DNEG: Dedicated page about TOGO on DNEG website.
© Vincent Frei – The Art of VFX – 2020
