Nigel Denton-Howes told us about Prime Focus’s work on THE EXPENDABLES 3 in 2014. He then joined Framestore and worked on AVENGERS: AGE OF ULTRON. He talks to us today about his work as Overall VFX Supervisor on MOWGLI – LEGEND OF THE JUNGLE.
In 2014, Max Solomon told us about the work of Framestore on GRAVITY. He then works on JUPITER ASCENDING.
How was the collaboration with director Andy Serkis?
Nigel Denton-Howes: Amazing. Andy is a true gentleman, not only as a person just to spend time with but also as a creative collaborator. He has a very clear vision of what he wants to accomplish in his shots in terms of performance, and works really well with our animation team to achieve that, but he also allows a very open collaboration on the details of the shots. For instance, lighting design on all-CG sequences, details of the effects, how we designed the all-CG sets, were all a genuine hand-in-hand collaboration between Framestore and Andy.
Max Solomon: Andy was amazing to work with, always so positive, inclusive and energetic. He’s incredibly passionate about the work, and that’s so infectious. I started work on MOWGLI: LEGEND OF THE JUNGLE in the Spring of 2014 when Framestore was asked to produce a two-minute sequence to test some of the proposed methodologies. Initially we worked with a performance by Benedict Cumberbatch as Bagheera recorded in an ADR session and plate footage of Rohan Chand in costume as Mowgli on a jungle set opposite a performance capture actor. It was an interesting test, but we learned that trying to combine a voice and face performance of a lead actor with the physical performance of a stand-in actor on set really didn’t work – it lacked cohesion and felt unconvincing. So we reshot the sequence, this time with Andy providing both a voice and a physical performance recorded by witness cameras. By this stage Christian Bale had been cast, so we redesigned Bagheera incorporating Christian’s key facial characteristics and the animators reworked the shots only using the witness camera footage of Andy’s performance. To everyone’s delight the result was much more successful and this became the benchmark for the animation workflow on the movie.
How does he used his experience in performance capture on this show?
Nigel Denton-Howes: The process of creating the performances for this show was quite unique. Andy created a situation where the principal cast would play the scenes directly to each other, in character, as a sort of stage play. They would give very physical performances, which of course made the voice acting that much more believable, but also gave our animation team a great kick off point for translating those performances on to the physiology of the animals. Given how different the human vs animal features are the animation itself was keyframe animated, but the performances were very much authored by those great actors and interpreted closely by the animation team. This method of performance capture, where the actors really authored every bit of the performances, was a unique experience for all of us and one we feel was quite successful.
Max Solomon: Andy used a myriad of tools and tricks to give his actors as much context as possible allowing them to always give a more physically complete performance, whether that meant puppeteering a head for eye lines, getting his actors down on the floor and moving around with short arm crutches, blocking a space with rostrums and staging the actors together like a theatre workshop or acting opposite actors in ADR sessions. Rohan was puppeteered by blue-screen men in the Kaa tree sequence, while Andy and Christian got pretty physical in the Baloo/Bagheera fight, which served as great inspiration when blocking the animation. This meant that we always had great acting choices to work with when we started the animation process.
How did you split the work amongst the Framestore offices?
Nigel Denton-Howes: Generally the work was split by type of character, with of course some crossover. The London team dealt with the monkeys, wolves (including Akela, Bhoot, Vihaan and Nisha), Tabaqui and crossover with Bagheera, Shere Khan and Kaa. Montreal focused primarily on Bagheera, Shere Khan, Kaa and entirely on the elephants. For builds, London built the primary animals in their scenes as above, but also built Bagheera, and in Montreal it was similar where they built their primary characters. For Environments, the primary builds were done in London, with layout and lighting for the sequences done in the locations where the work took place.
Max Solomon: The work was split between London and Montreal roughly two thirds to one third, but I didn’t want the work spread randomly. So we kept all of the predominantly wolf, monkey and Baloo scenes in London and Montreal predominantly did the big cat scenes, along with the Kaa and Hathi scenes, and this roughly fit the work split. This also drove the asset builds with Kaa, Hathi and the other elephants and Shere Khan being built in Montreal and the other characters in London. This meant the animators could become really specialised in their characters. I had different teams on each sequence, which meant they developed a wonderful sense of ownership and knowledge of the characters’ nuances and physicality.
Can you tell us more about the previz and postviz work?
Max Solomon: Many of the scenes were prevised by both the Framestore team and other companies like Third Floor and Proof, Andy really bought into the shots and edits he had devised, but we also did more spontaneous previs during the shoot to explore ideas Andy had had for more complicated shots. He also blocked some sequences live on the mocap stage at Imaginarium with a virtual camera and actors in mocap suits. Most scenes went straight into animation blocking at Framestore, but the scenes that turned over later were developed with draw overs by a small team of storyboards artists to help inform the edit.
How did you work with the art department?
Nigel Denton-Howes: During the initial stages of the film we worked very closely with the production art department in order to refine the characters of course, but also in order to do a good job of building out all the plants we would need when moving forward with CG environment builds. The greens department in particular was a great source of information on the species used in the real sets. The idea was to blend the CG and real-world shoots seamlessly, something I think we succeeded at. Later in the process it was decided to open up the film a bit more with additional sets done entirely in CG. For these I worked directly with Framestore’s art department, lead by Martin Macrae, to design the new setpieces which were then built by Framestore’s London team.
Max Solomon: Framestore’s Art Department were involved very early on concepting Bagheera and redesigning him when Christian Bale came on board. They also worked up a great many paintings and digital sculpts for Baloo, exploring his size and bulk and various facial scars and swellings. The Art Department are a fantastic way of exploring ideas and get approval on design issues quickly.
How did you design and create the various characters?
Nigel Denton-Howes: We didn’t design the characters directly – those were handled by the production art department – but we were of course involved in minor modifications to those designs in order the facilitate the requirements of performances Andy wanted to achieve. The characters were built between Framestore’s London and Montreal teams.
Max Solomon: Andy had been developing his characters during pre-production with a variety of different artists but settled on some beautiful concept paintings and character sketches done by Kan Muftic, who was working in the Art Department at Leavesden and these really served as the blueprint in the post-production phase. Andy was very keen for the animals to resemble the actors he had cast, and for them to have the ability to emote strongly, which meant we gave them eyebrows, articulated lips, cheek bones, and more human eyes with smaller irises. Kan did a series of morphs for each character, where over a series of six paintings he transitioned between a photo of the actor and the head of the animal they were portraying. We normally settled somewhere between the fourth and fifth painting, but would often take elements of others. The creatures also had stylised anatomy, being bigger than the real animals, usually with larger chests and front legs. The designs all went through a rigorous validation process where we checked their proportions allowed them to walk, run, sit, lie down etc. The rigs were built with the ability to stretch and squash the bones where necessary, while always checking we hadn’t deviated too far from the original design.
Can you tell us more about their rigging and fur?
Nigel Denton-Howes: The rigging was, of course, extremely complex. Not only were we doing these very nuanced facial performances, but there were large physical performances by most characters as well. The rigging included not only the more conventional controls for moving the limbs and faces, but also direct control over muscle-tensing and jiggle on the body and the face as well.
The fur was interesting as it’s not exactly what you see on normal furry animals. Andy wanted the animals to be very worn down, with patches of fur missing or worn away, and the fur itself to be very greasy with lots of clumps, dirt, grit and detritus like leaves and twigs mixed in. As such, the fur on the animals can be at times very dusty, or very shiny as the grease comes to the surface. The fur is then simulated, of course, not only to match the physical moves of the animals but also to add in wind movement.
Max Solomon: The quadruped rigs were homogenous with identical control sets to optimise build times. This also meant animators could easily switch between different characters, and we could copy animation between similar creatures like the wolves. With the cast of characters being so big (there are 20 talking characters and another 15 supporting animals) it was important to find efficiencies, particularly among the wolves. Face shapes were transferred from the hero characters like Akela to the supporting cast and then tweaked where necessary to add nuance. Similarly, animation cycles and pose libraries were built for each character and stored in Framestore’s animation library. Particularly useful were the libraries of phoneme poses, which meant animators could quickly block lip-sync whilst staying on-model. And because the face rigs were all the same these libraries were quick to build as we could copy them over from hero assets and tweak per character.
Can you explain in detail about the animation challenge and especially the faces?
Max Solomon: The main challenge for the animation team was to faithfully transpose the actor’s performances, honouring every detail and nuance, onto characters of such different anatomies. Of paramount importance was the eyes. I was constantly fascinated by how significant the angle of the head along with the eye-line was in communicating an intention. We relentlessly scrutinised Head Mounted Camera (HMC) footage to recreate every detail of facial micro-motion. Early on we processed the HMC footage with facial capture software, but the animals’ faces were so different from the actors that we found it of very limited use, and the capture data was very dense and difficult to work with, so we decided it was more efficient to let the animators work up the performances more methodically from the ground up.
We usually had two performances to work with. Andy had filmed his principle cast in theatre workshop style sessions and then used stand-in actors during plate photography on set and on location working opposite Rohan. Frequently we incorporated parts of these performances into the final animation as they often worked directly with what Rohan was doing or in context of the set or camera movement.
We also had to constantly consider the various sizes and anatomies of the creatures. Movements had to be adjusted to allow for the creatures’ greater bulk, and human gestures needed to be interpreted onto the quadruped creatures. In the action sequences a performance may have been delivered with the actor static and we had to work that into a believable physical action.
A particular challenge was Kaa the snake. As with the other characters we worked slavishly with the HMC footage of Kate Blanchett for the facial performance, but the body was animated and choreographed in a complex relationship with the camera often using movement and gestures the stand-in actor had done opposite Rohan on the blue screen stage.
Andy was also keen to continue developing the performances in animation, either enhancing or reducing what the actors had done, sometimes adding new ideas or splicing separate takes or performances into one shot.
Although much motion capture data was recorded of the stand-in actors during the shoot, the only footage we found useful was for the monkey sequence where Terry Bannon had trained and led a large ensemble of actors to perform like monkeys. They carried Rohan through the ‘Cold Lairs’ set and having the capture gave us a great head start in blocking the shots. The animators then reworked the performances to add more physicality and stronger poses and also added many more background monkeys animated using monkey reference.
Which character is your favorite?
Nigel Denton-Howes: That’s kind of like asking which child is your favourite! In my heart of hearts I’m very fond of Tabaqui, but of course all the animals have their own personality that you become attached to.
Max Solomon: I’d have to choose between two of the supporting members of the cast. Vihaan has the first line of dialogue several minutes into the film – “How did it get here?” in a big close-up – and we knew that line had to be totally convincing, and I think it is and he looks great: full of life and personality. But my favourite character is Tabaqui the hyena. We had so much fun with Tom Hollander’s wonderful mischievous performance, and I thought the design of the character was wonderful, with his host of flies constantly buzzing round his head.
Can you tell us how you choose the various VFX vendors?
Nigel Denton-Howes: Framestore was the primary vendor, with the London and Montreal teams holding the primary assets and doing the vast majority of the show (just under 1200 shots between the two locations). There was a requirement to do VFX work in South Africa due to the shoot location being granted there, and we chose to award Black Ginger based on their long history of producing exceptional work. Towards the end of the shoot it was clear we needed some additional help on the finale sequence, so we brought Montreal’s Rodeo FX on board to take on some of that work. In order to do some cosmetic fixes we tested a few companies and ended up loving the work done by Vitality, based out of Vancouver.
How did you split the work amongst these vendors?
Nigel Denton-Howes: The Framestore splits were determined by character, with a 30/70 split between the two countries.
Black Ginger was brought on to do a lot of DMP set extensions in the village, as well as quite a few shots where they added bugs and atmos into plate shots. They also helped out with a few of the fire shots where Mowgli runs into the village after the wolf flight sequence.
Rodeo were approached to look at an epic bit of the end battle, starting from where Shere Khan charges Mowgli all the way through to him getting shot by the hunter. They did a great job on that piece and we couldn’t be happier with the work done there, lead by VFX Supervisor François Dumoulin.
Which sequence or shot was the most complicated to create and why?
Max Solomon: It’s between two shots in the monkey sequence. The first is the reveal of the waterfall and the gang of monkeys dragging Mowgli over the ridge. There are over 40 monkeys and a digital Mowgli carefully choreographed without any reference. It was an amazing labour of love for one animator that took about five months and much sweat and tears. The second is the long shot at the end of the monkey sequence which starts as Kaa’s point of view and then develops into following her up and round the throne where Mowgli lies unconscious. It was so complicated, comprising three separate takes with seamless transitions and retimes, over a hundred monkeys plus Shere Khan and Tabaqui and, of course, Kaa herself, who wasn’t long enough to loop twice around the throne so we had to use two rigs and hide the join off camera. It was much more of a team effort but still took about four months for three animators to complete.
What is your favorite shot or sequence?
Max Solomon: There’s a moment after Akela has been shot and as Shere Khan staggers malevolently towards Mowgli he steps over Akela’s body and indifferently knocks past his head. It was a lovely detail the animators in Montreal devised and illustrates the wonderful collaborative atmosphere Andy created on the show encouraging artists to do their best work.
What is your best memory on this show?
Max Solomon: The experience of working with Andy, and how his positivity and enthusiasm for the work filtered down through the whole team both at Framestore and in the production team, despite such a long and complicated production.
What is the VFX shot count?
Nigel Denton-Howes: The total shot count across all vendors was something just under 1500 shots.
What was the size of your on-set team?
Nigel Denton-Howes: Depending on the shoot it was anywhere from five to 15, plus additional participants from vendors depending on the sequence being shot.
Max Solomon: I attended all of the shoots with a number of VFX supervisors depending on the number of stages that were shooting.
What is your next project?
Nigel Denton-Howes: I’m just finishing ALITA: BATTLE ANGEL.
Max Solomon: I just finished working on a short sequence of around 50 shots for ALITA: BATTLE ANGEL. Very different from the work on MOWGLI but a lot of fun! I’m now working on AVENGERS: ENDGAME.
A big thanks for your time.
WANT TO KNOW MORE?
Framestore: Dedicated page about MOWGLI – LEGEND OF THE JUNGLE on Framestore website.
© Vincent Frei – The Art of VFX – 2018
