Steve Emerson worked at Rainmaker Entertainment Inc and IMAX before joining LAIKA in 2008 as 2D supervisor to work on CORALINE. He then worked on PARANORMAN and THE BOXTROLLS.
What is your background?
I started working in post production for television and film immediately after graduating from college. I created graphics for television at The Post Group in Hollywood, and then transitioned into compositing and visual effects after they brought in their first Flame in 1995. I loved computers and I loved movies. So as soon as I saw there was the potential to build a career out of a fusion of the two, I did whatever I could to learn the skills and start comping. After the Post Group I bounced around several boutique VFX shops and post production facilities in Los Angeles and worked on many films and television shows. Then I found LAIKA in 2008. I was very fortunate to be given the opportunity to contribute to CORALINE. Stop motion animation, the artistry of LAIKA’s films and Travis Knight’s vision for the studio hooked me from day one.
How was this new collaboration with director Travis Knight?
Travis is an incredible director. Having built this studio, he understands every nook and cranny of the process. He’s a great communicator and had a very clear vision for KUBO. The hardest part of being a VFX Supervisor is getting inside a director’s head. You have to understand the types of things that they are sensitive to and the characteristics of particular visuals that make them excited about the work. There was never any guesswork with Travis. His notes were always very specific and clear.
What was his approach and expectation about the VFX?
It was very important to Travis to make sure that every shot had a unified sensibility. He knew that because of the scope of the film that we’d be using many filmmaking techniques in order to realize his vision. But everything, regardless of technique, needed to feel physical and that it belonged in KUBO’s world. The environments, characters and effects were all heavily stylized, but it was critical that the blend of techniques was seamless and that the shots appeared to have been captured at a single moment in time.
Can you describe one of your day during the preproduction, on-set and then during the post?
One thing that is very special about LAIKA is that the VFX team is very involved in the entire process. As soon as designs and materials are dropping in pre-production, we’re looped in for feedback and informed about how things will be realized when shot practically (in-camera). It’s an extremely collaborative process from the very start. – Then, once sequences are locked, we’ll sit down as a group with all of the department heads and key creatives and begin breaking down the film shot-by-shot using animatics (animated storyboards). The push is always to get as much in-camera as possible using physical elements. The talent here at the studio is truly remarkable in terms of what they are able to achieve in-camera using creative problem-solving. But when – for whatever reason – stages get blocked, our job as a visual effects department is to step in with a solution and help realize the shot.
During production we work very closely with the camera team to make sure we have adequate exposures and reference images. The thing about stop motion animation is we have a huge advantage in that we are able to stop time. We capture performances frame-by-frame and are able to shoot multiple exposures while animating. As an example — if we need to tuck something behind a puppet — rather than shooting it on a greenscreen — the animator will pose the puppet in hero lighting without the green. Then, after that exposure, the animator will slip in the greenscreen for a separate exposure at the same moment in time. This enables us to get the matte we’ll need without any greenscreen contamination in the environment.
Most of the post production work is done out-of-house. At this point VFX is providing support to the vendor and/or the director if a change is suddenly requested. Also, we’ll oftentimes be asked to provide articulate mattes for the color grade.
How did you used previs for the show?
Because of the increased scope and complexity of our films, our need to inform puppets, animation, sets, camera and VFX using previz has grown significantly. We’ll use previz to identify how big sets need to be, where a set extension may begin, how far a puppet may travel through space or how a camera move might translate from storyboards to the puppet world. – One instance of effective use of previz happened during the creation of the Hall of Bones Skeleton. — Early on it was decided that we’d be building it at 100% puppet scale and capture the monster in-camera. This was a daunting effort as the puppet, when realized, would be 18 feet tall. Range of motion was a big concern and there were several ideas put forth as to how to get it to move. In the end, he was placed on a hexapod from the waist up. This was decided after previz did a pass and was able to convince animation and ultimately Travis that the setup would provide enough range of motion to get the best possible performance.
The movies opens with an impressive sequence on a ocean. Can you explain in details about the ocean and waves creation?
During KUBO pre-production, I was shown the animatics (animated storyboards) for the opening sequence. We had never done water at that scale before at the studio. In previous films, we’ve had rain, splashes, distant rivers and waterfalls… but this was going to be an entirely new challenge. It was a very exciting opportunity. I couldn’t wait to get started.
David Horsley joined our team while we were still in production on THE BOXTROLLS. He had led the water efforts on THE LIFE OF PI and we were really lucky to get him. He fit in perfectly with the team and was able to guide us toward establishing a solid workflow for creating photo real water very quickly. Then it all became about how we make the water feel like it belongs in KUBO’s handcrafted universe.
Nelson Lowry, KUBO’s production designer, provided us with some beautiful artwork. Then, as with all VFX for LAIKA films, we launched the animation and rigging team on providing practical examples of what a water system and giant waves would look like if it were done entirely in-camera using physical materials.
The tests were incredible. Oliver Jones, our rigging supervisor, is a truly unique talent when it comes to translating nature with physical animation. The tests were done in-camera using materials such as garbage bags and shower curtains attached to amazing, intricate contraptions.
At that point the task at hand became delivering the look of the artwork, with the character of the physical tests and blending the result with ocean simulations in order to drive authentic water motion into the heavily stylized environment.
While the simulations were done using Houdini, the big wave textures and detail were hand-painted in Mari. We then used flow vector maps for the illusion of movement and a procedural displacement to tie it in with the look of the choppy water.
The entire development process took about 6 months, but we kept learning along the way. After about a year we were really in a groove.
KUBO explores many beautiful locations. Which one was the most complicated to created and why?
There’s a helicopter shot in the film. It takes place early on in the movie, as KUBO is leaving his cave to journey down into the village. Because the puppet animation takes so long (animators typically deliver about four seconds a week), oftentimes animation and shot elements are captured months apart. The helicopter shot included practical puppet animation, a puppet-scale cave, a miniature mountain, and additional rock elements shot practically at different scales as well as CG water and extensions. The animation happened early on, but because of the enormity of the camera move, we decided to wait until we had enough stage space in order to shoot the additional miniatures. They ended up dropping late in the production, probably a year after the puppets were animated. And even though we had waited for the extra stage space in order to get the move, we still fell short and had to pick up the tail of the shot with a virtual camera. – Because of the myriad of techniques, passage of time and the fact that it was a critical shot that needed to be perfect, it was certainly one of the most complex.
Can you tell us more about the skies creation?
Typically we try to get a jump on skies as early as possible in the production. In the case of KUBO, there were many instances of in-camera lighting transitions and cloud movement in the set photography that needed to tie in with the skies. – The development of our sky systems is done in close collaboration with the art department. They’ll provide visual effects with extremely specific artwork and materials. Clouds are never just clouds, but rather clouds created from some unique practical material, like gauze fabric or tulle netting. – Joseph Beckley, our matte painter, spearheaded the sky development with Nelson Lowry, the film’s production designer. – The work was done primarily in VUE and Photoshop and then comped in Nuke.
KUBO tells stories with animated papers. How did you manage their animations and transformations?
Almost all of the origami paper animations were handled in-camera by the stage animators. There were a few that came our way – one was when KUBO creates a flock of birds from paper and a few of them become mosquitos. We built the paper birds and mosquitoes by doing photo real recreations of practical versions that were provided to us by the puppet department. The transition mimicked a look that had been established by the stage animators – quick, clean and artful shapes. Nick Craven, our lead animator, worked closely with stage animator Kevin Parry to get the look right. After that, we iterated with our director Travis Knight until it matched the look and feel of the stage animation.
Did you used procedural tools for specific actions such as KUBO wings?
The process of building KUBO’s wings began with the puppet team providing us with physical, origami birds that we did photo real recreations of in post. We were then given very specific artwork and direction regarding layout and performance of the assembly and the separation of the wing effect. Rick Sevy, our CG Supervisor, spearheaded the wings by developing custom procedural tools using Python and MEL.
Can you explain in details about the creation of the various FX such as the snow storm, fire and water?
The majority of the FX work in KUBO was created using Houdini under the direct supervision of our FX lead David Horsley. If a shot contains elements that are too small to rig and get in-camera (such as a snow system made up of fine particulates) the production will look to VFX to work closely with the art department and animation to develop an approach that fits the aesthetic of the show. In the case of the snowstorm the puppets were animated on snowy surfaces with stage animators displacing the ground snow one frame at a time. The snowstorm was primarily composed of two elements: a particle-based flurry and a denser ground snow. David and his team had developed some beautiful solutions when realizing the smoke demon monsters. We were able to key off of the smoke demon characteristics with the ground snow and ultimately deliver something that felt both interesting, on-style and brought even more uniformity to the show’s FX work.
As far as fire goes, we really only had two sequences to contend with – one inside the whale and another inside a cave. In both cases they were fairly simple campfires. We’d done fire work on all of LAIKA’s previous films and have learned a lot in terms of what feels right in these worlds. In KUBO’s case it was about creating fire that felt natural in terms of performance, but held up to the show’s style. As is typical, we received very specific designs and, under the guidance of CG Supervisor Rick Sevy, used Maya to develop a solution that was both simple and elegant.
Beautiful FX appears when KUBO or his mother are using the guitar. Can you tell us more about its creation and animation?
KUBO’s world is inspired by the visual style of Japanese woodblock artist Kiyoshi Saito. His work is typically characterized by, among other things, an economy of visual information and a soft color palette accented by bright colors. With the film’s magic, it was really about taking creating something that was elegant and powerful, but it also needed to draw from those key characteristics. We also wanted to make sure that KUBO’s magic was distinctive from his mother’s and showed a progression over the course of the film as he became more and more powerful. This was largely accomplished through color palette and by ramping up certain style elements as we progressed through the story.
Which references and indications did you received for this effect?
For the magic we started out with some very specific artwork and designs. Our task is always to bring the production art to life as literally as possible and then iterate through collaboration. During development of the FX, Nelson Lowry, the production designer, would work very closely with David Horsley and the FX team. If something didn’t quite feeling right, he’d oftentimes disappear back into the art department and then re-emerge with physical materials or additional art to help get us back on track.
The action sequences are amazing especially for a stop motion show. Did you have the created full CG puppet for specific shots or action?
The fight sequences were animated in-camera with practical puppets. There are no CG dupes in the film. The animators were able to use a lot of in-camera blur to aid with strobing artifacts caused by the fast motion. They’d use pieces of clear plastic or multiple arms to smooth the movement from frame-to-frame. If what they accomplished in-camera still strobed or called attention to itself, we’d sweeten the in camera-efforts or add spot motion blur in post.
The skeleton is a massive puppet. What was the main challenge with him?
Moving it. Animating it. The skeleton puppet was huge. 18 feet tall when fully assembled. It’s the largest fully articulate stop motion puppet that has ever been created. – The rigging and motion control teams kicked around many ideas during the puppet build on how to bring it to life. They considered using an industrial robot but, in the end, mounted it on a hexapod. If you’ve ever climbed into a ride film, you’ve likely seen one mounted underneath the rider’s compartment. We’d already started using one for KUBO’s boat, so the stage techs decided to give one a go for the skeleton as well. Once they proved that the hexapod could deliver adequate motion, they got to work figuring out how to animate the skeleton’s enormous arms. They were so huge that they couldn’t support themselves or hold a pose. The stage techs ended up created a nest of overhead cabling for support wires that would drop down onto the set. Needless to say, this created many challenges with wires and cabling crossing over forward surfaces of animated objects. Fortunately we have a team of incredibly talented digital paint artists that were able to rebuild those surfaces in post.
At a moment KUBO is building an origami boat. Can you explain in details about it?
There are two moments in the film where KUBO builds the origami boat. The first instance is on the shore of the Long Lake and happens off-camera. The second follows the scene where Monkey battles a sister aboard the boat in the midst of a raging storm at sea. In that case, we needed to see it happen. — One thing that is interesting about how we execute debris performance at LAIKA is that despite the fact that we know that we’ll be introducing computer generated elements into a shot, we still make sure that practical, hand animated elements are in the scene as well. Having to match and integrate practical leaves and twigs that were captured on stage and have them perform alongside computer generated ones not only keeps us honest, but it’s also great reference. — In the case of the onscreen assembly, the boat was mounted on a hexapod and animated in two parts on the stages. We created collision geometry for the boat and set it into our established KUBO water system. The bulk of the debris that is traveling through the air was created in visual effects. The initial idea was to have the sail sweep past camera and reveal a fully assembled boat that we would capture as a separate plate on the stages. But in the end it felt like too much of a cheat. So we made sure that the assembly was still resolving following the wipe and right up to the moment where we cut out of the shot.
The heroes are confronted to big eyes in the sea. How did you enhanced this sequence?
The underwater sequence was another first for the studio. All of the underwater elements – the Garden of Eyes, KUBO, seaweed and plant life – were animated practically on the stages. We enhanced the sequence in Nuke using a 2.5D approach. VFX added atmospheric perspective, plankton, volumetrics and color grading to drive the visuals to a near-literal match of the original production artwork.
KUBO grandfather is turning into a bid dragon. Can you tell us more about it?
The idea for the grandfather transformation was that he would appear to dry up, like a reptile shedding skin and form something of a husk. We would then register a struggle from within the husk, followed by a burst of light as the Moon Beast rocketed skyward. – At the start, I really thought we’d end up fully digital with this effect, but Oliver Jones and the rigging team were confident that they could come up with an in-camera solution. In the end, it was a hybrid of multiple techniques. Stages ended up creating a wrinkled, baglike facsimile of the grandfather, then rigged it in such as a way that, when animated, you’d sense there was something inside struggling to get out. They also animated the tearing open of the husk frame-by-frame and shot a separate plate of the Moon Beast puppet. Our job was to handle the transition of the puppet to the husk and then composite the Moon Beast element along with additional volumetric light and some carefully-designed FX goo. We tried morphing the puppet to the husk, but it felt too synthetic for the style of the show. In the end, Eric Wachtman, our lookdev lead, hand painted each of the frames for the transition and our compositing team brought all of the stage and digital elements together to make it work.
Can you explain in details about the transformation of the ceremonial lamps to the birds?
The first thing that comes to mind when I think about that moment in the film is sitting in editorial with our lead animator, Nick Craven and Travis watching version after version of ideas for the lantern to bird transformation. It takes place at a very emotional point in the film and it was very important that it not only mimicked what had already been established with the other origami performances, but also needed an added sense of elegance. Nick came up with some very creative ideas for shape changes, but Travis really pushed for elegance through simplicity. We held on to that direction as we continued toward a full realization of the effect. They were so many paths that we could have gone down, but we kept it graceful by avoiding overly-complex FX and drove the final look using a simple optic glow and show texture.
Is there any other invisible effects you want to reveal to us?
The unsung heroes at the studio are – without question – the roto/paint team. Their work is intensely detail-oriented and the process can be a real grind during production. We do an enormous amount of cosmetic work on the puppets and well as rig removal and plate repair when an animator bumps a light or a tree or steps into a reflective surface. On KUBO, we decided to add one more task to their already overloaded plates. We blended the gap that typically exists between the face of the puppet and the eyelid. It’s a subtle thing, but it gave the characters a more human look. It was different. It looked so interesting. And I think it helped further humanize them in a way that hopefully draws more of a connection or empathy from the audience. It was an enormous task and they shouldered it.
What was the main challenge on this show and how did you achieve it?
The biggest challenge of the show was its scope. In term of environments, effects and characters, KUBO was enormous. It was far beyond anything that we had accomplished previously at the studio. But despite the enormity of the task, we still needed to work within a similar budget with similar resources as previous productions. Fortunately many of the artists at LAIKA have been here for all four films. We’ve all gotten better at what we do and we’re able to work far more efficiently. Part of this is due to the refinement of the workflow. Part of this is due to the increased collaboration and inter-departmental communication. Part of this is due to the more robust technology in our infrastructure. And part of this is due to the amount of experience we have in creating these types of films. We’ve made plenty of mistakes together in the past. But we’ve learned from those mistakes and are now able to leverage that experience to achieve higher quality results within a tighter timeframe. Our experienced artists are truly our greatest assets.
Was there a shot or a sequence to prevent you from sleep?
We work long hours. When I get a chance to sleep, I sleep.
What do you keep from this experience?
I think the VFX team had a lot to prove with KUBO. The film is epic and, in terms of scope, was so far beyond anything we had previously accomplished. That said, our team really hasn’t grown much in terms of size from the CORALINE days. We knew that because of the scope, VFX would have a big impact on many of KUBO’s visuals. It’s very important to me that everything that ends up on the screen, if it’s created on a computer, honors the art of stop motion animation. We achieve that goal through intense collaboration and practical reference materials. We defined the process during PARANORMAN and THE BOXTROLLS, but KUBO was the film that really needed the machine to start humming. And the team came through in a way that was awe-inspiring. So as far as what I’ll keep from the KUBO experience – it’s a trust in the VFX team at LAIKA. I now know with absolute certainty they will step up and overcome the challenges of realizing these films, no matter how daunting the task at hand may be.
How long have you worked on this show?
I started working on KUBO in 2013. It was a year before animation began. That’s typical at LAIKA for VFX. We’ll jump in during pre-production, collaborate during development, and then start shot work as soon as the lights go on and animators start pushing puppets.
How many shots have you done?
In the end, the VFX team will touch every frame of a LAIKA film. In some cases, we won’t do much more than a quality control pass that might include some simple touch up or the removal of a dead pixel. It’s our job to make sure that every frame is flawless.
What was the size of your team?
We have a core group of about 25 artists that will swell to 60 or so during the height of production. Of that sixty, about half is dedicated to cosmetic work and compositing.
What is your next project?
I wish I could tell you about it. I suspect there may be an announcement by end of year.
What are the four movies that gave you the passion for cinema?
It sounds cliché at this point, but STAR WARS was the first film that really rattled my brain. I not only needed to see it multiple times (mind you – I was 8 years old – so getting to the cinema was no easy task), but it completely overtook my world. The action figures, the models, the posters… I surrounded myself with all of it.
Another would be E.T. I remember it really affecting me emotionally in a way that I hadn’t experienced before. I totally bought in, empathized and wept with Elliot when we thought E.T. had died.
TERMINATOR 2 was the film that opened my eyes to the potential of computer-driven effects to enhance storytelling. After seeing it, I driven by curiosity to understand how those visuals were created. It led me to the VFX world. I’m grateful for that.
And the fourth – I’d have to say – is THE SEVENTH VOYAGE OF SINBAD. I watched it countless times on television in my youth. The Cyclops is, hands down, my favorite Harryhausen monster.
A big thanks for your time.
KUBO AND THE TWO STRINGS: Creating the VFX Masterpiece
// WANT TO KNOW MORE?
– LAIKA: Official website of LAIKA.
© Vincent Frei – The Art of VFX – 2016
