THE WALK: Kevin Baillie – Overall VFX Supervisor – Atomic Fiction

During his last visit, Kevin Baillie talked about the work of Atomic Fiction on STAR TREK INTO DARKNESS. He then worked on NEED FOR SPEED and TRANSFORMERS: AGE OF EXTINCTION. He then takes of the effects supervision of the new film by Robert Zemeckis, THE WALK and opens at this occasion an Atomic Fiction office in Montreal.

You already worked many times with director Robert Zemeckis. How was this new collaboration?
Working with BobZ gets better and better with every film. I think that’s half human nature (the better we get to know each other, the better I can anticipate his desires and the more we can play off of one another’s ideas), and half Bob’s continuing growth as a filmmaker. It’s truly remarkable to see how someone as accomplished as Bob continuing to push the envelope, both in terms of both storytelling and visuals. I’ve seen him blend ideas from his live action movies with principals he developed in mocap films, and do things that are truly unique. No other director has the particular flavor of experience to do it like Bob does!

What was his approach about the visual effects?
His approach to visual effects is something I really admire: Priority #1 is for VFX to support the story. Priority #2 is – you guessed it – to support the story. 😉 That’s what I feel makes Bob’s movies so special. The effects are there to make the audience feel something. In the case of THE WALK, it was to make people feel like they were up there in between the Twin Towers, walking on the wire with Philippe. He gives moviegoers the chance to experience the wonder, satisfaction and adrenaline that Philippe did during his walk. It’s simply unlike anything I’ve ever experienced in cinema before.

A bit of my own color commentary on why that’s special: I feel that, in today’s blockbuster-driven world, effects are the story more often than they help to support the story. As a consequence of failing to relate VFX to humanity (and plausibility), they can lose their emotional, primal “lizard brain” impact. Those effects are still amazing art, are super fun to work on, and result in thrill-ride-style box office successes. But, all to often, I feel like they’re numb compared with the emotional impact created by Bob’s use of VFX.

THE WALK is also the first project for your Montreal office. Can you tell us more about it?
Well, I can tell you this: building a beautiful, state-of-art 100 person office while shooting 15 hour days on set is quite a challenge 😉 The good news was that we had a 15,000 square foot “blank slate” with beautiful, old hardwood floors to configure exactly the way we wanted to. The bad news was that we had to do it all from scratch, and had very little time to do it! Luckily, we had some great partners in Montreal at FOR Design Planning (Melanie & Andre) and Gold Castle Holdings (George & David) who helped us to get it built in time to start working on THE WALK.

Once we had construction under way, our attention turned planning the team and workflow. We knew that we wanted our Oakland office to collaborate closely with the team in Montreal, so we had to make some “major” pipeline changes. We hired a talented producer, Julie Cardinal, and the core of our CG/Development team for the project, Laurent Taillefer and Marc-Antoine Paquin, to get the show going. They did a fantastic job of helping to staff our 70 person team, and implementing all sorts of new tools: Shotgun for asset management and cross-site data synchronization, Katana/V-Ray for lighting and rendering, and more!

At the end of the day, we found that it was incredibly important to develop tools that allowed the two offices to communicate well. It was also very important for our Head of Production, Sandra Scott, and I to spend the majority of our time with the team in Montreal, since we wanted the new office to really feel like an extension of Atomic Fiction’s brand, rather than a totally separate office. In think we’ve succeeded in avoiding “Atomic Oakland” vs “Atomic Montreal.” We simply operate as a single “Atomic Fiction.”

Can you describe to us one of your typical day on-set and then during the post?
I honestly don’t think there’s such a thing as a typical on-set day! Well, maybe 14-15 hour days are typical, but that’s where the similarities end.

My partner in crime, VFX Producer Camille Cellucci, and our wonderful on-set colleagues Francesca Mancini and Ken Caines, had our hands full pretty much every day. Sometimes my duties involved taking extensive reference photos and HDRs, sometimes it was conferring with cinematographer Dariusz Wolski about how to achieve a tricky shot, and sometimes we were balancing budget-related decisions for an under-construction set with Production Designer Naomi Shohan and Supervising Art Director Felix Lariviere. On-set life is HARD work, and is also the furthest thing from boring. 😉

My favorite moments on set, which happened quite frequently, were when Bob would holler “Kevin!!!!!!” That was the sign that he’d come up with a new and interesting idea, or had found himself up against a production challenge of some sort. That’s not to say that VFX are always the solution or he ever used “fix it in post” as a crutch – it’s that he wanted to involve VFX as a possible tool to achieve his vision, or solve that problem. In that regard, Bob is the master of collaboration, and is always involving the key members of his team in the problem solving process. No egos, no agendas. Just “what’s the best solution for the film?”

A big part of the shooting was on greenscreen. How did you help the cast and crew to visualize the environment?
There was, indeed, quite a bit of green screen work on the show. The WTC plaza and lobby, as well as Notre Dame and French pond, all featured copious amounts of green. The biggest green screen was, of course, for the climactic sequence where Philippe Petit walks the void between the Twin Towers. We had a 40’x60’ set piece for the WTC rooftop, a very long cable for the wire walking action, and everything else was green!

In order to help the team visualize the environment, we tapped both SolidTrack and The Third Floor to provide real-time composites of the world. The raw green screen would be fed to one monitor for Bob, and the other monitor would show the “simulcam” comp. Luckily, Bob only ever shoots with one or two camera simultaneously, so it was possible to give him a real-time view of the world that his hero was seeing most of the time.

Robert Zemeckis likes really long shots. How did you manage these shots?
He certainly does! And THE WALK was the perfect film to use that style of filmmaking. Bob knew that the film was going to be 3D since the day he conceived of it, and we spent a bunch of time in pre-production establishing the “rules” for how to shoot footage that would result in an exceptional 3D experience for the audience. Between Bob, myself and Stereo Supervisor Jared Sandrew, we agreed that giving the audience time to “soak in the view” is critical for a successful 3D experience, so long shots were the goal from the get-go.

As a result of Bob’s natural style, combined with the 3D “rules,” the entire movie only had 826 shots in it. That’s not a typo. Whereas most modern movies have around 2,500 shots, THE WALK had a third of that! 672 of those were VFX shots, which equates to about 2,000 VFX shots in “normal movie screen time.” Our opening shot alone is almost 2 minutes long!

In general, when a shot increases in length everything associated with creating the shot increases: complexity, man hours, computing power, etc. The teams don’t have cuts to hide between, so often times an environment has to be “movie ready” in 360 degrees, much like it does in VR. Perhaps the greatest challenge with a long shot is for the VFX to maintain the audience’s suspended disbelief for an extended duration.

The movie have a lots of really nice transitions. Can you tell us more about them?
Bob has always been a master of subtle, but mind-bending, transitions. Just look at CONTACT’s opening shot!

We did a bunch of tricky transition work in THE WALK, and all of the show’s vendors – Atomic Fiction, UPP, Rodeo FX and Legend 3D – had to share in that challenge. One of those shots, in which we transition from Philippe balancing on one leg during his first South Tower visit, is over 3,000 frames long, and proved to be a particularly brutal challenge for our wonderful 3D Conversion team at Legend 3D.

The great thing about those transition shots is that artists tend to love working on them. So do I! When they’re done well, the audience is left scratching their heads. Again, carrying forth the general theme of Zemeckis’s VFX, they make people feel something!

How did you approach the New York recreation?
Recreating New York in 1974 was an immense challenge. We started by doing a massive amount of research, scouring the Internet for reference photography, searching libraries for books on the topic, and even calling the New York Port Authority for information we couldn’t find elsewhere. We also flew a helicopter above modern-day New York for two days, in order to gather reference footage of how a city feels from that height: how much motion is visible in cars, people, stop lights, etc? All of the photographs from back then are on film, and New York looks significantly different today from what it did back then, so we couldn’t use any of those photos literally. As a result, the team had to re-create the entire environment entirely digitally.

Can you explain step by step about the creation of the huge New York environment?
Rodeo FX handled the views of New York from the ground level. Their Montreal-based team built amazing reproductions of New York city streets, the under-construction World Trade Center Plaza, and the lobbies of the iconic buildings themselves. The elevator banks in the lobby areas presented a particularly nasty challenge: they were coated in chrome from floor to ceiling! This meant that any time the camera shot towards the elevators, we were actually seeing the entire world reflected behind us. That was a lot of work, but it was well worthwhile since it resulted in a sense of scale the Bob really wanted!

For the climatic “walk” sequence of the film, the lighting and weather needed to evolve in lock-step with what Philippe’s character was feeling. In order to accommodate that requirement, Atomic Fiction’s team built all of Lower Manhattan in excruciating detail. Every single rooftop air conditioning unit, rain gutter, and hot dog stand was built in 3D using Maya and Modo, textured in Mari and Modo, and rendered with V-Ray for Katana. Brian Flora and his environment team painted on top of those lit 3-D assets in Photoshop to give them that final “kiss of love,” as we like to call it. Nuke was the final stop for the shots, where all of the environmental haze, flocks of birds, and crowd systems (which were developed in Nuke 3D) were added.

Have you developed specific tools and pipeline to create New York?
The fun thing about VFX is that every show presents an opportunity to develop tools in the name of achieving better, faster, bigger and cheaper (or some combination of those). New York was particularly challenging, because of the amount of data that went in to recreating it. Not just the buildings themselves, but also all of the movement in the city. Every car had to move, the fans on rooftops needed to spin, and little crowds of people had to gather to view Philippe’s epic walk.

Since we had a large amount of work to do, and a relatively small team of lighters to work with, we decided to unleash Katana for the first time at Atomic Fiction. Katana, paired with V-Ray for Katana, was the perfect solution for the demands of the show. We also replaced our internal asset manager with Shotgun, in order to handle the massive amounts of assets and cross-site challenges from having studios on either side of the continent. When all of those components started to work together, it allowed us to make a change in one place, and have “the system” automatically and instantly populate that change across hundreds of shots.

We also developed a number of automation tools for our matte painting department. The hero paintings took over an hour to open and save due to their massive size, and contained 26 master layers (with MANY sub-layers), each of which needed to be exported, mip-mapped, checked in to Shotgun, and connected in to the master Katana render scene. Managing that level of complexity would’ve been insanely labor intensive, so building some automation allowed the team to focus on where their real value and enjoyment comes in: making stuff look great!

New York is seen is various lights and weathers conditions. How did you handle these changes?
That was a lot more challenging than the subtlety of the effect might lead audiences to believe. The changes in lighting meant that we couldn’t simply create a single painting and re-use it for every shot. Instead, we’d have to build the city in obsessive detail, so we could keep the lighting “live” in 3D, with matte painting layered on top of that to give that final touch of realism.

In order to give the team a starting point, Atomic Fiction’s Art Director Brian Flora created a series of concept images that were representative of the various stages of progression, looking in all of the “hero” directions. We were able to get approval on those fairly quickly, after which the teams went town matching the target looks in 3D.

An interesting point on this topic is that we had to create 360 degree skies for all of the points in the lighting’s progression. Because the towers are such an important component of the film and they’re very reflective, even things behind and above the camera’s view affected the way that they looked. So, it was important that we took a VR-like approach since, in one way or another, the audience was going to see & feel the entire world around them.

The twin towers themselves are almost like characters in the film. Can you tell us about how you created them?
It was incredibly important that World Trade Center Towers looked (and felt) the way that people remember them. This film is, in a way, Bob’s love letter to the towers, so they needed to be absolutely perfect!

Atomic Fiction’s Associate Visual Effects Supervisor Jim Gibbs devoted his substantial talents to overseeing the task. We started off with blue prints of the actual towers, as well as those from the production Art Department. By combining those, we were able to build all 110 stories in a way that was really accurate, and also matched in to the production footage that we’d shot. We had to be careful to not make the construction “CG perfect” so, between modeling in Maya and texturing in Mari, we made serval passes at adding “chaos” to the surface. We found that introducing too much imperfection made the towers looks small. Since scale was so thematically important in this film, we spend a lot of iterations, viewing the model from every conceivable perspective, to find the perfect balance between “perfect” and “small.”

For the shading of the towers, our mission was to emulate that anodized aluminum finish that gave the towers their distinctive look. Thousands of microscopic bumps in every inch of their finish made the towers almost chameleon-like. In bright sun, they looked white; on a cloudy day, they appeared to be charcoal gray. Looking at the broad face of a building it would look completely matte, whereas at steep raking angles it would seem almost mirror-like in the way it reflected the surrounding city. To recreate such a versatile surface, we turned to V-Ray’s GGX shading model, which does an amazing job of emulating the surfaces’ microfacets. This was the first large-scale use of the GGX model in feature films, and boy did it pay off! The difference it made vs traditional BRDFs was like “magic,” and is hard to put in to words. An object that looked like “great CG” using Blnn or Ward, for example, made the almost-intangible leap to “real” using GGX. It was critical in nailing the look of the towers!

With such a big and detailed environment, how did you handle the render time and storage challenges?
With a pipeline that enables a small group to be incredibly efficient comes the need to do A LOT of rendering, and iterate very quickly. We turned to Conductor, a cloud-based rendering platform, for the show’s rather substantial rendering needs. During production, Atomic Fiction regularly spiked to over 15,000 cores in our cloud-based render farm, and ended up rendering 9.1 million hours’ worth of VFX (that’s over a millennium on a single processor!) in Katana/V-Ray and Nuke. I’d owe a HUGE shout-outs to Conductor, The Foundry and Chaos Group for their support in enabling a feature-scale cloud rendering workflow. THE WALK is proof positive that giant, A-List shows can be run through the cloud. Not only can they be, but the fact that Atomic Fiction saved about 50% thanks to Conductor proves that VFX should be run through the cloud for economic reasons too. It’s a happy story all around!

How did you help the conversion work?
I worked closely with Legend 3D’s Stereo Supervisor, Jared Sandrew, who did an amazing job crafting the 3D on the film. Before production even started, we got the entire crew together and laid out the rules for “good 3D.” Bob wanted everyone to know what to strive for, and avoid, in the name of stereo. Even though we shot the movie in mono, Bob, his cinematographer Dariusz Wolski and I were thinking about stereo with every setup during the shoot. Bob and his editor Jeremiah O’ Driscoll carried the 3D-savvy mentality through the editorial process, using long shots to give the audience time to soak in the view.

In post-production, Jared and I would look at every shot together at multiple stages of the conversion process. Jared and his team would first create a rough “depth pass” of entire scenes, which we could adjust together in real time. Since the conversion process and VFX were running side-by-side throughout the production, the depth pass was often done using temp VFX. Once VFX had been completed on a shot, the vendor responsible for it would package up their Nuke comp, and send the entire live script to Legend 3D. By delivering the Nuke script with all of its associated inputs, it saved the VFX vendors from spending time to rendering our stereo-specific layers, and allowed Legend 3D to break out elements in to layers exactly the way they wanted to for every shot. Once the delivery to Legend 3D was complete, their team would put the finishing touches on the shot, and we’d review complete sequences with Bob for final approval. Thanks to the constant communication from pre-production all the way through to the final shot, we never really ran in to any surprise disconnects between Bob’s vision and what Jared’s team had executed. It was, as per Bob’s original marching orders, “all awesome 3D, all the time.”

I think it’s worth noting that, thanks to THE WALK, I’ve been converted (no pun intended) to a 3D conversion believer. I think conversions aren’t just as good as natively shot stereo – now I think they’re actually better. Why? With conversion, we have the ability to dial 3D in shots interactively. That gives filmmaker the flexibility to emulate what the human visual system does. Our eyes do so much cool stuff, selectively filtering parts of our world to make them palatable to our brains. Shooting natively force-feeds literal reality into the audience’s eyeballs. Converting allows us to show the audience imagery that feels natural, and is the cinematic version of reality. After all – slightly altered reality is what cinema is all about!

Many VFX studios worked on this show. How did you split the work amongst them?
At the time the show entered production, Atomic Fiction didn’t have the capacity for the entire VFX workload on THE WALK. For the shots that Atomic wouldn’t be doing, VFX Producer Camille Cellucci and I were given the task of finding a small handful of vendors that would be able to deliver the combination of quality + budgetary efficiency that the film called for. Bob, his producer Steve Starkey and Sony’s Lori Furie fully entrusted us to make the best picks, which put more than just a little pressure on us to put together the right team from the get-go! 😉

After an exhaustive process of bidding and discussions with an amazing roster of studios, we ended up with three. UPP in Prague, led by their VFX Supervisor Viktor Muller, was entrusted with the Paris-based scenes, as well as the nighttime rooftop work during Philippe’s coup. Camille had worked with them before, and had loved their quality and user-friendliness.

Rodeo FX, one of the longest-standing shops in Montreal, was tasked with the ground-level New York work, including rather substantial WTC plaza and lobby set extensions. With Sébastien Moreau VFX Supervising their work, Rodeo also handled the “interview” shots on the statue of liberty torch.

Every company on the project did an AMAZING job in realizing Bob’s creative vision, and did so within a budget that demanded effective teamwork & an ample helping of ingenuity. My hat is off to every single person who contributed to the VFX on this wonderful film!

Did you share assets between the VFX studios?
Absolutely! It was the only way we could make the movie on budget. Plus, asset sharing just makes sense…

Atomic Fiction was responsible for building the hero WTC Tower assets, and shared them with both Rodeo and UPP. Since UPP’s nighttime rooftop shots were in the exact same location as Atomic’s daytime ones, the city below from Atomic also served as a starting point for the UPP team. They did an amazing job in “turning the lights on” in the city. Night shots are surprisingly difficult to achieve, since “cinematic nighttime” and real nighttime are two very different things, and they absolutely nailed it!

Every vendor made sure that their “comp package” deliveries to Legend 3D included z-depth and, if available, deep data. While those things aren’t exactly “assets,” they helped Legend do a 3D conversion that has all of the detail of an asset – every bump, every ridge, every leaf. I think that level of collaboration, from the begging to the end of the process, made a big difference. The movie doesn’t just look 3D. It feels 3D!

How did you organize your team to ensure the consistency of the shots look?
Well, in some ways, the super tight budget made that easier. We had smaller teams to work with, and needed to figure out how to achieve greater levels of efficiency amongst them. Fewer people means less places for things to go astray! 😉 Tools like Katana really helped Atomic’s team maintain consistency amongst tons of shots at once. Shotgun, RV and cineSync were all critical parts of the review workflow between vendors and BobZ’s fantastic editorial team. Technology really helped!

On the production side, I had the pleasure of working with the most amazing group of people! Camille Cellucci, Jean Huang, Kelvin Padfield, Francesca Mancini and Greg Ruffs were absolutely instrumental in keeping everything straight in postproduction. Managing all of those 2D versions and, at the same time, keeping track of the 3D conversion process, was a herculean feat. They handled it with style and grace, never batting an eye when things would get crazy. I really leaned on them, since I was simultaneously facility-side supervising Atomic, studio-side supervising the other vendors, and tending to my duties as CEO of Atomic Fiction and ConductorIO. Needless to say, that was insane. I wouldn’t have been able to do it without their hard work and support!

What was the biggest challenge on this project and how did you achieve it?
From a VFX execution standpoint, I’d say that one of the biggest challenges is the one effect that absolutely nobody is talking about: how was Joseph Gordon-Levitt going to do all of those wire walking stunts? Well, as talented as he is – and he did actually walk the wire himself – he couldn’t physically accomplish the more complicated wire performance bits that take decades of training to master. For that, Bob brought in professional wire walker and performer, Jade Kindar-Martin. Zemeckis wanted to see Joseph’s likeness on the wire at all times, however, so Atomic Fiction built an exact, performance-capable digital reproduction of his face. We tapped Pixelgun for the photogrammetry-based scanning of Joe’s face and body, which gave us detailed geometry and perfectly registered textures for every pose that was scanned. We were able to use that data to accurately replicate not just facial shape animation, but also blood flow underneath the skin. Since Joes’ performance was so subtle on the wire, we had to replicate every single aspect of his face, no matter how subtle it was!

In post-production, Atomic Fiction’s Oakland-based crew transferred Gordon-Levitt’s acting, which had been recorded and first-pass tracked by Faceware Technologies, to the digital face. Atomic’s matchmove and animation team transposed that onto Jade’s body, and proceeded to sweeten the performance to perfectly match Joe’s. The result is an indistinguishable likeness to the Gordon-Levitt-acted Petit! Pig props to Rudy Grossman, Michael Palleschi, Jesse Kim and Julie Jarros for leading the charge in creating a truly invisible illusion!

On a much more personal note, my biggest challenge was to instill our team with the belief that they could actually get the show done. The amount of green screen in the first cut of the film’s climactic scene was truly intimidating. That, combined with a pretty brutal push for the movie’s teaser trailer, made it feel like we might have a nearly impossible path ahead of us. We even had some key VFX artists quit in the middle of the show which, given how much I believed in the project and care about the company, hurt me pretty badly. But, just like Philippe does in the film, we soldiered ahead! I’d go as far as to say that the adversity actually made the team better, and strengthened its bonds. Now that we’re done with the film, and it’s receiving such amazing critical feedback, it occurs to me that Petit’s quest and ours might not have been so different. They both seemed impossible, we had defectors and detractors along the way, yet the team who committed themselves to achieving the cause accomplished something truly beautiful!

Was there a shot or a sequence that prevented you from sleep?
If a show doesn’t have one of those, it’s probably not pushing the envelope! 😉

One of those scenes was the film’s teaser trailer, in which Philippe initially visits the South Tower rooftop. It was the first “1974 New York” scene that we’d done for the film, so we had to address a lot of workflow and creative questions in a very short amount of time. Being that we were just setting up a new pipeline and building assets while in shot production, the hours turned out to be pretty rough. I ended up literally not sleeping on several occasions, jumping on to the box to help the matte painting team after they’d gone home for the night to recharge. Everyone worked really, really hard to make that deadline!

The scene right before Philippe steps out on to the wire was another worrisome one. Bob wanted Philippe’s world to dissolve away in that moment, acting as a metaphor for the intensely-focused state of mind that most adrenaline junkies are familiar with. I suggested that a cloud/fog bank moving enveloping the towers would be a naturally-plausible way to create the “void effect,” as Bob called it. It would allow for the world to disappear, while still giving the audience depth cues to hold on to. Bob dug the idea, so off we went! Being inside of full-screen volumetrics for a few minutes of screen time wasn’t going to be easy though. Ronan Tanchum led the charge on the effect, simulating the clouds in Houdini and rendering them in Mantra. We used deep compositing on the scene, primarily in order to reduce render iterations. We were able to achieve incredible amounts of flexibility in comp by using the deep compositing tools in Nuke. Jed Smith and Mark Joey Tang controlled the lighting color in comp, “composited” the 3D holdout of the tower tops from V-Ray (also rendered deep), and were even able to remove or modify clumps of cloud detail that were in the midst of other volumetrics. Thanks to great 3D elements, and being able to sculpt and light volumes in comp, the scene ended up being truly beautiful!

What do you keep from this experience?
Postproduction on THE WALK was truly a case of life imitating art. The movie’s inspirational message is simple: anything is possible. We started out with what seemed to many like an impossible goal, and we achieved it in spectacular style. So, like Philippe’s walk but with less physical danger, the hard-working teams on the project proved that anything is, indeed, possible!

How long have you worked on this film?
It’s been about 8 years since I was in my first meeting with Bob about THE WALK. That was long before the documentary MAN ON WIRE, and it was originally conceived as a motion capture film. I feel very fortunate to have been a part of the movie’s evolution over the years, and am super proud of the finished film! In my opinion, it’s a spectacular piece of filmmaking, as only Robert Zemeckis can create. On top of that, when seen in IMAX 3D, it’s the only movie I’d consider to be just as effective as VR, if not more, in giving the audience an “experience” that they react to – both emotionally and physically.

How many shots have you done?
Atomic Fiction did about 250 shots on the show, which was the equivalent of 750 on a “normal” movie 🙂 That equated to almost 40 minutes of the film!

What was the size of your team?
At our peak, we had over 120 people working on the Atomic Fiction team. UPP, Rodeo FX and Legend 3D had amazing teams dedicated to THE WALK as well. I’d guess that, in total, probably 500 VFX artists’ hands touched the film!

What is your next project?
Atomic Fiction is currently in production on DEADPOOL and STAR TREK BEYOND. BobZ always has something new and interesting up his sleeve for us too, so we’re in early pre-pro on that. We’ve got a pretty full house in our California studio, and are continually our Montreal team. We’re looking for exceptionally talented artists to join our incredible studio there (it really is an awesome facility!), so if any of your readers think that they’re a good fit, please send ‘em our way!

A big thanks for your time.

// WANT TO KNOW MORE?

– Atomic Fiction: Official website of Atomic Fiction.

© Vincent Frei – The Art of VFX – 2015