In 2012, Ryan Tudhope explained to us in detail about the work of Atomic Fiction on LOOPER. He then take care of the visual effects of projects such as FLIGHT, THE LONE RANGER or GAME OF THRONES.
How did you and Atomic Fiction get involved on this show?
Jonathan Rothbart (production-side VFX Supervisor) is a close colleague of mine, and we were excited about the project the moment he told us he was involved. Jonathan was one of the founders of The Orphanage, which my business partner Kevin Baillie and I were the first two employees of, so he has a long history with us and our team. It was great to have the opportunity to work with him again after 7 years or so. We joked about how he was my boss back then, and he was my boss again on DEADPOOL, so it was just like old times!
How was the collaboration with director Tim Miller and VFX Supervisor Jonathan Rothbart?
Tim and Jonathan were amazing to work with. They really embraced the essence of DEADPOOL and gave us guidance and encouragement to do the same. Tim’s vision of the movie was contagious. He established so much about the look and feel before we got involved, whether from the test footage that his team at Blur Studio created, or the pre-vis we started from, or the live action photography we inherited as plates. The guys always knew exactly what they wanted, which made the project really enjoyable to work on.
Jonathan is great at pushing teams to do their best work. He’s a master at designing action sequences and has a great eye for every type of effect. I’ve worked on many projects with Jonathan, so I know his style and what’s important to him. Having that history gave us an advantage since we could have an honest dialogue and simply focus on making shots look great. He’d text me, only half joking “What the fuck is going on with that shot?” and I’d reply “I know dude, we’re on it”. It was pretty unique.
What was their approach about the visual effects?
The entire crew was simply trying to find the best solutions given the time and resources available. What’s great about this is that it forces you to work harder and be more creative, and DEADPOOL is a great example of this. The project was in that sweet spot where we had just enough resources to do great work, but not enough that we could be irresponsible. Every decision mattered.
What are the sequences done by Atomic Fiction?
We were fortunate to work on two sequences — the “Freeway Chase” which was based on the footage that helped get the movie green-lit, and “Deadpool Counts Bullets”, which occurs on the viaduct where the Freeway Chase ends.
Can you describe one of your typical days during the post?
Well, there were several phases to our process so it’s hard to identify a typical day.
In the first month or so, I spent a lot of time meeting with my Associate VFX Supervisors Jim Gibbs and Mike Janov, Animation Supervisor Webster Colcord, CG Supervisors Rudy Grossman and Laurent Taillefer, Environment Supervisor Seth Hill and Comp Supervisor Woei Lee. I recall a few meetings where we grabbed beers at the restaurant below our office and discussed our strategy for the sequences, what technology and pipeline tools we’d need to focus resources on, and the general vision for the creative. Through the course of this meeting and many others that would follow, one of the most important things I was hoping to do was build strong bonds between the supervision team. I knew this would be important months later, as we found ourselves scattered and in the throws of production.
Casting required a lot of focus in the early weeks and months. As our crew grew, we held pretty standard daily reviews in the morning and check-ins in the afternoons. I’m a big believer in full-crew dailies early in a project, as I feel there’s a lot of valuable information that comes out of those early sessions. I think it bonds the team but also gives everyone perspective on our broader goals, since they can see what each other is working on and can hear notes that might be useful to them down the road.
By mid-show, a typical day involves reviewing everyone’s progress and keeping an eye on things from a gut level. Through the course of that, Jim, Mike and I would generate lists of all sorts of things that need to be improved or kept on our radar. Some things we might identify needing attention might be management related, such as how well dailies are working between our two studios, the morale of the crew, the casting of the work, the need to hire additional resources, etc. From a creative standpoint, we might have thoughts about the look of a particular asset or character, a concept frame we need to do, or have an idea for a way to achieve an effect procedurally across an entire sequence, such as caustics bouncing off the vehicles and windows. Or there might be an entire type of effect that we have yet to figure out. We keep track of all these things on a whiteboard in our office, and then attempt to solve them. That is, figure out who is best to own the problem and then work with them toward that solution.
In the final month of the schedule, our team is generating hundreds of takes each day for review. By that point, we spend most of our time looking at shots, reviewing them in the cut and sharing notes. I divide responsibility with the rest of the supervision team, so we each have an area we’re keeping an eye on. We still have dailies every morning but not always with the full crew. Outside of dailies, I sit at my desk and watch Shotgun as the takes drop so I can give feedback immediately. If it’s a simple note, I might just type it in and send it to the artist via Shotgun. If it’s more complex or require brainstorming or discussion, I’ll run to the artist’s desk or see if they can swing in to mine. We spend a lot of time working one-on-one with each other – our crew is absolutely stellar, and working with them was really rewarding.
The freeway sequence is a massive sequence. Can you explain step by step about the creation of this great sequence?
1) We started with pre-vis that our friends at Blur had developed with Tim and Jonathan, so that established the intent of the sequence as a whole.
2) We knew that car interiors would be filmed on a green screen stage, while exteriors would be full-CG.
3) We started by sketching plan views of the city to get a sense of what we were building. We were able to use the pre-vis to determine how much freeway needed to be developed, since we knew how long the sequence played and how fast they were traveling in that period of time. Through brainstorming with Jonathan, and hearing his viewpoint of the layout, we determined the sequence should start on the outskirts of our city (in an industrial area), and head into town, through a tunnel, emerging into a downtown area, and eventually on the other side of the city on a viaduct. The viaduct had to match the Georgia Viaduct in Vancouver where our “Deadpool Counts Bullets” sequence was being filmed.
4) Once we had a plan for our city, we started building a low-resolution model of the environment, and placed the pre-vis cameras along the route at the exact point they occurred in the sequence. At this point, you can imagine a 2-3 mile stretch of freeway with clusters of cameras along the route.
5) Knowing where all our shots took place within our fictional city enabled us to prioritize the areas that needed high-res modeling. Webster, Seth and Laurent worked together to split the city into zones, which was a strategy that came out of those early meetings. The zones allowed the crew to load only what they needed when working with large files, but it also defined sections of the sequence that would be animated in continuity (rather than per shot). Further, separate zones allowed our team to split up the work and move entire sections of the sequence forward.
6) Meanwhile, our asset team was hard at work building the hero SUV’s, digital doubles, overpasses, buildings, props (road dividers, cones, trash, etc.) background vehicles and everything that would populate our world. Additionally, we were setting up the pipeline that would allow us to travel large distances without having to do everything by hand. A lot of time was spent on the look of the road surface itself, which was handled by Seth Cobb (Texture Lead) as well as texture variation between dividers, overpasses, etc. We built upon our Katana pipeline to enable a lot of the variety in our world to happen procedurally. We were also developing the various FX systems that would be used, such as water spray from the road, exhaust, etc.
7) The first reviews with Jonathan involved “blocking animation” where we essentially recreated the pre-vis within the world we were building. From here, Jonathan and Tim either approved the design of the shot, camera motion, framing, etc. or asked for changes.
8) With the rough blocking approved, the animation team would begin refining the motion of the shots and work toward final animation.
9) At any point, our asset / environment team could pull in the latest checked-in camera to see how the hires assets were looking in a given shot. On the flip side, our animation team was able to import the improvements to the environment itself. So, as the animation was improving, the environment around the animation was getting more filled out as well.
10) As a separate process, our world was populated by a traffic system developed by Will Muto (TD). Mike Janov had climbed up on the hills overlooking US-101 near his home in Marin County, and filmed the traffic as reference. Will used that reference and built a system that emulated the subtle motions of the traffic perfectly, from the slow-vs-fast lane speeds, to changing lanes, and everything else.
11) As animation neared completion, our lighting team would pull in all the latest caches, dial the lighting to be just right, and render 3 test frames for us to review the look of everything. This allowed us to see how the assets were holding up, and we often went back and made subtle tweaks based on this.
12) Completing animation also unlocked our FX team, led by James Kirk, to be able to run out the various FX passes that would establish the atmosphere of the sequence. As I said, we added spray from the tires as well as smoke and fire during the crash shots. We also added subtle spray in the FG of a lot of shots, as if it was coming from the virtual camera car that was filming the shot.
13) Of course, there were many other steps related to the full-CG characters; shot modeling, cloth sim and hair sim. Our pipeline on DEADPOOL resembled a full-CG animation pipeline, with all the same steps.
14) The comp team was led by Mark Casey (Compositing Supervisor) and Jed Smith and Mark Joey Tang (Compositing Leads). They built comp templates that established standards regarding the LUTs and overall grade, sky dome orientation, haze, chromatic aberration, lens distortion and grain.
15) Through all this, our production team, led by Julie Cardinal, was integral in keeping us on track. That team, along with Robin Lamontagne and Cindy Lin (VFX Editors) spent countless hours ingesting and tracking everything, keeping our cuts up-to-date, making sure the artists always had what they needed and sending shots for review.
How was this sequence filmed?
Outside of the full-CG shots, the interior green screen shots were filmed on stage in Vancouver during principle photography. Jonathan, along with the DP and others, designed a massive LED panel array that played back footage around the SUV for more realistic reflections and interactive lighting. It worked great. I had the opportunity to supervise the filming of this footage, which we acquired via a 7-camera Red Epic rig built on a stunt car. The City of Detroit shut down a major freeway for us one morning, where we shot all of the plates. We also gained access to the locked-up freeway on foot, which allowed us to run around shooting hundreds of reference photos.We got photos of everything: light posts, barriers, trash, gravel, and the road surface itself. It was hugely valuable and set the visual tone for the sequence.
How did you created the massive freeway environment and the various cars?
As for the cars, we split them into three groups: Hero, Stunt and Background. The hero cars were assets such as the SUV’s themselves, or the BMW they crash into toward the beginning of the sequence. Stunt cars were also hero-resolution, and were based on a selection of 12 vehicles that Jonathan and Tim approved. Webster then decided where those vehicles should be placed on the freeway, to avoid seeing them repeat. Interestingly, this is how it works in live action too, since there’s typically a limit to the number of vehicles that can be rented and driven by stunt drivers. Lastly, we had another group of background cars that fed into the traffic system. Again, mostly generic vehicles with some busses and trucks thrown in.
Can you explain in details about the shading and lighting challenges for this sequence?
From a lighting standpoint, it was important to Jonathan and I that we varied things to keep the audience on their toes. That is to say, we knew that shooting a sequence like this on location would take at least 2-3 days, so the weather and quality of light would change dramatically during that time. So, we varied our lighting from overcast to sunny, and played with the sharpness of shadows in the same regard. We also “wet down” our CG road at varying levels for the same reason.
For the majority of our assets, we used V-Ray’s GGX shading model, which does a fantastic job at representing metallic materials. We used this shading model extensively on THE WALK as well, so our team is already familiar with how it behaves. Manuel Huertas and Jonathan Fleming-Bock were both responsible for many of our hero assets, such as the SUV’s, Ducati and a host of props and other elements.
As for rendering, we knew the biggest challenge would be memory, due to the size of our scenes and the number of textures. Of course, we wanted all the bells and whistles turned on too. Laurent, Bruno Gagne (Lead Lighter) and Marc-Antoine Paquin (TD) continued to build upon our pipeline, which is based on The Foundry’s Katana. They ended up rendering the hero vehicles and surrounding environment with brute-force ray tracing, and were able to keep memory under control by rendering the environment in layers when needed. Our implementation of Katana is one of the main reasons we were able to organize and template the full-CG shots. For many of the more complex shots, we bumped up to higher memory (128GB) instances on Conductor and we were off to the races.
The sequence ends with many car crash and one in particular that goes in slow motion. Can you tell us more about it?
The SUV crash, flip, and tumble was one of the most complex sections of the sequence to be sure. These shots involved the most intricate animation, 5 digital doubles and Deadpool in a single shot, complex FX work, rigid dynamics, etc. By this point in the sequence, we’ve also progressed to an area that exactly matches the live action Georgia viaduct location in Vancouver, so our environment had to match that world exactly.
For the slow motion shots, Jonathan and I decided to treat them as if they were shot with a high speed camera, so we animated the shots to a higher frame rate, and then sped them up in “post”. This gave the shots a more realistic sped-up look when they went from slow motion to regular speed. We were also able to determine exactly which frames would be throw-away, so we only rendered the frames that would be needed in the final composite.
Can you tell us more about your render techniques?
All of Atomic Fiction’s heavy rendering is done with ConductorIO, a cloud rendering platform we developed for internal use, and have since spun off as a stand-alone company. On DEADPOOL, 7 million core hours were rendered in the cloud. 80% of the rendering on DEADPOOL occurred in just 8 weeks of the 36-week schedule, with one single week accounting for about 20%. This really demonstrates how important it is to be able to scale resources up and down. Another way to look at it is that we would have needed a local render farm of around 32,000 cores to get through that one week. That’s a lot of horsepower for any sized company, let alone a mid-sized studio like Atomic Fiction.
Lots of destructions and FX are involved in this sequence. How did you created these elements?
Our FX team, led by James Kirk, was responsible for smoke, dust, water spray, blood, fire, glass, muzzle flashes, sparks, etc. Nearly all of these effects were treated as “systems” that we first developed and then later applied to shots. We also had an amazing team dedicated to the rigid body simulations, which was working closely with Webster as part of the animation process. For anything larger than fine particulate or small debris, we would run rigid simulations and subsequently cache that geometry as part of the animation. For the lighters, all the debris showed up in the scene just like the larger assets. We also treated rigid simulations as “mo-cap” for the car tumbling, first simulating the car flipping and then animating on top of that simulation to adjust things that could be improved.
Deadpool makes impressive jumps to kill the mercenaries. How did you help him doing them?
The stunt team was amazing, so those performances, directed by Tim, really set the tone for how Deadpool should move. Tim has an amazing eye for motion and stunts like this, so we really relied on his vision of shot design and pacing. He also directed a motion capture session and provided us with many of the key moments, such as these Deadpool flips.
From there, our animators integrated the mocap into the scene and made any modifications that were necessary to improve the motion. Even though we had great mocap to start with, there were still quite a few tweaks made to get the action we all wanted.
We can see the Deadpool bullets in close-up shots. Can you tell us more about them?
Sure, being that our sequence was titled “Deadpool Counts Bullets”, it was pretty obvious the close-up bullets were an important story point. At one point in the development of this asset, I called Jonathan to ask him if Deadpool was really crazy enough to etch a number into the firing pin of the bullet, and he said “Yeah, he’s Deadpool.” Enough said.
How did you create the bloody elements?
The blood was a mixture of Houdini fluid simulations and practical blood bag elements shot on green.
Did you share assets with Digital Domain?
Digital Domain built the hero Deadpool asset for the film, and provided us with his final model and textures. From there, we incorporated him into our pipeline, rigged him for animation and cloth sim, and did the look development.
We provided Digital Domain with our CG Environment for the Georgia Viaduct location, which they needed for the sequence immediately following ours.
What was the main challenge for you and your team?
I would say the schedule was the primary challenge, since it was fairly compressed and the release date required us to work through the holidays. Regardless, our crew (with the support of their families) powered through and delivered amazing work – I’m extremely proud of what we accomplished.
Was there a shot or a sequence that prevented you from sleep?
Thankfully, I rarely get so stressed out that I can’t sleep, but I was certainly thinking about DEADPOOL every moment that I was awake. A lot of ideas, inspiration or strategies come from a bit of distance to the work, so I would often think about the project before heading to bed, or while having breakfast, and would come to work saying “I have an idea!”
What do you keep from this experience?
DEADPOOL was one of the most fun projects I’ve ever been involved in, and it was great to work with so many amazing people, old friends and new. I’m so thankful for the opportunity!
How long have you worked on this show?
In total, we worked on the project for about 9 months.
How many shots have you done?
247.
What was the size of your team?
Around 130, including artists and support staff.
What is your next project?
We’re currently in production on STAR TREK BEYOND and PIRATES OF THE CARIBBEAN: DEAD MAN TELL NO TALES. We’re also gearing up on Mr. Zemeckis’ next film!
A big thanks for your time.
// WANT TO KNOW MORE?
– Atomic Fiction: Official website of Atomic Fiction.
© Vincent Frei – The Art of VFX – 2016
It was interesting to read, congrats to the team!