Robert Sethi worked for over 10 years in VFX. He worked at Double Negative on films such as THE CHRONICLES OF RIDDICK, BATMAN BEGINS or STRANGER THAN FICTION. He then joined The Mill and work on many projects.
What is your background?
I have worked in the industry for around 14 years at a handful of different companies; they range from small commercial shops and shops that produce episodic TV and feature films. My role has varied over the years, but my background has always been CG generalist, versatile in many different roles including lighting, compositing, pipeline work, FX simulation, layout, rigging and animation. For the last eight years I have been heading up and managing 3D departments as well as working as a VFX Supervisor and Creative Director. I have been with The Mill for the last 5 years.
How was your collaboration with director James Mangold?
It was great to work with Jim! He is a great character director, but he also has a fantastic cinematic eye and definitely understands VFX. He was very keen on trying to get everything he could practically in camera and to not overload the very short post schedule. Jim was very collaborative and working with him and 72andSunny was a very dynamic relationship where everyone was open to new ideas.
Jim also brought along Phil Brennan as his production side VFX Supervisor and we collaborated together and planned out the shoot. Jim and Phil have a great relationship and it made the whole process very smooth.
What was his approach about the visual effects?
Jim wanted to ground every shot in reality and he shot as much as he could. As always there are physical and budget limitations that impact what can be achieved practically and that’s where our work came in. Jim laid out a great strategy for all the VFX work with the live action shoot. If it was possible we always tried to shoot quarter loads, have practical explosions and squibs. The spot is about the four buddies having a good time and the focus was to find strong character moments mixed in with all the action. This was always the main objective, when you only have 90 seconds to tell a story you never want the VFX to take away any of the main focal point, the characters.
What was your role on this project?
On this project I was Creative Director for The Mill. I planned out the project together with my producer and lead artists, created concepts, supervised the work, pushed the creative and made sure we executed and developed both the agency’s and Jim’s vision.
Can you describe to us your daily work on-set and then on post?
The project was executed in three phases. First when the project got awarded we started working with Jim and the agency to plan out the shoot and at the same time we started to break down the script, refine the pipeline, build assets and start the previs. When you start working on assets this early you have to be able to adapt quickly, because at this stage the script keeps developing and changing. At times we were reevaluating what the team was working on twice a day. It was during this phase we built the CG Caracas, some of the space assets and pre cached a lot of explosions and smoke elements. For the Caracas sequence it wasn’t known how many shots we would have at this time, but in the end it turned out to be one of the bigger sequences and we were very happy we had prebuilt the environment.
During the shoot we advised on VFX methodology for shots and collected measurements of sets and props, LIDAR scanned the more VFX-heavy locations, scanned the actors and hero props, shot grey and chrome sphere, Macbeth charts and of course HDRIs.
As soon as we had finished the shoot I was back in the office working with the team temping out CG to help the edit, and also to focus work on the areas we now knew more about. Once we got the edit and finalized the conform it was full steam ahead. We were running tracking, animation and lighting at the same time, using our assets and scene building system to continuously update the scenes.
Can you tell us more about the previz process?
We previs’d out the space sequence and part of the Caracas scene leading into it. The previs got started based on storyboards from Jim and then we developed them further together with Jim and 72andSunny. We started with a much longer sequence with a lot more action. As soon as it started to flow, 72andSunny started cutting it into a boardomatic. The key was to get in a few choice moments of the hero characters realizing they were in space, and include a bit of action with a gun fight in space; in tandem with transitioning shots in and out of the sequence. As soon as the previs was locked and we knew what shots we were going to shoot live action, we started to transition the previs into animation.
Halon also previs’d a few of the other sequences to help the team figure out the timings of the different sequences.
The trailer features many environments. How did you design these locations?
All of the basic designs originate from the Call of Duty game. The guys at Infinity Ward had done a great job laying down the different styles of the locations. We used the games as a starting point and early on started to create concepts for each environment. We wanted to keep the essence of what the game looked and felt like, but evolve it more in realm of the world that Jim was shooting. We studied a lot of different environments. For Vegas we were going for a war torn and abandoned look, so we used a lot of images from war zones as well as abandoned cities like Chernobyl and parts of Detroit. Caracas was heavily based on game references, but also Dubai, Singapore and of course the real Caracas. For the arctic sequence we had a lot of reference photography in our library from a previous project that had been shot on Iceland which really helped.
Can you tell us more about the environments creation and the shooting?
The opening shot – Welcome to Las Vegas:
This shot was shot on an empty parking lot at Burbank Airport. We had minimal set dressing so we pretty much just kept the car and some of the sun and sky. The shot was done in a layered approach where we created a layout in CG for the environment based on our early concepts. Most of the close to mid environment is rendered in CG, some of it was matte painted on top of, and the beginning of the shot and far distance in the end is just matte painting. We added a few animated objects to bring the shot to life a little more: in the opening of the shot there is a little lizard running by, some tumble weed blowing by and some simulated smoke plumes.
Outside the Casino:
The location for the outside of the Casino was a fantastic set built on the Warner Bros. back lot. Todd, the production designer, had done a great job. We used this as a starting point and extended the buildings further than what the lot would let us do and what could practically be built. For the shots where we are looking at the casino bank (the building with the Romanesque columns) we kept the fore and mid ground set dressing but replaced all buildings. When we looked at the direction of the actual hotel we tried to keep as much as we could of the set, but there was a big tree covering up much of the set and all of that needed to get re-created. And of course, we extended the hotel upwards. During the shoot we LIDAR scanned the sets and that gave us a great starting point for the set extensions.
The interior Casino:
This was shot at a hotel lobby dressed as a casino and the set looked great so we didn’t really need to do any digital set extension for this sequence. But we did end up adding a lot of extra damage to all the walls and floors in post. All of this work was done in Nuke as 2.5D projections.
Caracas:
Caracas was shot on a sound stage and we had a set built of the roof they are standing on. Tom Graham, our environments lead for this sequence, had created a CG layout based on 14 high res buildings and a bunch of background buildings based on game assets and our concepts. The setup was customized per shot to create a nice composition together with the actors’ framing. The high res buildings had enough details to hold up for close ups, but what really helped is the window shader that uses cubic HDRi maps of rooms to simulate the parallax of the interiors together with some clever tricks to make the window glass look good. A lot of attention was placed in finding a good balance between the lit interiors and good building lighting. We wanted a nice textured city with a futuristic feel, but without that style becoming the focal point. In the game, the Caracas sequence takes place during Federation Day, hence all the fireworks in the sky. This gave us an opportunity to let the city become a little more alive and gave a few of Megan Fox’s shots a little more sparkle… if that is even possible.
For the section where the heroes are rappelling down the building we had a practical section of the wall built on set. The wall could pivot and we shot the sequence with the wall in various angles to simulate the rappel but also enabled the actors to perform. This was great for the interaction of the wall. The wall was replaced in post with a vertical digital building that interacted with the rest of the scene and that also had interior offices. This part of the sequence used the same layout of Caracas as the rooftop, but for a few shots we added CG streets with cars and street life.
Arctic:
This sequence was a combination of a blue screen shoot for the close ups and for the wider shots we used the same empty parking lot in Burbank. We used a practical fake snow for the cars interaction with the ground in some areas, but ended up adding a lot more as its a little hard to predict exactly where the car chase takes place. The environments were primarily 2.5D matte pairings projected on simple geo in Nuke.
The end shot:
The two end shots were also shot on the blue screen stage. The end shot started as a concept, transitioned into a CG build to really match the live action lighting and then additional damage was matte painted onto the render.
At a moment, the guys are in space. How did you created the zero gravity effects?
Space was a fun sequence. From day one our look-dev and lighting lead Chris Bayol was all over this sequence. He researched an enormous amount of space material to really try to nail the look and he became our main go-to guy for space.
After the previs had been finished we knew that two of the more complex action shots would be full CG, so we started setting up the look on those two shots first and transitioning the previs into full animation. On the live-action shoot we focused our time trying to capture the moments when the actors realize they are in space, and on the two end shots with the mag toss. The actors here rigged to a see saw setup to let them bob around. We avoided using wires to ensure we could get the performance we were after without restrictions—using actors, not stunt people.
We scanned the space suit and the actors’ faces so we knew we could pull off any action shots we needed. For the Federation soldiers we based our work on assets from Infinity Ward, even though they needed to be up-res’d. And we retextured the game assets so they were really high quality. Some early animation tests with the game assets through our look-dev and lighting pipeline had people think they watched real tests of live-action suits.
To create the zero gravity effect, we used a combination of techniques, all from full CG shots, 2D manipulation and a combination of different live-action takes to CG replacement of legs on the live-action actors. The floating debris and the animation of the mag really helped the live-action parts to blend well with the CG shots where we could allow for more movement.
How did you created the various explosions?
We added a lot of explosions and smoke plumes throughout the spot. One of the bigger ones we did was the exploding drone. It ended up as a combination of CG and 2D elements as it was so quick in the edit. The exploding windows on the building the guys are rappelling down on was a simulated explosion in Houdini. When we shot the actors we used an interactive light to simulate the explosion.
The exploding helicopter was a tricky one. For broadcast reasons the helicopter could not fully explode and it needed to be plausible that the pilots survived. We also didn’t want to make something that was too big and took away from our main heroes’ performances. As the scene was also entirely backlit from the sun we wanted to avoid covering it up too much and changing the mode of the lighting in the scene. We animated the helicopter so that the impact of the missile hit, together with the rotors momentum, pushed the helicopter out to screen left without getting too close to camera. We used a couple of different simulations; one for the main impact, and one for the burning tail, both using the helicopter blades to push velocity into the simulation for added reality and complexity.
The water fountain in the end was one of the trickier simulations we did. We explored different sizes of explosions all from missile explosions to bullet hits. The bigger explosions were impressive, but got a little too far from the classic Vegas routine shots we were using as a reference. We settled for a hybrid of 50 cal rounds hitting water with a little more upward impact.
Simulating water takes time and with a 2 1/2 week schedule it doesn’t give you too many goes. To ensure we hit the deadline we broke down the sim into three different passes: water, spray and mist. We used the heavier water sim to be the base and once that was working, drove the other sims from its movements.
Have you adapted your pipeline for this show?
When you do big jobs like this its always a good opportunity to define how you want the future of your pipeline to function and to push your current pipeline. We tried a lot of new things on this commercial. A few of those were new ways to use cached Houdini simulations rendered out through Maya and Arnold for interactive lighting with the rest of the scenes. Our main pipeline is built around Maya and Arnold but most of the FX work is done in Houdini and Mantra. We also improved our shading pipeline for rendering physically accurate lighting models for explosions.
One of the new things we played with this time around was to do all the VFX work on the raw alex c-log material and to go as far as to have a graded beauty lut per shot. Later on when all the VFX was finished, Adam Scott, our colorist, went back and crafted the grade even more. But it was graded to have a pretty good match during shot production.
Felix Urquiza, our CG supervisor, also pushed our asset-based pipeline really far making sure every department could work on shots and assets at the same time; and easily update and rebuild their shots with newer assets when they where ready.
How did you take advantages of Nuke and Flame?
It was a good mix of both. We used one Flame, one Flare and around 6 Nukes. Chris Knight, who ran the 2D team, used the Flame to run the whole project through and also used the Flame to comp some of the bigger VFX shots; but most of the shots where composited using Nuke. We used a lot of Nuke 3D capabilities to create some of the more matte painted environments and to add in all the extra damage we did on the interior casino sequence.
What was the biggest challenge on this project and how did you achieve it?
The biggest challenge was the production time we had to finish all the shots after we had a conform. It all worked out pretty smoothly even though it was a lot of hard work. What made it work was the strategic prep work we had done upfront, the team and the kick ass leads we had. The commercial is about the camaraderie between four guys going to Vegas to have a good time. We tried to keep that feeling and have fun as we worked the long hours. Also our assets-based CG pipeline helped us manage this amount of shots and assets in the very short time frame. Leighton Greer, our producer, really held it all together and made sure we all had a good time doing it.
Was there a shot or a sequence that prevented you from sleep?
Not really, but when you have 2 1/2 weeks of shot production it is inevitably pretty long hours and not very much sleep at all.
What do you keep from this experience?
It was a great experience working with Jim! He always had the focus on what the essence of each scene was. He also has a great sense of humor, which made every meeting and shoot day pretty exciting. It was an equally great experience as always to work with 72andSunny. The agency always pushes everyone as far as they can for the pursuit of perfection, a passion we both share. But also one of the biggest takeaways from this project was how good and passionate the team is that we have at The Mill. It was really impressive to see how far many of the younger guys in our crew have gotten; may started their careers at The Mill and to see them turn into some of the finest artists I know is tribute to their hard work and really inspiring.
How long have you worked on this film?
The prep time was around 5 weeks, a 7-day shoot and 2 1/2 weeks of shot production after the conform had finished.
How many shots have you done?
The trailer has 63 shots and most are pretty heavy VFX work. Many had digital backgrounds or augmented ones, CG explosions, squib hits, muzzle flashes, tracers, CG vehicles and even some digital doubles.
What was the size of your team?
The size of the team changed during the project’s duration. We started with a handful of the lead artists setting up the pipeline and a few concept artists setting the looks. During its peak during shot production I think we where around 35 CG artists and 8 compositors, 1 producer and 1 coordinator.
What is your next project?
I have a couple of commercials starting up and we are in full swing ramping up for Super Bowl.
What are the four movies that gave you the passion for cinema?
BLADE RUNNER, STAR WARS: A NEW HOPE, TERMINATOR 2 and THE BIG BLUE.
A big thanks for your time.
// CALL OF DUTY – GHOSTS: EPIC NIGHT OUT
// WANT TO KNOW MORE?
– The Mill: Dedicated page about CALL OF DUTY – GHOSTS: EPIC NIGHT OUT on The Mill website.
© Vincent Frei – The Art of VFX – 2013
