In 2013, Mark Bakowski talked to us about the visual effects of Framestore on IRON MAN 3. He then worked on films like GRAVITY and JUPITER ASCENDING. He then joined ILM London to work on AVENGERS: AGE OF ULTRON. SPECTRE is his third James Bond film.
You have already worked on James Bond movies. What was your feeling to be back in this universe?
I’ve enjoyed Bond shows previously they have a distinct ‘it’s got to be grounded in reality’ vibe, although invariably that gets forgotten a bit here and there! This one was working a few steps up in terms of scope and variety of visual effects from any Bond movies I’ve worked on before, it was interesting to see that would play out.
How did ILM get involved on this show?
Steve, Andrew Whitehurst (Steve’s co-supervisor) and Leslie Lerman (VFX producer) asked us to bid on a bunch of different sequences. ILM London was quite a new facility at the time and I think they were interested to see what we could do. We of course were very keen to be on board, so before long it felt more a question of what we’d work on, not if we would. It was a great experience; a little tiring at the end but all in it was most definitely worth it. One thing I find interesting looking back now is to see how the sequences we didn’t end up working on panned out. It’s odd position getting this glimpse at script stage and then being able to see nothing until the work is finished.
How was your collaboration with director Sam Mendes and VFX Supervisor Steve Begg?
I really enjoyed working with Steve and Andrew, they work well with each other – they compliment each other. Steve looks at the big picture and makes sure the boat is steering the right way and avoiding icebergs. Andrew is the details man – checking that everything is shipshape below deck (if this isn’t mixing metaphors too much..?)
Right from the off it felt a very collaborative relationship between all of us, and it needed to be as the demands of the show developed and changed a lot. Some of the work picked up in ILM Vancouver especially was chalk and cheese from bid to final delivery. We had to explore a few dead ends and show them to Sam before we could be sure what the real direction for a shot or a sequence would be.
What was their approach about the visual effects?
They followed quite a traditional formula, so we worked some development shots and assets for them, which they’d show to Sam periodically for feedback, that and some trailers allowed us to progress the look. Meanwhile we pushed the bulk to hit temp deadlines. After temp it was then an upgrade across sequence to push to final. The variables were sometimes it was only when the sequence really started to take form that Sam would hone in on final details, there were a few curve balls as we progressed which made it more ‘exciting’.
What are the sequences done by ILM?
Both ILM’s London and Vancouver studios worked on the film. We did the opening pre-titles Mexico sequence in London, and then we did the villain’s lair inside the giant crater. As a roughly accurate split the exteriors were done at ILM Vancouver and the interiors at ILM London.
Can you describe a typical day on-set and then during the post?
I spent a lot of time on location in Mexico & Morocco and at Pinewood. Mainly with Steve and his hardy band of data rangers. I enjoyed every minute of it, because Bond being Bond they would at least try and do everything for real – even if we then sometimes ended up replacing it all with CG. So they’d fly helicopters very low, blow things up and knock them over down. There was quite a lot of action, and in the inevitable downtimes there was always Craft services to keep it interesting.
Post was quite high tempo, we didn’t have a huge amount of time, and so it was a case of hitting one deadline after another. I’d start the day looking at work from Vancouver and then move onto London after that. Periodically in the late afternoon we’d check in with Steve and Andrew and visit them to talk through feedback. Late in the day I’d check in with Greg Kegel who looked after the Vancouver work as our time zones overlapped.
NDR: Check this video blog to get a good idea of the Mexico shooting:
The movie opens with an impressive continuous shot. How did you approach it?
An incredible amount of visual effects work went into the opening shot which comprised of 6 different plates shot in both Mexico City and at Pinewood Studios and totaled around 6’000 frames. The shot work included large crowd simulations, digital matte environment extensions, prop additions, rig removal and cleanup and digi-doubles.
The first task was to block in the transitions between the different plates. We didn’t use motion-controlled cameras for any of this so the approach involved tracking the camera movement for each plate and generating smoothed blend cameras to re-film the original re-projected plates. Plenty of new content was generated to stitch the disparities between the different plates as the start and end of each plate would be in entirely different locations and often ending or starting with green screens, so interior & exterior walls were modeled and textured in Maya and rendered in Nuke through the smoothed blend cameras to stitch the gaps.
The transitions were defined as early as possible to give us as much time as possible to approach the considerable set extensions and rig removals – at the time of filming the equestrian statue of Carlos IV of Spain in the Plaza Manuel Tolsá was being renovated and was covered in scaffolding so this needed to be removed and the buildings behind re-created. The street was extended to place Zocalo Square in the far distance setting up the continuity for the later chase between Bond and Sciara. CG Crowds were generated to fill the street extension and add a denser population into the existing crowd in the plates, as well as the addition of CG street stalls and shrines to fill empty spaces.
A fair number of extras throughout the sequence had costume enhancements such as facemasks adding or costume changes, which involved facial tracking and digi-double replacement with dynamic cloth simulations to bed the CG characters back into the existing crowd.
Once the camera transitions to within the hotel we had to do significant changes to the architecture such as a full ceiling replacement and rebuilding the door and wall of Estrella’s hotel room to match the door and wall of the Pinewood set used for the next portion of the shot. A lot of work went into this transition that happened over a very short duration. We started with the alignment of Bond and Estrella to match between the different plates and rebuilt the wall and door around them. When the door is opened you’re actually looking from Mexico into a Pinewood set and out through a blue screen behind the open balcony door back into a digital matte painting of the view in Mexico City.
Inside the hotel room there are various props added as set dressing, then when Bond steps outside onto the Balcony the entire hotel room is digital re-created around him, as well as a digital street scene below including CG trees, buildings, stalls and crowd generation. At the time of filming the street below was used as the crew vehicle parking which all needed to be removed and replaced. Whilst Bond walks along the balcony he is wearing a safety harness, which was painted out for the remainder of the shot. Additionally we digitally aged and weathered the garden and the rooftop around him to add fallen leaves and dirt, air-conditioning units and vents, dead plants and additional props to make what was a fairly pristine garden feel like it was abandoned.
The leap Bond does over the gap between the rooftops was filmed as a green-screen patch that was removed and replaced with a sheer wall showing the bustling street scene below. The finale of the shot shows Bond approaching the ‘Fru Fru’ building which was done as a combination of an entirely matte painted building facade with a plate insertion for the room interior, again shot in Pinewood and inserted into the matte painting facade. The shot had a variable re-speed of 106% applied so we had to remove all the retime artifacts which made for quite a lot of late nights. A total of 55 VFX crew were involved to create the final assembly of this nearly 5 minute shot.
How did you recreated the Mexico City environment?
The Mexico City environment was a fully digital asset created by the Generalist department at ILM London. Focused around the famous Zocolo plaza in the heart of Mexico City’s historic center, each individual city block was created through a process that started with an aerial drone shoot to capture the material needed for photogrammetry. The resulting photogrammetry meshes were remodeled and re-UV’d in Maya to provide a clean asset for lookdev and rendering in Arnold. Texture cleanup and lookdev was all handled in Mari in a process that involved removal of the baked in shadows from the drone photography and generation of utility textures & masks for the final shader look development.
The individual city block assets were arranged in a master asset and set dressing such as street lights, trees, stalls, flags, aerials and additional dressing was all created using instances to populate the final scene. Additionally, 360-degree panoramic virtual background stills were taken at various heights from a helicopter flight over the square that were stitched together to create pano-spheres.
The environment was then used by the crowd team to populate the square with thousands of crowd agents who would all react to the out of control helicopter throughout the duration of the sequence.
There is an explosion and building destruction. Can you explain step by step about this big destruction?
The process began with close collaboration with modeling to construct models appropriate for destruction, all the way down to modeling the individual tiles on the front of the collapsing building. R&D developed new tools for volumetric fracturing in Houdini, and to help ensure maximum levels of instancing throughout the destruction pipeline. These components were combined with others to create an efficient; materials based approach to building destruction. For example, a rule could be easily created to ensure all window frames could support bending and breaking behaviors. Custom shaders were created to allow for dynamic fractured edge detail in destruction close ups, along with procedural rebar tools to additional detail.
The combined set of simulation and caching stages for the most complex shots grew very large, so effects leveraged new tools developed by ILM R&D for managing the most complex dependency graphs. This allowed for overnight turnaround on notes that would impact the very earliest stages of simulation preparation, with the result a render featuring the complete set of effects elements.
The action moves to a helicopter with a big fight. How did you enhance this sequence?
We did head replaces and rig removals for the stuntmen who were used for some location shots, augmenting the background by adding thousands of digital crowd members. We also did more complex shots which were fully digital when the helicopter was doing a move too dangerous to pull off over the city – so here we generated the helicopter, environment and crowd, and for some shots a smoke trail coming from the back of the helicopter all as CG. We also did a sequence of shots when the helicopter was shot on a gimble with blue screen backing. Here we had to add the background, the rotors, the helicopter glass and then sometimes do face replacements and rig removals also.
What do you keep from this experience?
Never underestimate a retime! But you live and learn….
How long have you worked on this film?
Just about exactly one year.
How many shots have you done?
313 made it to the movie.
What was the size of your team?
About 140 worked on it at various points, throughout VFX production.
What is your next project?
I’ll be working on Marvel’s DOCTOR STRANGE – It looks mind blowing!
A big thanks for your time.
// WANT TO KNOW MORE?
– ILM: Dedicated page about SPECTRE on ILM website.
© Vincent Frei – The Art of VFX – 2015
