In 2017, Alexis Wajsbrot explained the work of Framestore on THOR: RAGNAROK. He is back in the MCU with SPIDER-MAN: FAR FROM HOME.
How did you and Framestore get involved on this show?
Framestore recently worked with Cyndi Ochs (VFX producer) on THOR: RAGNAROK and they got in touch to ask us about our availability to help on SPIDER-MAN: FAR FROM HOME. I went to visit them in Leavesden before the shoot started and when the script was still very much in flux. Cyndi and Janek Sirrs (VFX Supervisor) really believed that the best sequence for Framestore would be the “Illusion Battle” and I agreed – it was definitely a creative and interesting sequence. Janek described the sequence as a horror movie parenthesis where Mysterio would tap into Spidey fears. He showed us the tone he had in mind; a Magritte horse painting, playing with perspective and references of night time environments only lit by a drone, so the light was moving and creating a very interesting effect on the shadows. We started to run some tests of this Magritte effect as well as the drone lighting. We were awarded the sequence of roughly 150 shots.
How was the collaboration with director Jon Watts and Overall VFX Supervisor Janek Sirrs?
It was a very collaborative experience. We started to work on the sequence which Proof had already previs-ed for the most part. The goal was to define the look of the key beats before Christmas, as we knew we would have very little time with the studio in the New Year as Kevin Feige and Victoria Alonso would be fully focused on delivering AVENGERS: ENDGAME. We did some rapid prototyping which was really well received and manage to get a first pass render of quite a few shots within the sequence. During the first test screening our sequence was the favorite! Jon Watts (the Director) and Marvel Studios decided to “plus” it significantly to make it a bit longer and even better. We went back to the drawing board for the whole thing and literally started from scratch with only four months to go!
From there, Framestore drove a lot of the creative process offering ideas / concepts / early animation blocking for the whole sequence. Jon and Janek were both great in giving feedback on our ideas, briefing us, making sure there was always an element of Mysterio in every shot and that the action was emotionally connected to Peter Parker. When we were done with the creative process we only had two and a half months to deliver the sequence!
There are two other examples of how good the collaboration between Marvel Studios and Framestore was, the first one being the way we handled the mocap: We blocked the animation of the whole sequence to a very good level, to nail timings / story / ideas. As soon as the studio was happy they organised a mocap shoot with Tom Holland that they achieved in the same way as an ADR session, delivering mocap per shot while looking at our animation / timing and even the framing. This allowed our animation team, supervised by Nathan McConnel, to just drag and drop the mocap from Tom Holland where needed to elevate the sequence.
The second example would be the way we handled MJ in the Eiffel Tower beat. We blocked the animation ahead of the shoot with a CG version of MJ, pre-lighting the scene, with precise lighting information for the position of the moon and the bright lights from the Eiffel Tower, and rendering it before passing to production. We provided techviz to the studio, for MJ placement, motion, as well as lighting information. This meant that what was shot on-set was the plate we needed – an example of how we, in the VFX department, can help to drive the shoot.
What was your feeling to be part of a Spider-Man movie?
I love Spider-Man and I really enjoyed the Sam Raimi movies and loved HOMECOMING so I was thrilled when we got awarded such a cool sequence on a Spider-Man movie!
What were their expectations and approach about the visual effects?
Even before the sequence was awarded, Janek and Cyndi were clear that they needed a partner to come up with lots of different ideas in an efficient way: “We need it fast, not too polished, we don’t need to make it work within your pipeline, we want ideas, rapid prototypes.” It was all about the creativity, all about how we can make that sequence special. So we treated it quite differently to a normal project where we spend a huge amount of time looking at asset turntables, rigs, then lighting, then going into detail in the comp. Here we worked the VFX like you would do when painting; you start with a pencil to create the shapes, you get some initial feedback, then you move forward with big brush to get the mood in terms of colors and lighting and then you get your second round of feedback. On most of our ideas, that’s where we knew if we had the right direction or if we needed to start again – then of course you start with your smaller brush to get into the very fine details. It was a really interesting process that worked really well for all these one-off shots.
How did you organize the work with your VFX Producer?
I had the chance to work with a very experienced VFX producer, Annette Wullems, at Framestore and she is amazing; she sees the big picture and makes sure that we are on track in not only delivering to the next deadline but the whole project.
We started with ‘Key Shots’ dailies, so we had all of our leads being able to jump in and discuss any shots together. In parallel we had the Framestore Art Department available to use for around 12 weeks (which is huge!), and each time we needed them, whether we had started the CG or not, they jumped in to help us shape the mood, the ideas etc.
From January 2019 until the delivery in June we organised the show a lot more traditionally with department dailies, apart from the special occasions where we really needed the whole team together to solve a challenge.
How did you split the work amongst the Framestore offices?
The schedule was so tight that we kept it all in our London office.
What are the sequences made by Framestore?
We worked on the Berlin sequence also called “the Illusion Battle”.
The Illusion sequence is a really challenging and creative sequence. How did you approach it?
As I said previously, it was all about showing ideas to our clients in the most efficient way possible. I decided very early on to heavily involve the Framestore Art Department, even when we had only 3 months to go and our client wanted to see everything moving. If the brief or the idea was changing, I always returned to the drawing board to start fresh and try to get a target. Some of the concepts were only used internally to give a clear aim to the artists. Then it was all about making sure we only model what we see within the light, lighting the shot asap and judging all the assets in context, using DMP to our advantage. A lot of the shots we had to do were one-off which is always tricky as you don’t have the economies of scale, so we really had to be clever to not overdo or underdo the shots.
What was your range of freedom to submit ideas and transitions?
We were very free, Jon and Janek were very responsive to receiving ideas. Of course most of them did not stick but a lot of our ideas allowed the Director to bounce off of them and shape the sequence. We started by submitting emails with ideas, then lots of movies references, then storyboards, quick concepts, rapid nuke test etc. We actually have a whole showreel of ideas that did not make the final movie which is almost as cool as the sequence itself!
Did you used some of the Framestore experience from the James Bond’s Spectre title sequence?
Interesting that you ask this as it was Fiona Walkinshaw’s (Framestore’s Global Managing Director, Film) first piece of advice to go and check in with the team behind all the Bond title sequences made at Framestore. I had a long meeting with them so they could explain their workflow from ideas to final. There were definitely some similarities in the process but they are a lot more 2D driven, as they go and shoot a lot of elements whereas most of the Spidey Illusion Battle is Full CG. Having said that we definitely used it as a reference for the graveyard shots with Zombie Iron Man.
Can you explain in detail about the creation of Spider-Man and Mysterio?
Regarding Spider-Man, we had to create 3 different suits : the “Homecoming” Suit, the “Home-made” Suit and the new “Stealth” Suit.
We had photogrammetry of the three suits and because we were one of the last vendors to get onboard, we received the asset from the other vendors: I believe Sony Pictures Imageworks and Luma Pictures.
It was still a massive amount of work to ingest it, making sure the topology fit our needs and the textures were compatible with our in-house renderer Freak.
The Homecoming Suit and Stealth Suit lookdev went really smoothly and one of the first shots we delivered was Spider-Man crouching in the Stealth Suit – it must have looked good because the studio believed it was a plate of Spider-Man! The suit that was by far the most challenging was the Spider-Man Homecoming Suit. We built this asset with a lot of confidence as we already delivered it for Avengers: Infinity War. Janek, having delivered SPIDER-MAN: HOMECOMING, had a very precise idea on what the suit should look like, how many wrinkles we should see, the amount of muscle that should show through the suit and especially the way the fabric reacted to harsh lighting, creating subtle sparkles as only the small displaced bits of the fabric would catch the light.
This caused us the most amount of work, we had to entirely redo the lookdev of the suit with one of our most talented lookdev artists, Claire Michaud.
Can you tell us more about their rigging and animation?
Every character at Framestore goes via the same workflow – creating a skeleton, muscle and tension shapes that animators can action. We use our in-house muscle system named Flex and Flesh for skin simulation. Our CG Supervisor Sylvain Degrotte really wanted to push the rig as much as possible, and was actually doing shot sculpts on the muscles rather than the skin to get the best possible skin sim on top.
Every suit needed a very specific eye rig, as Spidey eyes are made of different blades that move like a camera shutter to create different expressions to allow Spidey to emote. Janek wanted to make sure that all the different shapes we hit were creating smooth curves which was quite a challenge for rigging.
Regarding animation, our Animation Supervisor Nathan McConnel reviewed previous Spider-Man movies to make sure we hit the right pose, especially looking at the way Tom Holland moves. We tried to shoot mocap with our own actors but no one managed to really hit the way Tom Holland moves and performs as Spidey.
Can you explain in detail about the creation of the various transitions?
We have quite a few transitions into our sequence, the first one being the B.A.R.F. transition. We had to closely match the CAPTAIN AMERICA: CIVIL WAR effect which was quite a challenge as we needed this effect to work in multiple different situations within very large scale environments, which was very time consuming for our 2D artists, lead by Stephen Kennedy and Mark Gostlow. On the very first shot when we transition from the EUROPOL modern offices to the construction site we also had to make sure that both geographies were matching so we could easily transition from one space to the other.
Going from the Eiffel Tower to the miniature world of Aunt May’s house was also a fun one. We knew that we wanted to play with the fact that in the illusion world Spidey could be at the top of the Eiffel Tower but in real life he is only 1 meter high. During that transition Spider-Man runs after MJ to save her from falling, then falls into a thick layer of green smoke that looks 300meters high only to hit hard a floor painted to look like green smoke.
We then transition to Aunt May’s world into a hyperlapse effect on the background environment, which was really interesting to design; all the lines have a bit of jitter as the camera was shaky to differentiate from the light speed effect used in the STAR WARS films.
Our miniature Spidey is then hit by Mysterio into a New-York skyline landscape only to realise it was just a printed backdrop.
With some transitions we just rendered Spider-Man twice in different lighting conditions, for example when we transition from the glass shards environment to the Valley of the Fallen.
We used the traditional smoke explosion to go from Valley of the Fallen to the graveyard environment where the fist of the Captain America statue falls onto Spidey.
Which transition was the most complicated to create and why?
The most complicated transition was by far the one where we go from a close up Zombie Iron Man’s face to a spider jumping onto the lense before transforming into an army of Mysterios, then entering the cloak’s clasp to find Spider-Man running towards us from a tunnel of green smoke to hit a giant snow globe with buildings appearing out of the ground. Just in describing it you can understand that it was complex to setup and nail all the different ideas and timings. One of the challenges of that transition was to go from the tiny eyes of the spider to the army of Mysterios walking towards camera, as it was such a different scale. The other one was to convey the idea that Spidey was colliding with the glass surface of the snow globe. It was really hard to make sure we read that gag, and had to add a lot of scratches and specular highlights on the surface to nail it.
The sequence is full of FX elements. Can you elaborate about their creation and animation?
The main effect is definitely the green smoke; all kinds of green smoke! We have green smoke in the school hallway that is almost like dry ice, green smoke around the Eiffel Tower that needed a larger scale sim, a more comic book green smoke to allow Mysterio to appear from the moon, green smoke explosions and destructions into the Valley of the Fallen when Spidey and Mysterio are fighting each other, green smoke on the ground into the graveyard and even a tunnel of green smoke (that has been mainly cut from the latest edit) and I almost forget the green smoke within Mysterio’s helmet : it was all handled in Houdini and managed by our FX supervisor Igor Jovanovic. It’s always a challenge to get the right behaviour and the right amount of detail with a smoke simulation, you don’t want to completely kill your render times but at the same time you don’t want your smoke to end up soft.
We also had to generate the FX blast from Mysterio during the fight using a combination of the Doctor Strange triangle eye and the Iron Man blast look.
A Spider-Man movie would not be a Spider-Man movie if we don’t have to generate the spider web, and we had to simulate a lot of them as Spider-Man falls into a dramatic web environment.
We also had to do big destructions on the beat we call “environment switcheroo” where Spider-Man moves back and the environment around him changes to all the different environments he just visited but fully destroyed in an apocalyptic way.
The most challenging effect remains the snow interaction within the snowglobe, as both Spider-Man and the iconic New York buildings pierce through and collide with the snow. It was really hard to define the look of the snow and not have it look too jelly-like or powdery.
How did you create the various environments?
It was all about making them surreal, stylised, simplified and staged like a theatrical piece. We started with the school hallway; asking ourselves ‘what is it that makes a school hallway a school hallway?’ We decided to only keep the lockers, hanging lights, doors and some fire extinguishers. We made the lights flicker to also tap into the horror movie language, and of course added the eerie green smoke on the floor.
We then moved onto the graveyard, and as I said previously we used the James Bond title as reference and also used the plate that was shot in a real graveyard. In the same vein as the school hallway we only kept the gravestone, added some silhouetted dead trees and green smoke.
The Valley of the Fallen environment was one of the most time consuming to build as we had to model all the giant statues of the superheroes that died during AVENGERS: ENDGAME: Iron Man, Captain America, Black Widow and Vision as well as Mysterio. All the statues needed to be on the ground fallen, destroyed, apart from Mysterio who would be standing up. Rocky / stony environments are always hard to achieve in CG as they need a lot of detail to convey the scale and to make sure it looks like rock / stone. Even the ground of the environment is made of debris fallen from the statues.
The Snow globe shot was also a challenge as there was a question of scale – is Spidey tiny and the snow globe regular size? Is Spidey regular size and the snow globe giant but the buildings relatively small? What scale should the snow simulation be? We ended up with a regular size Spider-Man as Jon Watts did not want to repeat the gag of miniature Spider-Man. With the buildings, even if they are only roughly 3 meters tall, they have been modelled, textured and shaded as if they were true to scale, with lights in the window, architectural lights etc. The snow is simulated at a normal scale in comparison to Spidey so around 5m by 5m.
Perhaps surprisingly, one of the environments I am most proud of is the construction site environment because it had to be photo real. During principal photography, they did shoot some shots in a construction site in London at Millennium mills. During the re-structure of the sequence, Jon Watts wanted quite a few more shots within that environment for the audience to have a bit more time to digest that the whole Europol environment was just another illusion. Our lead environment artist, James Harmer, went with his team to shoot a lot of reference over there, and we manage to fully rebuild the asset in CG. The environment is half destroyed, a bit rusty and I feel that we really managed to match the real one. I believe that only a few people will be able to tell that it is a full CG environment.
Which shot was the most challenging?
There are so many one-off shots that they all presented their own challenges and of course the very long shot that starts on Zombie Iron Man’s face and finishing inside the snow globe was one of the most complex to prep.
But I would say the mirror sequence, for a few reasons:
We needed to make sure to have reflections of Spidey in all shards so we had to add at least 5 or 6 other Spidey’s which are only visible in reflection rays, in every shot in different scales to make sure there is always something interesting in the shards. We also rotated each shard in order to get the maximum amount of reflection. We scripted a way for animators to be able to visualise the reflections quickly in their viewport.
We had a lot of Spideys to animate as one of the references we used was the Mr Smith fight scene in THE MATRIX RELOADED (it made sense with Janek being one of the VFX supervisors of the film). We decided to not use a crowd system as they were way too many interactions with our hero Spidey so we just had to animate them by hand using some mocap when we could.
The last challenge was to also find a clever way to get a reflection of our hero which then turns to a ‘baddie’ Spidey coming out of the mirror – we had to use multiple instances of Spidey to create that effect with rules of who is visible in reflections rays.
Finally it was a very, very slow sequence to render with all these assets and reflections!
Is there something specific that gives you some really short nights?
I believe my CG Sup Sylvain Degrotte was a lot more stressed than I was about all the crazy ideas we were bouncing around with Janek. It was an amazing creative ride and I enjoyed every bit of it and I am ready to start again! I also had the chance to work with a team of very talented artists at Framestore in every department so I was very confident that we were going to deliver something special (and on time.)
What is your favorite shot or sequence?
The whole sequence! It’s very hard to choose a specific shot, although I do really like the shot where Spidey falls onto the spider web.
What is your best memory on this show?
I would say seeing the sequence with the final sound mix in the cinema for the premiere. It had been such a creative challenge and then seeing the reaction of the crowd watching the sequence was amazing!
How long have you worked on this show?
Around 9 months, I believed we were the first company to deliver all our finals!
What’s the VFX shots count?
We worked on 170 shots and 140 made the final cut!
What was the size of your team?
We peaked around 120.
What is your next project?
I was actually supervising SPIDER-MAN: FAR FROM HOME as well as WONDER WOMAN 1984 during the past year, so now it’s all about delivering Wonder Woman and hopefully make a sequence as cool and noticeable as the Spidey one.
A big thanks for your time.
WANT TO KNOW MORE?
Framestore: Dedicated page about SPIDER-MAN: FAR FROM HOME on Framestore website.
© Vincent Frei – The Art of VFX – 2019
