In the past year, Mohsen Mousavi explained the work of Scanline VFX on THE MEG. He is back today to tell us about his work on the final season of GAME OF THRONES.
How did you and Scanline VFX get involved on this show?
Scanline has a reputation for delivering large CG environments combined with complex simulation effects. We were involved in the season finale of season 4 (where we won a VFX Emmy) and since then have been looking for an opportunity to work with Game of Thrones on a larger scale. So when we were approached to work on the final 3 episodes of the final season, it seemed to be a perfect fit for Scanline.
As a VFX Supervisor at Scanline I have been involved in some very environment/effects simulation heavy projects such as SAN ANDREAS, INDEPENDENCE DAY: RESURGENCE just to name a few. Helming the battle of King’s Landing and the final episode seemed like a natural progression of my previous work.
What was your feeling to be part of this iconic TV series?
GAME OF THRONES has raised the bar in TV on every level, from storytelling, to craftsmanship. GAME OF THRONES has managed to establish an amazing, believable and utterly entertaining universe with visuals so memorable that many of us diehard fans will remember for years to come! Being part of such monumental project has been “dream come true for myself and the team”! We had a lot of passionate artists putting their best at work with an unbelievable amount of motivation and energy!
How was the collaboration with VFX Supervisors Joe Bauer, Stefen Fangmeier and VFX Producer Steve Kullback?
Steve Kullback was the Overall VFX Producer and did an amazing work organizing the logistics and the extremely demanding schedule. It takes a lot of patience and experience to plan and schedule such complex project. Steve had a key overview of all assets/tasks dependencies across the board and made sure we were fully prepared to succeed with delivering some of the most complex VFX work to date on the show.
Joe Bauer oversaw our work on episode 5 and 6 and Stefen Fangmeier was responsible for episode 4.
Joe and Stefen are both veterans of GAME OF THRONES and have a long history in the industry. Even though they were responsible for separate episodes, they were fully collaborative when it came to assets and development that were shared across all the episodes. They did an amazing work preparing the shoot with VFX and post in mind. Joe definitely knew King’s Landing inside and out, almost as if he grew up in its streets! The same level of detail went into the dragon, and both were very patient on making sure we have all the information and material needed to finish the work.
What was their approaches and expectations about the VFX?
Every sequence, every shot had to be perfect. We kept telling each other that “the world is literally waiting for this”. We all felt the extra weight which was very unusual compared to anything else I have been involved with. HBO wanted the best and they chose Scanline to help tackle it! Scanline took that challenge very seriously. Successful VFX all starts from planning the shoot long before we start the post production and I must say that Steve, Joe and Stefen did their part to perfection. Every shot was planned with complexity of the VFX and post in mind. They always took the approach of shooting as much as possible of practical effects and environments. They shot a tremendous number of effects elements for each set up, which gave us an amazing foundation to build on top of. When you work on a plate that already looks great in every aspect, you are pushed to bring the VFX to the same level.
How did you organize the work with your VFX Producer?
Marcus Goodwin was the VFX Producer on the Scanline side. He is a machine with a great knowledge and experience on how to approach scheduling and planning large quantities of shots, assets under a tight delivery schedule. He worked directly with Steve on a daily basis with constant back and forth on many different levels and did his best to manage expectations both with the client and internally. Marcus took care of all the schedules and logistical planning along with Josephine Dwyer and Keya Mukherjee (Digital Production Managers), and together they spear headed the production crew at Scanline including all production and department managers so I could fully focus on the creative and technical aspect. Marcus was always present in most of our internal reviews and with an birds eye view of the entire process, made sure that we delivered the best work on time and within budget! VFX Producer is one of the main pillars of any successful project and Marcus delivered above and beyond!
What’s the VFX shots count?
In total we delivered 615 shots scattered over the final 3 episodes.
How did you split the work amongst the Scanline VFX offices?
Vancouver was the lead facility (where Myself and Marcus are based) and we got additional help from our offices in Germany/Munich, LA and Montreal. All Scanline facilities work in the same pipeline and network regardless of their time zone. It is an amazing infrastructure which allows us to easily distribute the work without dealing with any additional logistics surrounding syncing or uploading of materials to one another.
Munich worked on 150 shots from the “Mountain and Hound fight” aka “Clegane bowl” sequence in ep05. They collaborated with us by focusing only on the backend departments of FX, lighting and compositing. Our offices in LA and Montreal helped round off the rest of the team by tapping into specific artists expertise on any given shot or effect across the board.
What are the sequences made by Scanline VFX on this final season?
In ep04 we were responsible for the negotiation sequence in front of the King’s Landing gate including the Missandei’s death.
In ep05 we did a large variety of sub sequences including:
Attack on the scorpions,
Destruction of the gate
Destruction of the city
Destruction of Red Keep,
“Mountain and hound fight” as well as “Jamie and Euron fight”
…
In ep06 we were responsible for :
The entire Dany’s speech sequence with full CG, 360 environment of the damaged KL and Red Keep.
The sequence of Jon Snow walking towards the throne room and his confrontation with a sleeping Drogon.
Throne melt sequence inside the throne room and the final sequence of the dragon carrying Danny out of the throne room and to the unknown!
Can you explain in detail about the creation of King’s Landing and the Red Keep?
Historically, King’s Landing was shot on location and referenced on Dubrovnik, Croatia. The complexity of ep05 and ep06 required Scanline to build a full CG version of the historic city. The battle of King’s Landing unfolds on a variety of locations across the city. Scanline needed to make sure that the environment is covering all the angles and is fully consistent from shot to shot, which was essential to the flow of the story. The digital city needed to be built in a way that could technically undergo various different stages of destruction throughout the 2 episodes, as well as be covered in a thin layer of snow.
HBO’s Joe Bauer and Steve Kullback scanned most of the historical section of Dubrovnik using photogrammetry techniques. Of course, you are limited on how far you can go with capturing detail at this scale, but it certainly gave us a head start to base our digital environment on. We had to be as efficient as possible with our building methodology. Our CG supervisor, Ryo Sakaguchi developed a semi-automatic process of converting all the high res photogrammetry assets into managed, optimized and compatible assets within our modeling and texturing pipeline. However, the photogrammetry assets only covered a part of King’s Landing environment. At its core, King’s Landing was based on modular building assets similar to Lego building blocks. We took the technology we developed years ago for the creation of the CG damaged San Francisco in SAN ANDREAS and pushed it to the next level for the season 8 of Game of Thrones.
We were able to build many intact and damaged assets fast and efficient, which on one hand were in sync with the Dubrovnik style of architecture, but at the same time were engineered carefully to work in our simulation pipeline. We were able to generate multiple variation of damaged building from the same intact assets using the same module base workflow. Every damage building used a collection of instances of predefined small modules from the database, which meant we were able to load every module only once at render time even though it was used 100 of thousands times across the city.
For the intact city part of Kings Landing, we used Clarisse Software to layout the highly detailed photogrammetry and combined that with our in-house layout manifest system. For the damaged part, we relied on our own implementation of city engine which ties tightly into our modular system mentioned above.
How did you create the various shaders and textures for this environment?
We took the same approach on the textures and shaders and came up with a procedural pipeline/workflow which allowed us to texture 100s of unique intact and damaged building in a fraction of time. All the building blocks and modules had consistent UVs, and various different predefined metadata structure which made it possible to modularize our textures as well. We had a combination of different façade textures modules that could be combined on multiple layers using procedural masks and get the desired aged look on the building. The same methodology was used on broken buildings with layers of burnt façade mixed in.
How did you handle the lighting challenges?
For plate-based shots, production provided standard elements of grey ball, HDRI which became the foundation of our shot lighting. You always want to have a consistent lighting model in the cg renders wherever possible.
Episode 6 was mainly shot in an indoor stage with controlled lighting setups. However, you still have slight differences in the plates due to the nature of different lenses and different cameras with slight differences in their color temperature. Our lighting model and textures are based on a neutralized setup. All the provided material such as reference balls and HDRIs are neutralized consistently in order to use a unified lighting/rendering and compositing workflow. We neutralize all the plates based on the provided color charts and record the grading offset in our database per plate, per shot. At this stage we are able to use the same lighting and compositing template as a starting point in the sequence across all the shots. This way we have consistent plate and consistent digital extension. This process is fully non-destructive and we are able to invert the entire neutral grade using the pre-recorded offset at any point on the comp, including the CG additions which brings us back to the original plate lighting for sending it back to the client.
How did you approach the melting of the throne?
The Throne melt was something we started early on while Joe and Steve were still shooting on set. It was such an iconic moment, and we all were very excited to be in charge of this difficult FX work! The throne melt simulation was developed based on a mixture of Houdini granular solver as well as our in-house flowline framework. Our FX lead, Marcel Kern, used some very clever methods in laying out the effect based on some predefined curves that guided streams of viscous melt. This allowed us to cater the edit and the timing in a fully controlled fashion and still be able to drive our simulation based on physical attributes of the fluid which gave us the best of both worlds!
Joe, wanted us to deliver the most complex and realistic melting simulation ever seen on the show. It needed to be dramatic, spectacular and at the same time resemble a realistic look you would expect from such event. Resolution and fine flow detail were the key to achieving this look. We came up with some very advanced and clever shaders to get insane amount of details on top of the already high res simulations. We were able to provide dozen of different looks for the shader in one go, so we could narrow down the final look with Joe before kicking off the hero renders. The glowing lava and the underlying fire simulation mimicked the practical fire elements, this would then seamlessly light the surrounding environment, which gave an extra level of fidelity and realism to the shots.
Can you elaborate about the various destructions and especially the Red Keep?
We received the Red Keep assets from the previous season as a base, and redesigned the entire courtyard and the main plaza in front of the castle. We needed to rework most of the assets to make it technically sound for various destructions scenarios. We started very early on using the blocking animation from Image Engine which gave us a clear indication of the timing and the layout of when and where the dragon fire would hit each part of the Red Keep. This was represented consistently with a cone across all our reviews with Joe and Image Engine. The tricky part was to make the fire extremely forceful but staying away from the wrecking ball look. We represented the fire as an extreme passive force which instantly pushes everything out of the way and vaporized them to fire and ashes.
Another example was the destruction of the King’s Landing walls and the scorpions (Giant Crossbow like weapons meant solely to kill Dragons). Joe wanted to make sure everyone was aware how powerful the Dragon’s fire is, and that it could easily destroy wood and stone structures. We sent a lot of different versions and the feedback was always “it needs to be more forceful”. We cranked up the effect of the fire force and made sure it was consistent across the whole sequence.
For the scene when the dragon blows up the King’s Landing gate, onset production built a third scale miniature of the gate and blew it up with practical effects, but unfortunately it did not have a big enough fireball to tell the story. They were able to shoot this from five different angles, it looked great, but they wanted more of a fiery event . So they gave Scanline the challenge. It turned out to be one of the most complex digital pyro-shots of the series for us. Technically the resolution and the details of the simulation was as high as we could crank it. We came up with one simulation that could be used in all the angles consistently, similar to the original idea of the third scale miniature. Our simulation mimicked the original miniature effect in terms of timing and force plus a lot more on top, which helped us mix back here and there some aspect of the practical miniature.
In the Clegane bowl sequence, Scanline Munich was tasked to feature a collapsing tower around the Hound and the Mountain over the course of 150 shots with a heavily art directed build up towards the final murder/suicide jump out of the window. The tricky part was the continuity from shot to shot, as well as making sure we sandwiched the characters inside the falling dust and debris without covering them too much.
Did you develop specific tools and procedural process for the destructions?
We are constantly improving our core toolsets and methodologies. GAME OF THRONES was all about efficiency without cutting corners on the quality. One example was some very advance additions to our state of the art fluid framework “FLOWLINE” which pushed our pyro explosion workflow to the next level and allowed us to simulate insane choreographed fiery explosions with an enormous level of detail.
What was the main challenges with these destructions?
Joe was very methodical when it came to simulation layout and wanted to make sure everything is physically correct and is not cheated for a specific angle. So it was a core part of our workflow to deliver a variety of witness cameras for all our simulations. These witness cameras were designed very carefully to also feature the drama of the effect as good as the master camera. The interesting part was that some of the witness cameras were so successful that they became actual new shots and made it into the final cut!
We had to adjust a lot in our pipeline and methodology to be able to deliver a lot of iterations in a short amount of time to meet our deadline. Everything was developed based on practical references and we were constantly pushed to make sure it is not only cool, but it is based on what is out there in the real world!
How did you work with the other vendors for the Dragon?
Image Engine was the main vendor for delivering the dragon animation for Scanline’s portion of the Season. In a nutshell, they delivered the animation in the form of raw alembic caches and Scanline did the final rendering and compositing the dragon into the shots. The dragon asset in GoT is something that has evolved over several years. Over the past 7 Seasons various different vendors including Pixomondo, Image Engine and R&H contributed to the development of the dragon assets. Scanline did handle some of our own Drogon Animation as well in Episode 4 and 5, which required us to fully rig Drogon from scratch all the way to fine muscle sims using ZIVA Software.
Very early on, we received a dragon lookdev package from the client which included the model and various different generations of the textures that were established over years as well as animation caches, and final composite of some key shots from Season 7. We needed to match the final look of the dragon as close as possible to what was already established and at the same time make sure the dragon assets is compatible with our pipeline. Joe was acutely aware of every aspect of the dragon and knew its features in and out. We spent a great deal of analyzing the look and rebuilt the shaders from scratch for a better integration into our pipeline.
Can you tell us more about the Dragon’s fire?
It has been an ongoing tradition in game of thrones to shoot the dragon fire practically and Season 8 was no exception. The credit goes to the client VFX and onset Special Effects teams, who made sure shots were grounded on practical elements whenever possible. This of course pushed us to match all our effects were seamless to make sure the audience don’t see a sudden jump form a shot with practical elements to a shot with digital elements.
Image Engine provided blocking animation of the dragon which was locked at a very early stage. The dragon fire was based on a mounted flame thrower on a motion controlled Spyder rig which was driven based on an accurate techviz driven from blocking animation provided by Image Engine.
In EP06, Joe took the dragon fire process to the next level and incorporated a third scale partial build of the throne set for the fire to interact with. For every shot, they were able to shoot multiple takes thanks to the motion control setup which gave us a wide range of perfectly synchronized elements to combine for a more complex looking effect. On one end the flamethrower represented accurately the pre animated mouth of the dragon and on the other side we had fire interaction with the third scale partial throne set which became the foundation of our fire extension. Regardless of where we used the practical fire elements and the level of digital extension, we simulated digital fire mimicking the timing and the dynamics of the practical element and used it to light the shot.
How did you create the various armies?
We created full CG digital crowd of Unsullied, Dothrakis on horses, Lannister’s soldiers, Golden Company and scorpion’s operators as well as variety of King’s Landing civilians. Assets were built base on very high res Lidar scans and were textured and lookdev in our pipeline. We recorded a variety of motions for different factions in our motion capture studio and fed them through our established massive pipeline. We spent a lot of time on the assets build on every level and made sure that they held up for close up shots. In some scenes we were placing our digital characters right next to the filmed extras with seamless integration.
Which sequence or shot was the most complicated to create and why?
We had a number of challenges that needed extensive development. King’s Landing full 3D environment as well as the melting of the throne were what we started first and finished towards the end of the project.
Is there something specific that gives you some really short nights?
Working on such an iconic sequence of once of the most anticipated TV events in recent history gives you short nights regardless of how much you have everything under control. Expectations were above and beyond and there was absolutely no room for any short comings. The energy of everyone involved eased the process and made it a highly memorable experience!
What is your favorite shot or sequence?
In ep06, when Jon Snow walks towards the throne room, he gets confronted with the dragon which is buried under a pile of snow. I love the composition and how those shots turned out. We spend a long time to get the mood right. Very dramatic moments!
What is your best memory on this show?
Working on such iconic sequences of the last 3 episodes of the final season gives you the chance to work with the best of the best! It all felt like a big family with a unified goal and I will be forever thankful for their amazing dedication and passion. I remember when we saw the first cut of Dany’s death sequence in ep06, even though it was all filled with greenscreen, and we did not have any audio, some of the crew got very emotional and literally had tears! Powerful images indeed!
How long have you worked on this show?
We started with initial RnD in March 2018 and delivered our last shots in April 2019.
What was the size of your team?
Over 250 artists in Scanline across all facilities worked on GAME OF THRONES. Everyone in the facility wanted to be part of it!
What is your next project?
Scanline is always busy doing multiple projects at the same time. We have a wide range of projects in line for the following months. Unfortunately, due to confidentiality I can’t yet speak about the next project.
A big thanks for your time.
WANT TO KNOW MORE?
Scanline VFX: Official website of Scanline VFX.
© Vincent Frei – The Art of VFX – 2019
