Last year, Sven Martin had explained to us in detail about the work of Pixomondo on BRIDGE OF SPIES. He talks today about the dragons for the season 6 of GAME OF THRONES. Be sure to watch the new making of about their work at the end of the interview!
What was your feeling to be back in Game of Thrones universe?
Being in our fifth year now, we actually never left Westeros since Season II.
We´re usually discussing about new ideas for the upcoming season while still working hard on the final touches of the current one. It’s a great honor for us being part of this extraordinary and epic series and I get always a smile on my face seeing the team being thrilled and exited with every next season rolling in.
How was this new collaboration with VFX Supervisor Joe Bauer and VFX Producer Steve Kullback?
I guess we’ve never work for such a long time within the same constellation before. It is a very unique experience in our current vfx business and I’m very thankful for it. What Steve and Joe manage to bring on screen is a marvelous piece of visual effects, meeting the highest expectations and still raising the bar with every new season. What I love and appreciate very much is getting involved from the early stages, long before the first script pages are written.
It’s a relationship build on honesty, trust, quality and friendship making us feel part of the family. Working together for such a long time has the advantage of an easier understanding and support, giving us the chance to play with ideas and testing concepts before going on set. A lot of work is done before the actual shooting starts – work you never see, but influences the final images on screen.
What was their approaches and expectations about the visual effects and the dragons?
What I like very much about their approach is always merging digital elements with photographed plates, avoiding calling out shots easily as full CG. Even shots looking like 100% CG have always some real elements in it.
A big sequence this year was the encounter of Tyrion and the two starving dragons Rhaegal and Viserion. The look and feel of the emaciated bodies was very important for Joe and we spent a long time defining a new sculpt revealing their inner skeleton structure and a new skin shader showing their delicate health. The animation of this scene was also very challenging. We had to balance their weakness and vulnerability along with their power to threaten and kill Tyrion. It was a thin line not falling too much in one or the other behavior. Especially if you only have their faces and body language to express. Animating the dragons is very much acting like a pantomime, but without going into an exaggerated play.
How did you organized the work amongst the various offices of Pixomondo?
The only season where we split the work between all offices was season II. Season III was mainly London, Stuttgart and Frankfurt. Since three years our Frankfurt office is the only team working on the show and the artists have become real experts in creating dragons. I enjoy very much how the artists take responsibility about their work, exploring new ways of pushing the detail level every year. It’s a well-rehearsed team and it is great to see how they make Drogon, Rhaegal and Viserion grow.
The Dragons have evolved a lot through the seasons. How did you approach them on this season?
We always start with Drogon first and then adjust from here the smaller brothers in shape and texture.
First step is always defining the new size and adjusting the changing proportions. Sculpting designer Dan Katcher has a very good understanding of anatomy in the different ages and does his first drafts with the previous year model in zBrush.
For the emaciated dragons we thought we could go straight from the final Drogon model to a starving version via blendshape adjustment, but had to find out this would not be sufficient enough. Our start was a very rough image Dan had created during the design phase and Joe liked the idea of a nearly exoskeleton look and translucent skin.
The emaciated dragons required a complete rework of the model and our sculptor Anthony Sieben created a super skinny model exposing skeleton, muscles and veins. Spreading these elements on different layers gave us full control over their visibility. A big task was the creation of the skin shader. We explored various ideas from a glibbery face-hugger skin to a dry and nearly transparent version showing fine veins and arteries.
Can you explain their creation step by step?
After the sculpt was completed, rigging started to work on a new simulation rig taking care of the underlying structure, pushing bones and tendons through the contracted but also lose hanging skin. The look dev team around Sebastian Lauer started parallel defining the look and textures of the skin. We rendered the normal skin version as well as the emaciated shader with multiple AOVs for the comp department, where compositing lead Travis Nobles defined the final look with a special setup, giving easy control to all aspects of the shading.
How did you enhanced your assets from the previous season?
Enhancing the assets means for us especially increasing the model and texture detail, reducing the render time and increasing the speed and control of the animation rig.
With your experience on the dragons in the previous seasons, did you change something in you work methodology?
Getting closer and closer to the dragons every year makes us integrate more and more extra geo into the base mesh. We try to avoid this to keep the polycount of the base mesh low and fast. But having the camera sometimes only centimetres always from frills and skin makes it inevitable to merge the geometries into one.
Can you tell us more about the textures and shaders work?
The textures are based on real animals, predominantly lizards and crocodiles. At the beginning we prepared a huge collection of photo references and together with Joe’s favorites, this was the base selection to pick of. We use Mari for texturing and build the skin from bottom to top with mattes to mix the layers. We carefully spent time on the UV layout to get sufficient detail in all areas close to camera. Our dragon model currently consists of 160 UDIMS and about 60 different types of textures handed over to shading. Lead lighting artist Sven Skoczylas sets up the shading tree and adjusts the look in a studio environment as well as in a special location representing one of the key shots. The shading tree is pretty complex meanwhile and contains lots of connections to the rig publish for animation driven effects.
With season VI we switched from Vray to Arnold for rendering. This was an important step for us, as we got to the limits of Vray in terms of render time and memory usage. Pixomondo is a Vray house by tradition and it’s still a powerful renderer for all our hard surface projects. But the amount of textures, heavy use of displacement and subsurface scattering brought us to render times I couldn’t accept. The move was pretty easy and we had converted the shader in about a day. Especially the strong and robust realtime preview is a great tool to spend time on creative adjustments in an interactive environment.
Can you explain in details about the rigging?
The rig gets build up from scratch every year, but of course with all the experience of the previous assets. We don’t use any autorigger for our dragons, but try to keep the handles and functionality consistent over the years and also between our different projects. This way our animators can skip the learning phase and can jump right into the fun part. Because of the dragon’s enormous size – and therefore increased amount of control objects – we did group and hide different levels of control within the rig.
The tricky thing is to be setup for wide shots, taking all major body parts in action, while also giving enough control over close up areas, fine animating a single frill or spike. The simulation is rig get’s fed by the animation rig and contains all hair, muscle and cloth simulations. It doesn’t look like it, but the most difficult thing is still the setup and skinning of the wing membranes. From a super condensed space between the fingers to full stretching while flying, the cloth simulation and the transfer from sim geo to hires geo has to survive all extremes.
The face and mouth required a lot of work. Can you tell us more about it?
The inner mouth creation is pivotal for us in terms of believability. It references a combination of snake and wolf with plenty of detail for full close up shots. Irregular shapes, patterned skin and a glibbery shading make sure it doesn’t get to clean. Adjustable blood layers add a final disgusting touch.
The face has a very high detail sculpting and texturing with nose and eye area often close to camera. The facial performance is created as combination of blendshapes and control objects. This way we can add a very individual performance without being stuck too much in predefined shapes. It’s the responsibility and art of the animator then to make the face alive enough without taking the path of humanization.
As every dragon, they can spit fire. Can you explain in details about this aspect?
The biggest design change happened from season II to III, where the idea of magical dragon fire was dropped and a realistic approach was chosen. The fire itself is always based on flame thrower footage shot by production and enhanced with CG fire where needed. This made us design two glands at the each side of the mouth ejecting burning liquid, ignited by super hot air coming out of the throat. The visual fire cone builds short in front of the mouth, starting with a cool color fading into the yellow, red fire blast. That’s a least the technical and anatomical idea behind it.
How did you help the cast and crew to work with the dragons on-set?
With Dany climbing on Drogon for the first time in Season V, Joe wanted to explore even more advanced ways for a plausible interaction. Making an actor believably ride, climb or touch a digital creature still needs a lot of clever approaches and technical effort. To make Emilia Clark move realistically with Drogon’s movements, we first developed a buck element representing the original shape of the dragon’s back. We pre-animated Dany and Drogon and could define a two-piece buck element which could be individually controlled by a motion base rig. It was a close collaboration under Joe’s guidance between ‘The Third Floor’, the motion base guys, the construction department and us.
Additionally, a tricky thing shooting for Game of Thrones is filming outside on location. Many of the beautiful locations have very strict regulations and a highly technical motion base rig is nothing you can easily use outside anyway. So making Dany ride and climb on the dragon is a clever combination of sound stage Motion Base greenscreen footage with a hand-controlled buck rig on location.
The dragons are now really big. How does that affect their animation?
With nearly doubling the size every year, we have to adjust their visual speed accordingly. Especially during the first pre-animations we play a lot with various timings and very often Joe sends back speed refs from editorial, where they adjusted the animation by 80 or 90 percent for more scale – sometimes it went up and down until we had found the right tone for the current season. Addressing 2D retimes in 3D scene can sometimes be very tricky. Animator Thomas Kutschera often used a retimed alembic cache as reference before changing the bunch of animation curves accordingly. This way we could quickly see if the framing would change to drastically, as the plate camera is still moving in original speed. Not making it look like a slow motion version of a realtime animation was very important to me. Even if the larger body parts move slowly through screen space, there are always elements still moving in ‘real’ speed – a flick of the tail, a twitch of a frill – this combination makes it plausible for the audience.
A very evident problem with their size was making them act in the dungeons. There is not very much space between the pillars and making the two dragons sit side-by-side, turning and spreading wings was a real challenge. Sometimes the animators cheated in their first version, letting parts of the dragons stick into the pillars, but I forced them to find a pose getting around the obstacles. The idea was to make the environment affect the animation as it would happen in reality. I wanted to make them look and feel grown up in chains, in a room gotten too small for their size.
Can you tell us more about the wings challenge when they are flying?
From an animation point of view, we explored various ways for the flight cycle over the years. It always based on similar reference footage, mainly bats and eagles, but got adjusted to size and specific actions. Giving enough power into the flaps carrying the weight was very important for Joe, especially when it comes to starting and landing. Finding the right balance between gliding and strong flaps should make it interesting and believable.
From a technical side it’s less challenging while they are flying, it got hard within the dungeons with limited space and folded wings. The wings are driven first by the animation, second by a hair simulation adjusting the fingers with gravity and friction and third by the wind blowing into the membranes.
One of the final shots of the season is a beautiful long shot. Can you explain in details about your work on it?
This shot has been a divided between two companies, with Mackevision working on the ships and ocean and us on the dragons. We discussed how to exchange all necessary information and Joe decided to let the dragons drive the camera. Based on the previs, developed by The Third Floor in close collaboration with Joe, lead animator Jonathan Symmonds started to block out the dragons along a path. Finding the right timing of all key moments and a natural play between the three dragons was crucial before moving forward. The camera itself also played a big role in the perception of the shot, falling back and getting back close to the action where needed. The scene was setup in real scale, so the travel path was pretty long and the speed quite high. Small changes on the path had an enormous impact on the framing, so a constant adjustment of the camera was needed. We kept camera and dragons independently for purpose and did not constrain them together. At all cost I wanted to prevent it from looking like a computer game, goal was to make it look like a camera man following the dragon, with all its natural irregularities.
Compositing lead Mark Spindler combined the background elements with our dragon renders and created the final look adding additional haze and fog in 3D space. For the water interaction he combined filmed water splashes with a digital water simulation done in Houdini.
Was there a shot or a sequence that prevented you from sleep?
Definitely Tyrions scene with the emaciated dragons in the dungeons. It was the first sequence we started to work on with the majority of the shots. With all it’s subtleties in performance it was a real challenge and kept animators busy till the last moment. At the same time, the look of the dragons had to be adjusted from shot to shot, as the skin shader was very sensible on the lighting conditions. Joe wanted a specific look, waxy and translucent, but the only available light was the torch in Tyrion’s hand, illuminating the dragons right from camera. This light angle would have killed all of the sculpting which made them look starving and exhausted and did not trigger the subsurface scattering the way we wanted it to look. The only solution was to come up with a light setup, carefully adjusted on a per shot basis, referencing but ignoring the set lights, creating the desired look without making the elements fall apart. A lot of tweaking was needed in comp to find the right balance between waxy skin and enough color information from their original skin tones surviving the low light and dark scenery.
What do you keep from this experience?
Try to be as much prepared as possible, but don’t fear to walk off the path if needed.
How long have you worked on this film?
We started to work on the assets in June 2015 and finished the last shot in April 1016.
How many shots have you done?
Roughly about 50 shots.
What was the size of your team?
The team varies a little bit with the distribution of shots between the episodes, but round about 30 artists have contributed on Season VI.
What is your next project?
We already started to work on the new season and are currently working on XXX3 – THE RETURN OF XANDER CAGE while doing dinosaurs and huge environments for IRON SKY II.
A big thanks for your time.
// GAME OF THRONES: SEASON 6 – NEW MAKING OF BY PIXOMONDO (click on the picture below to watch it)
// WANT TO KNOW MORE?
– Pixomondo: Dedicated page about GAME OF THRONES – SEASON 6 on Pixomondo website.
© Vincent Frei – The Art of VFX – 2016
