Max Riess began his career in visual effects more than 10 years at Pixomondo. He has worked on many projects such as HUGO, STAR TREK INTO DARKNESS, GAME OF THRONES and IRON SKY 2.
What is your background?
I started out as a 3D generalist about 12 years ago for Pixomondo – Commercials in Frankfurt. After a couple of years, the company’s scope of work changed more to feature films and I got interested in the possibilities of compositing. In 2012 I moved to Los Angeles to focus more on matte painting, look development and compositing. Three years later, I moved back to Germany with my family and have been working as a VFX Supervisor for Pixomondo Frankfurt since.
How did you and Pixomondo get involved on this show?
Lea Prainsack, VFX Producer and Betsy Paterson, VFX Supervisor on CARNIVAL ROW, Season One, had just started pre-production and shot stunt tests to figure out how the fairy wings should behave in flight. They were looking for a VFX house with an experienced flight animation team and invited us to join. Even though we gained a lot of experience with wings over the years with Daenarys dragons on GAME OF THRONES, we quickly realized that fairy wings work very differently than dragon wings! We did a lot of research, presented first flight cycle render tests and concept work to Betsy and Lea, and then they placed a large part of the wing work with Pixomondo.
How was the collaboration with VFX Supervisor Betsy Paterson?
I enjoyed working with Betsy Paterson very much. She has a specific idea of what she expects in a shot and for the overall look of the show. From the beginning, she gave us a good understanding of the world of CARNIVAL ROW, while still leaving space for the team and I to contribute our creativity. Considering the complexity of the shots and the high quality level, it was a pretty smooth ride and fantastic overall experience.
In the midst of working on the re-shoot material, Betsy had to leave for her new show, handing the remaining work over to Todd Shifflett. Even though we were working on one of the most challenging sequences at that time, they made the transition very easy with a well-coordinated turnover. The rest of the show was handled graciously by Todd for which we are very thankful for.
What was their expectations and approach about the visual effects?
One of their main goals was finding the right balance in CG effects to achieve the realism needed. Seamless integration of CG elements was crucial. All of the set work was very well organized. We always had a lot of reference material like chrome ball, clean plates and light stand-ins with prop wings at hand. Their art department created very elaborated production artwork which gave us tons of inspiration and ideas.
How did you organize the work with your VFX Producer?
Pixomondo VFX Producer Mona Mohr and I have been working together for several years now so there is a good sense of understanding between us. I oversee the creative direction and the technical side, while she deals with the timing, crew management and everything related to the budget.
How did you split the work amongst the Pixomondo offices?
Frankfurt was the lead facility. Since we have worked on a lot of creature assets in the past, our team was ready to put its knowledge into the fairy wings. The groundwork on asset and animation was done here and then shared with some of the other branches. The Toronto team was led by Bojan Zorico and Celina Zoleta, while the Stuttgart crew was supervised by Julian Lojek and Sebastian Meszmann.
They took on full sequences with wings and set extensions. They all did amazing work and brought in the individual strengths of their teams. This is one of the reasons why we decided to split the big aerial fight sequence above the roof tops in Episode 3. It has been a shared venture between Stuttgart handling the full CG environment and Frankfurt creating the animation and final compositing.
What are the sequences made by Pixomondo?
We worked on all episodes and did mostly wing replacement or enhancement. In a couple of shots we did some CG set extensions as well. The sequences include the library environment in Tirnanoc, most brothel interior shots, the wings of the dead fairy, Philo’s baby wings, and the big aerial fight over the rooftops, just to name a few.
How did you work with the art department about the Faerie wings?
The art department sent us a lot of reference material and as a starting point we received hires scans and texture photography. The prop wings looked magnificent and convincing in a lot of shots, but had their limit when it came to action shots or special abilities, like the fluorescence.
Can you explain in detail about the design and creation of the wings?
The fairy wing asset was initially developed by Pixomondo and shared with the other vendors on the show, who adapted it for their own pipelines.
We started out with basic 3D sketches and concept artwork even before we received the hires prop scans. We spent a lot of time researching animal wings in nature. One of the biggest issue was the scale. A dragonfly has very filigree wings, which we needed to modify in order to carry a human-sized creature convincingly. Some of the first models had wings in a more realistic proportion but that created massive problems in the way they were supposed to fold and caused an inconsistency with the wings on set.
In the design process we spent a lot of time thinking about the anatomy of the fairy wings. The first version had more bone and skin in it. Later, we developed a look closer to insects with very flexible wings that can go from limb to solid. This way we were able to blend CG wings seamless into the rubber prop wings which affected the way we animated them.
How did you create the various shaders and textures?
We used Mari for the texturing and Zbrush for most of the surfacing. The base was done with classic texture work but there are some procedural shaders on top for further breakup and detail. The shading was done in Arnold. First we created one main shader and then we produced variations. Some of the later close-up shots required more detail, which was added in another layer of micro detail displacement and further texture detail in the areas that were needed. In addition, we set up a couple of extra render passes for reflection with iridescence shaders to get additional shimmer on the wings later in compositing.
The wings are translucent. How does that affects your work?
For some of the close-up shots we used a shader model with a lot of sub-surface scattering and different breakup maps for the individual cells, giving the wing a nice look of organic complexity. As a result the render times were quite expansive for a full 4K image. Whenever we had wings in higher frequency we rendered a less expensive transmission shader and controlled the needed amount of light scattering in compositing. In some cases we had to cheat the transmission manually to get a more visible shape in high frequency shots against bright backlight.
Can you tell us more about the glowing aspect during the sex scenes?
This one was a bit tricky and is probably the sequence that went through the highest amount of iterations. It was important to get the right balance and to avoid a cheap and unrealistic look. One of the major inspirations were deep sea fish and jellyfish. We created a couple of extra geometry inside the wings that were able to illuminate inside the cells. It took us quite a while to get the right pattern and speed.
How did you manage the challenges for the connection of the wings and the characters?
Technically this was the most complex part on the wing asset. We did an extensive body track and rotomation. We have developed a couple of tools that allowed the animators to bring in the wing rig and snap it automatically to the tracked wing position marker. Some shots had an extra CG back patch to integrate the wings on bare skin.
Can you explain in detail about the wings animations?
Especially time consuming was finding the right speed. The highest frequency that we used has around 4 full flaps per frame using about 24 sub-steps between each frame. We had to come up with new tools and changes in our animation and rendering pipeline.
Eventually our Animation Lead Philipp Winterstein created a motion library of wing cycles with different flap rates and styles. We also differentiated between male and female flight cycles.
Our Lead Rigger Johannes Wolz has a lot of experience rigging wings and creatures. The wing rig is very versatile and able to adapt for different kind of marker positions and body sizes. Each wing can be scaled and formed individually and a second level of deformers allow for all kind of more complex shaping actions. This was needed for a couple of shots where we had special interaction with the wings, like wings unfolding or someone grabbing and touching them. The goal was to achieve an organic cloth/skin feeling without time consuming simulation and keeping creative control.
Can you elaborate about the big fight sequences over the roofs?
This might be the most complicated sequence we worked on. I find it always difficult to transform a full greenscreen set to a 100% believable environment. We wanted the whole sequence to feel very dynamic with a frantic camera style supporting the “life or death” nature the fight. The wing animation is mostly very high frequent with a lot of subtle changes in speed and angle. A couple of shots forced us to re-project the actors to get rid of the feeling of having them glued to invisible ropes and give them weight shift coherent to the wing cycles.
Pixomondo Stuttgart handled the full CG environment. They started out with concepts and blockings, for all different types of houses and street assets. We used an aerial map for the art department to divide the city in blocks and create a rough street map. To create a diversity we gave the districts specific purposes. In the beginning we start near the river across the rich quarters, then we fly over Carnival Row and the adjacent industrial quarters with high chimneys and factories, ending up in a big, old backyard.
Which sequence or shot was the most challenging?
I think the aerial fight sequence between Vignette and Hamlyn due to the complexity of the shots. The stunt team on set did a fantastic job with the rope work on set, but the very dynamic camera work was a challenge for most of the VFX elements. Paintout and rotomation were very time consuming and of course bringing all the elements together. For one shot we had to create a full CG double of Hamlyn falling into the courtyard.
Is there something specific that gives you some really short nights?
In my opinion we handled the project quite successfully with a minimal amount of long hours. There was one shot in the brothel with a fairy lifting off during intercourse, that turned out much more difficult than expected. It was one of the earlier shots that gave us a bit of headache with getting the wings attached to the body. The shot was very long, almost 600 frames, and had flickering lights from several light sources, which made the paintout and CG integration very time consuming.
What is your favorite shot or sequence?
I can’t really decide. Even though the major focus of our work has always been on wings, the sequences and shots have all been quite different.
For the emotional effect and the mystery, I really liked the library scenes. They really give us a glimpse into the magical world of the fairies that is hidden to the human eye. From the technical aspect my favorite of our sequences was the fight above the city. It was very demanding and the most complex shots we have done on the show so far.
What is your best memory on this show?
I had a lot of great experiences. One specific moment was when we changed the way we animated the wings. I was sitting in dailies looking at new render tests that showed for the first time how we solved the High Speed Frequency Problems.
How long have you worked on this show?
Long… almost 2 years. We started in September 2017 with first tests for the CG wings and completed the last shot in June 2019.
What’s the VFX shots count?
268 shots.
What was the size of your team?
At peak times about 90 people worked on the show simultaneously. Due to the long production time the team changed quite a bit over those two years. In total around 200 people spreading over 4 facilities worked on CARNIVAL ROW.
What is your next project?
I am not allowed to say at the moment!
What are the four movies that gave you the passion for cinema?
There are a ton of movies that inspire me, but if I had to pick: JURASSIC PARK, BLADE RUNNER, STAR WARS and ALIEN.
A big thanks for your time.
WANT TO KNOW MORE?
Pixomondo: Official website of Pixomondo.
© Vincent Frei – The Art of VFX – 2019
