In 2012, Boris Schmidt explained the work of Pixomondo on THE AMAZING SPIDER-MAN. He then worked on many shows such as FURIOUS 7, POWER RANGERS, THE OA and FOR ALL MANKIND. He talks today about his work on RAISED BY WOLVES with VFX Supervisors Max Riess, Moritz Bock and Julian Lojek.

What is your background?
Boris Schmidt: I was born and raised in Germany. I started doing computer graphics as a hobby on an Amiga 500 in the late 80s using LightWave and Deluxe Paint. I studied media design in the 90s and did work on ArchVis as a freelancer at that time. For my thesis, I created a fully animated CG character short film using a Vicon Motion Capture system.

In 2000, I started working for Pixomondo in Germany doing CG and VFX for commercials including Porsche. That’s when I started to develop a generalist skillset, because most of the time the teams were so small, that you had learn how to do everything yourself. I moved to Pixomondo’s Los Angeles offices in 2009 to work on Roland Emmerich’s movie 2012. I have been L.A.-based since.   
 
How did you and Pixomondo get involved on this show? 
Schmidt: Another VFX house had already started to work on most of the shots but, for reasons not disclosed to us, the work was pulled from them. Pixomondo was already doing some techvis for a reshoot for the Boston Stadium scene. Pixomondo’s Sci-Fi and Creature Showreel was shown to Ridley Scott and they decided to award us a major portion of the VFX shots for Episode One and Two, and then it continued through the rest of the season all the way to Episode 10.

How was the collaboration with the various directors and VFX Supervisor Ray McIntyre Jr.? 
Schmidt: When Ray came on to the show, we were already in the midst of creating several shots and assets for the first 2 episodes. In his first week, we were able to help him to get up to speed with all our work and what we had been doing so far. Ray is a very experienced supervisor and easy to work with. He was always giving precise notes and has a good understanding of the technical challenges. He was managing the notes from the various directors and making sure their creative vision was transferred to us. Have I mentioned that his entire VFX team is also great fun to work with as well?

What was their expectations and approach about the visual effects? 
Schmidt: Ridley Scott is known for his iconic feature film work, so the bar was set high to create photoreal VFX that supported the story. Ridley Scott tried to solve some effects in camera. For example, there is one scene in RBW that we worked on, where one of the kids accidentally starts the engine of the android’s lander that was stuck in a hole. Ridley Scott had water dripping down from an LED Panel that was backlighting the water drops falling into camera, creating streaks of light and bokeh effects. He used that same technique in ALIEN when Brett (Harry Dean Stanton) enters the claw room, and BLADE RUNNER has plenty of scenes with backlit rain. We then augmented the plates with CG effects. 

At the early stages we got a large amount of concept art and even personal scribbles from Ridley Scott, which created a good baseline for the creative development and design of the various assets, creatures and environments. The onset data volume of Lidar Scans, HDRI, witness cams, photogrammetry and reference photography were massive and helped to make sure that everything lined up with the photography to create photoreal shots. 

How did you organize the work with your VFX Producer? 
Schmidt: Minh Ly was Pixomondo’s VFX producer on the show. Our biggest challenge was at the beginning, sorting through all the work that was done before we took over the shots, figuring out if there was anything useable from the previous VFX company, and how to split up the whole show across all our Pixomondo facilities. Then it was about putting a price tag on things and communicating that with all the other Pixomondo facilities. I was focusing on the creative part and Minh was taking care of the financial part. Minh was the hub for all the other facilities’ producer needs and making sure to communicate to the client VFX producer Ruth Hauer. Minh also dealt with all the other VFX vendors that we had shared shots or assets with. Minh and I were sitting right next to each other and were able to keep a steady conversation about the needs of the show and solving problems on the spot. A huge thanks goes to Pixomondo’s Alex Torres (DPM), Tim Weber (Coordination) and Angela Elgar (coordination) who were our special taskforce making sure that Minh’s and my needs were covered. Overall a fun and very professional team. 

How did you split the work amongst the Pixomondo offices? 
Schmidt: For Episode One and Two, Vancouver took on some of the set-extension work. After Episode One and Two had been delivered, we did a smooth transition to Vancouver becoming the main VFX facility. Pixomondo Vancouver’s in-house VFX Supervisor Edward Hawkins and in-house VFX producer Adam Harrison took over the lead on the following episodes. The Vancouver team did some of the remarkable creature work and the amazing sequence in which Markus is cutting his face. They also did most of the settlement extensions and all the snow flurries.

The creature and animation-heavy shots were sent to Frankfurt. Their rigging and animation team is phenomenal and Pixomondo Frankfurt ‘s in-house VFX Supervisor Max Riess is an old friend and colleague of mine when I was still based in Germany. He worked on GAME OF THRONES and many other high-profile creature heavy shows that PXO has been on over the years. The Frankfurt team did most of the shots involving the creatures attacking and fighting. Another big chunk of their effort went into creating the photoreal medical android sequence. They did quite a few set extensions and a scene where Mother gets impaled by a giant dragon skull tooth. 

Most of the heavy environments and ark shots were sent to Pixomondo Stuttgart. In Stuttgart, the experience and artist pool for environments is immense, and our in-house VFX supervisors Julian Lojek and Moritz Bock are an expert team in creating stunning visuals. They created the Boston Stadium sequence and worked on all the Ark shots including the wreckage of the crashed Ark. The team in Stuttgart also did all the CG extensions of the hole on the planet containing the cgfx heavy shots where the android lander is falling into the hole and exploding. 

We gave Pixomondo Toronto the look development and art direction for some substantial one-off shots. Our in-house Supervisor Phil Jones and Bojan Zoric have a strong background in matte-painting and look-development. Pixomondo’s Toronto team designed the amazing scenes where Mother is taking the Necromancer’s eyes out of the eye-socket. They also created the beautiful decloaking effect of Marcus lander. Toronto also worked on shots where we were adding heat-distortion and ash effects, and did some of the hole-extension environments. 

The teams in Los Angeles and Montreal also contributed to shots like hole-extensions, heat distortion, ash and snow particles.

Can you elaborate about the design and creation of the Mithraic Ark? 
Moritz Bock: The design was already done. I just tweaked it a bit since the scale of the Ark was tricky to tell in shots where you had more detailed structures positioned next to it, like the stadium shots for example. Since we could not greeble the surface of the Ark, we had to come up with something that would work well with the rest of the show’s aesthetics like the lander. We got inspired by modern yachts where you would see lots of carbon fiber structures and layers under a clear protective coat. In order to not get too noisy, we would then scale those patterns depending on distance and surroundings. 

Julian Lojek: We received an initial model from Mr. X, that I think had been used for previz, and based our asset on that. The general design and silhouette of the Ark changed only marginally. We then went ahead and concepted and modeled the details, line structures, hatches and gates, and created parts of the interior structure. We also knew we’d have to make the Ark crash into a mountain, so we kept that in mind and made sure we’d be able to hand the model off to the FX department without running into bigger issues. 

How did you handle the size challenge for the Ark? 
Bock: The scale of it was a real challenge. In the stadium we had to compete with the high frequency detail of the ruined stadium and the crowd below. We ended up adding broken up reflections and layering atmospherics. The space shots were an entirely different beast. We ended up treating the Ark as a giant parabolic mirror as well as adding multiple light selects. Then we would iterate on a per shot basis to tell the scale. 

Lojek: In terms of layout and rendering, there were no major hiccups to overcome. It was cumbersome at times, because you’re dealing with structures that are several kilometers big, but apart from that there were no big issues. We did run into interesting situations with the Ark crash site, because the location was smaller than the Ark wreckage would have required. We ended up cutting up the cg into sections and doing a weird 2.5D-3D-forced-perspective hybrid to make it all work. 
 

The Ark crashed on the planet Kepler 22B. How did you create the crashed wreckage views? 
Lojek: We extracted the main volumes of the wreckage from our Ark asset and went through a few rounds of layout until the client was satisfied with the general look of it. The rift created by the crashing ship was modeled and sculpted onto both photogrammetry from drone footage, Lidar scans from the set and DEM data. We modeled a base geo of the destroyed Ark that we would render and use as a starting point for matte paintings, which we would then project back onto the geo. And finally, we added a lot of debris and created hundreds of fires and smoke columns to populate the site with. 

Let’s talk about the creatures. How did you design and create them? 
Riess: We did a lot of concept work to get Ridley’s approval. This was mostly done as a combination of rough Zbrush work with paint-overs. In earlier designs the creature had a lot more hair. After getting the look approved, it was a lot of detail work in Zbrush, texture production in Mari and shader development in Arnold.

What kind of references and influences did you received for them?
Riess: We looked at a lot of different animal footage for references. Throughout the project the main influences change a lot. I would say it was a mixture of wolves, cheetahs and a bit of hyena. One of the hardest things was to find references for the running. The creature has a very unusual physique. All extremities are exceedingly long compared to the torso and the hind legs lead to a shifted high-hip position when running. Major inspirations for our running cycles were humans running on all fours and parkour runners. 

Mark Spindler: For the sequence in which the creature and Mother fight in a dark cave, we also used the reflective qualities of cat’s eyes as inspiration. Some of the shots had a strong backlight, but almost no fill, which made the creature’s face hard to read. By adding this cat’s eye reflection, we made sure that we could always see the eyes in an otherwise dark face, without making the lighting look fake. Plus, it looked cool and creepy. 
 

How was simulated their presence on-set? 
Schmidt: In some shots we had stand-ins such as a rubber torso. In others we had stunt crew in green suits. 

Can you tell us more about their rigging and animations? 
Riess: The creature has a classical human biped rig. There are some extra features in the facial rig, like the ability to extend the inner jaw, a bit like a shark, but I am not sure if it is visible in the shots. Since it is more of a human character, the animation team acted out a lot of the motions themselves, like running on all fours. Father wrestling with the creature led to some very intense dailies!
 
How does the grey aspect of their skin affect your lighting work? 
Schmidt: We always had good reference plates with a creature prop head for most of the shots. That was extremely helpful. We used the HDRI information provided by the onset team. Just like we would have done on a real set, we made some creative choices on lighting the creatures, especially because of the pale skin. But we always try to keep as close to the real light on set as possible. 

 
Can you elaborate about the shaders and textures work? 
Schmidt: For the pale skin it is crucial to get the subsurface scattering correctly, along with the amount of spec. We had used a combination of cavity and wetmap to give the creature a good mix of dry irritated skin patches and more moist areas around head, eyes, mouth and chest. In addition, we gave the creature a layer of dried dust and clots of mud, which served as a good visual breakup and helped integrating the creature into the real plates. 
 
Can you tell us more about their face animations? 
Schmidt: The creatures do have a variety of facial expressions, but we tried to keep it subtle. The mouth is doing most of the work in combination with some simulated saliva. We wanted to keep a low key complexity and in some shots, we went for a more human expression, while others show much more of a wolf-like aggression.

Which sequence or shot was the most challenging? 
Riess: In one, the creature was wrestling with Father. It was shot with Father carrying around a sandbag creature torso. This provided us with a very good reference, but since the creature scale and proportions changed a bit, the integration of the cg creature was a challenge in some shots. It took some time to figure out the delicate balance between the creature’s animalistic, instinctive reactions and its more human character traits.  

Is there something specific that gives you some really short nights?
Schmidt: Working long days, which was kept to a manageable amount on this project.

What is your favorite shot or sequence? 
Schmidt: I have multiple favorites. Here are some of them:
One is where the Ark is flying over the cloud-covered mountain and then crashing into the distant mountain range. These shots sell the scale of the Ark well and feel epic. 
Another one is where the medical android takes off his glove and we see the all the burnt muscles and technical details and underlying structures. The cg and compositing that went into this shot turned out to be exceptionally photoreal.  
I really like the shots where Mother takes out the necromancer eyes and we see them boot up. 
I really like the shots where Marcus is cutting his face. Kudos to (Pixomondo Vancouver’s VFX Supervisor) Edward Hawkins. 
 
Moritz: I really love the stadium sequence for its mood and lighting (and the cameos of some of our crew in the crowd). We even managed to sneak in one of our office dogs and immortalize him! 

Lojek: My favorite single shot would be the one where Campion is digging a grave and then the Ark appears over the low-hanging clouds, pushing them down. It is just such an epic shot, and I think the scale of the ship really comes through in it.  
 

What is your best memory on this show? 
Schmidt: Although I never talked to him in person, I was told by Ray, that Ridley Scott really liked our work. I am a huge fan of BLADE RUNNER and it felt good to get positive feedback from someone you look up to.

How long have you worked on this show? 
Schmidt: I worked a little over 5 months on the show.

What’s the VFX shots count? 
Schmidt: 540 shots total. 
Out of these I was in charge for 280 shots for Episodes One and Two. 
Edward Hawkins was the VFX Supervisor on 260 shots featured in Episodes Three, all the way through 10.

What was the size of your team? 
Schmidt: I would guess the core team that was working on the show was around 350 people, which is a remarkable size. 
 
What is your next project? 
Schmidt: I am finishing up a commercial right now.
 
What are the four movies that gave you the passion for cinema? 
Schmidt: When I was a kid in the 80s and saw the first computer generated images in movies, I was blown away by the look. I believe that was when I started to develop a deep interest in how to create this myself. I still have a great amount of respect for the pioneers and artists who founded this art form. The movies that sparked my interest in VFX and cinema as a kid were TRON (1982), BLADE RUNNER and the original STAR WARS film. And – it’s not a film, but – Kraftwerk’s 1986 music video, Musique Non Stop. The aforementioned examples obviously do not hold up to today’s technological standards, but at the time they came out, they were well ahead of their time, and continue to be influential to this day. 

A big thanks for your time.

RAISED BY WOLVES – VFX BREAKDOWN – PIXOMONDO

WANT TO KNOW MORE?
Ray McIntyre Jr.: My interview of Overall VFX Supervisor Ray McIntyre Jr. about RAISED BY WOLVES.
Pixomondo: Official website of Pixomondo.

© Vincent Frei – The Art of VFX – 2020