After explaining the work of Mr. X on SCARY STORIES TO TELL IN THE DARK, Matt Glover is back to tell us about his work on CARNIVAL ROW.
How did you and Mr. X get involved on this show?
We started early on the show – during pre-production, and helped created some very high resolution DMPs to be printed and used as translites on set during filming. From there we were awarded more work – including the set extensions for the eponymous Row, along with other CG creature shots and more environment work.
How was the collaboration with the show runners and VFX Supervisor Betsy Paterson?
Betsy was very easy to work with – all our shots flowed through her before being presented to the Showrunners. She was very open to us trying first takes, and gave clear, concise notes that helped us get to the right place quickly. We had weekly reviews and received notes on shots daily.
What was their expectations and approach about the visual effects?
Like most episodic projects these days – they expect feature quality VFX that complement the high quality of the rest of production. World building and selling the magical creatures was the priority – as they were integral to the story.
How did you organize the work with your VFX Producer?
I worked with two VFX Producers on this show – Greg “Ziggy” Sigurdson and Adnan Ahmed. I have a long working history with both producers. They worked with the Production-side VFX Producers to facilitate turnovers, reviews and notes – and with our internal production teams, including our Production Managers, Natalie Broomhall and Nicole Librodo and Production Coordinators Kyra Lepp and Elias Fattal. They scheduled and tasked the work – so that we could work as efficiently as possible – especially since we were able optimize our set extension work based on the recurring angles and layouts.
How did you split the work amongst the Mr. X offices?
Toronto, Bangalore and Montreal all contributed. The show was based out of Toronto, and we handled all the layout, lighting/rendering, FX and a fair amount of the compositing here. Bangalore composited quite a few shots as well, and contributed a lot of asset support to help fill out some of our bigger environment builds. Our Montreal team composited a few sequences as well.
What are the sequences made by Mr. X?
Our main task was the set extensions for CARNIVAL ROW itself. We built a 360-degree layout that worked for almost every direction the camera can be pointed. We also did a number of CG Centaur shots, Faerie wings when they appeared in our set extensions shots, other environment work, such as in the harbor or the police constabulary building. As well as a couple full CG establishing shots.
How did you work with the art department for the city Burgue?
We received an early turnover of some key shots that would inform the main views of Carnival Row. The Art Dept. had done some paint over concepts to set the general look and feel of the architecture as well as the level of wear and tear etc. They also provided a lot of style reference, inspiration and photo reference of real world locations that we should be aiming for.
What kind of references and indications did you received for Burgue?
In addition to all of the concept art, they also provided a lot of style reference, inspiration and photo reference of real world locations that we should be aiming for. We also got a full handover of set measurements, blueprints and Lidar of the entire build.
Can you explain in detail about your set extension work?
Our highest volume of work was to extend the practical set of the streets. We cleaned up the Lidar scans and created tracking geometry for the whole set build – and then started creating hero buildings to extend the hero FG set upward – usually about a story or two taller. We also built modular building pieces to work as a Lego set to fill in the MG behind the hero buildings. DMPs were used for the distant backgrounds and included more buildings and hillsides. Once the model was approved, we could use the tracking geo to help solve camera tracks and quickly show a layout for approval. Using the Art Dept. reference and the textures of the practical set build we textured and lookdeved our CG to match the lived in look and feel of the plate photography. Alan Bradshaw – our CG Supervisor made the call to render with Red Shift – since we would be using the same street layout for a large volume of shots – it made sense to utilize the speed of GPU rendering.
What was the size of the real sets?
They built a few city blocks, a long main street and several shorter cross streets. As well as rail station platforms, vendor stalls and a river wall. The streets all ended in massive greenscreens, as did the view past the river wall. At these points the world is entirely CG – from the ground up.
How did you populate the city?
They used a lot of extras on-set which was great – this helped fill up the FG and MG – which worked well for most shots. On the occasion we saw directly down a street or bridge that was going to be entirely digital, or for the high wide fully CG shots – we used Massive to generate digital crowds. We created assets for people and magical folk based of scan data provided by Production.
The city is seen in various light conditions. How did you manage this aspect?
We got great on-set lighting reference – HDRIs and chrome balls/charts for every setup – so once we matched to that our renders were in striking difference for our compositors. As we were working on the assets we tested them in various lighting conditions that we knew we would need to match to later – to ensure the lookdev was balanced and working as expected.
Can you tell us more about the design and creation of the trams?
The Tram came to us as an approved design from Art Dept. We received a 3D model witch we up-rezed and lookdevd to work in hero shots. We rigged it so that when we see the undercarriage we see parts moving etc. We also had some CG crowd inside along for the ride.
How did you handle the FX work for the smoke and the river water?
We did a lot of chimney smoke, and the CG Tram was also puffing smoke at all times. Hero chimney smoke and the Tram smoke were always simulated in FX, and distant to BG smoke was a mix between 2D elements or 3D elements placed in comp. Whenever we saw the river we simmed and rendered the water, and we had some shots in the harbor area, and aboard a ship later where we also had a CG ocean.
Can you elaborates about the design and the work on the CG centaurs?
On-set, the actors were filmed with a greenscreen horse-shaped stand-in object that they towed behind them at the waist. This gave us a good reference for the overall outline we needed to match. We also received cyberscans and texture reference of the actors who were playing specific Centaurs – this allowed us to match the fur colour/patterns. We generated tracking geometry from the scans, and used it to match-move the actors. We then animated the CG Centaurs limbs, body and tail – which also ran through a sim. We painted out the stand-in objects and composited it all together. It was always tricky to integrate the movement and animation of the CG limbs/body to feel properly integrated with the actor’s real movement of their upper body. An early shot features a mother and daughter Centaur sweeping, and the rotation of their arms needed to translate into the shifting of their hoofs and have the correct weight transfer – otherwise it didn’t feel like one complete creature.
Can you explain in detail about your work on the Faerie wings?
The asset was created by another vendor, and ingested by us for use in our shots. For any of shots that required CG Wings, we would animate render and final comp them ourselves.
Can you tell us more about their animations?
We received a detailed animation reference guide from the other vendor, that showed the flap cycles at different speeds and rendered with different sub-steps, so we could easily understand what the desired effect was. Betsy also sent us some completed reference shots from other sequences in the show. We had a lot of complex wing shots, like the sequence when Vignette steals the flag at the police constabulary. It was always a challenge to balance the speed of the wings and their visibility. Also when the wings either start or stop moving they needed to look natural and believable.
Which sequence or shot was the most challenging?
We had two big wide establishing shots, one of which was entirely CG. These took the most time and contained the most challenges. We see the whole city, the river, the bridge, and a CG crowd.
Is there something specific that gives you some really short nights?
At the start of the show – knowing we would be responsible for building what felt like a few entire city blocks!
What is your favorite shot or sequence?
I think the finale, the end sequence that culminated in our big wide shot of the city. There is also a shot in the first episode that is particularly long, and pans almost 360 degrees – showing off the whole world we were trying to create.
What is your best memory on this show?
We had a great team, and good clients. All you can ask for!
How long have you worked on this show?
From August 2017 to July 2019.
What’s the VFX shots count?
236 Final shots in the show.
What was the size of your team?
We had a core team of around 50 – but over 200 artists and production team worked on it in some capacity – across all 3 of our studios.
What is your next project?
Can’t say yet!
A big thanks for your time.
WANT TO KNOW MORE?
Mr. X: Dedicated page about CARNIVAL ROW on Mr. X website.
Betsy Paterson: My interview of Overall VFX Supervisor Betsy Paterson about CARNIVAL ROW.
© Vincent Frei – The Art of VFX – 2019
