In 2015, Ara Khanikian told us about Rodeo FX‘s work on The Last Witch Hunter. He went on to work on numerous projects such as Kong: Skull Island, The Boys, We Can Be Heroes and Jungle Cruise.
How did you and Rodeo FX get involved on this show?
I had been wanting to be involved in a Marvel show for quite some time so when I heard about this show and that it involved a fight sequence outside of a building in bamboo scaffolding I got very excited and curious about it. When Christopher Townsend and Damien Carr (VFX Producer) walked us through the work as we were bidding it, I was sold and really up for the challenges it presented!
How was the collaboration with Overall VFX Supervisor Christopher Townsend?
Working with Christopher Townsend was one of the best experiences ever. This was my first Marvel adventure and it was one of the most enjoyable experiences in my career and it’s all really thanks to Chris. He’s a veteran, world renowned VFX Supervisor who knows exactly what he’s looking for while giving you more than enough room to be very creative, which is truly fantastic. You can’t ask for a better partnership and relationship, especially on a show like Shang-Chi.
We also had the chance to work very closely with Joe Farrell, 2nd unit VFX Supervisor, with whom we collaborated very closely on our sequences.
What were their expectations and approaches to the visual effects?
Like most VFX in films now, and especially a Marvel film, expectations are always very high and Shang-Chi and The Legend of the Ten Rings was no exception to the rule. When we were initially discussing the Bamboo escape sequence, Chris was looking for a photoreal, very high concept look, believable yet just a bit flashy. The fight with Shang-Chi and Death Dealer against a bright colourful billboard was always meant to be a very graphic and stylized sequence. So we knew from the start the style of the visuals we needed to create and which would involve, photoreal CG assets apart, bright and colourful billboards, very dynamic lighting rigs, contrast and juxtaposition of bright and flashy colours, very creative compositing, lens flares, etc
How did you organize the work with your VFX Producer?
The split of work between myself and Annie Normandin, VFX Producer, was a very standard and classic split where I would oversee the technical and creative aspects of the work, client cineSyncs, dailies, etc. and Annie would oversee the overall budget, schedules, and crewing of the show.
What sequences were made by Rodeo FX?
Rodeo FX was involved with the scenes taking place in Macau. Ranging from the arrival of Shang-Chi and Katy to the city, going up the elevator to the Golden Dagger’s Club, and most notably the Bamboo scaffoldings fight sequence followed by the fight between Shang-Chi and Death-Dealer.
Can you elaborate about the creation of the Macau environment?
Production provided us with artwork for a very high concept, bright, flashing with neon signs and billboards, yet a very photoreal Macau city. The original methodology called for a team that would physically travel to Macau, shoot BG plates, HDRs, get scans and photogrammetry, shoot reference plates and elements so that VFX can use them as a base and through 2.5D techniques and CG enhancements, create the environment to composite with the bluescreen FG shoot, which was a more classic and typical approach to creating VFX for a scene like this one.
When Covid hit, that methodology got discarded pretty quickly with all the travel bans that were put in place. Upon further discussions and brainstorming sessions, we all agreed that building a full CG Macau was the only viable solution for this sequence for the obvious logistical reasons but also for additional creative freedom in designing and art-directing shots. And so our adventure began.
We were lucky enough that production had in their possession aerial footage of the Macau downtown area for scouting purposes. Once we all agreed on where we would drop in the CG “in-construction” skyscraper, we started the Macau build. Since we wanted the action to take place in actual Macau and not in a generic artificial asian city, we came up with the simplest way to build it without resorting to “brute-forcing” the CG and over-building it.
Matthew Rouleau, our Environment Supervisor, started the research and the work with Google maps and OSM (Open-Street-Map) data, using photogrammetry tools to extract a rough geometry of every single building in Macau and populated the empty canvas with them. We then extracted topography / elevation data and used it to accurately displace the ground plane onto which these buildings were. This gave us the perfect and most accurate starting point to build the city.
Since we knew where the approximate location of the hero skyscraper would be in Macau, we blocked it in on the desired street and rendered it in different positions, or addresses, and rendered the approximate views of the city we would get from different vantage points and heights.
This build was happening as production was doing the first edits of the action sequence, which meant that we did not have any visuals of the action scene itself. But, we knew that we were seeing the city from inside the Nightclub, through the elevator as our heroes made their way to the nightclub, and of course, from the main side of the building during the epic fight sequence in the bamboo scaffoldings.
Doing this exercise allowed us to determine which neighbouring skyscrapers would need to be hero builds, which ones would be of lower resolutions and from where on would we use a 2.5DMP 360cyc.
Louis Pare (CG Supervisor) and Marc Desmarais (Assets Supervisor) lead the team of CG artists that went ahead and built the hero CG buildings for all the skyscrapers that were directly adjacent to our hero skyscraper and the ones across the street from it which would be the ones we should be seeing the most. These builds were as accurate as possible to the actual buildings in that city block. We cataloged and used a huge amount of visual references from the internet for all these buildings, their floor-level storefronts, and their look and style of lighting at night.
We then proceeded with creating the mid-resolution buildings for all the hotels and casinos in the main area of downtown Macau. As we analyzed our generic geo extracted for the layout of the city, we determined how many types of different buildings would be required to “fill-in” the volumes and shapes of all the MG and BG buildings of Macau (ie: the slums, the residential areas, the apartment complexes, etc etc ) This step allowed us to create certain types of architecture, create texture and scale variations, and procedurally scatter them on the accurate ground plane we had built in a manner to would “fill-in” the volumes of proxy geos from the accurate Macau representation from the layout build. We designed all the building interiors (offices, apartments, etc) using HDR images using an OSL shader that procedurally maps these interiors onto low-resolution geometry.
We had our first version of CG Macau.
Upon reviewing our work on the Macau asset with Chris and the filmmakers, we decided that, to help with the sense of danger and height of the action scene, we needed to slightly modify the city just around the skyscraper by adding about approximately 15 additional stories to all the buildings around our hero skyscraper and also by bringing all these buildings slightly closer to us. We also moved a couple of buildings around for a better and more balanced composition.
We then shifted our focus to street level detailing and set-dressing the city with bus stops, stop signs, benches, rooftop props, etc. Referencing real-world Macau, we designed the look of the store-fronts, sidewalks, bus stops, trees and greeneries, etc etc all around the main focus of the action. We ran simulations for animated traffic and pedestrians. These simulations included roundabouts and stop signs for traffic and also pedestrians populating sidewalks and crossing streets. A total of 1508 CG pedestrians can be seen walking the streets below as well as 577 vehicles driving around the city. CG atmospheric fog and clouds were scattered throughout the city which allowed a lot of the lighting, and especially the interactive ones, to bounce on and light up these clouds for additional realism.
We spent a substantial amount of time accurately lighting the entire city. We added lights to every single lamp-post (approximately 5000 of them), to all storefronts, to the car’s headlights and tail lights, we added a lot of emissive textures and lighting from signs and logos on buildings, as well as a soft and subtle moonlight on the entire city. And finally, made sure all these lights, static and animated were getting properly reflected from all the different windows and buildings.
Once we had all components working well together, we rendered a handful of hero views of the asset and did a couple of back and forth with the comp team, led by Jeremie Maheu (Compositing Supervisor) to nail the final look. We were very fortunate to have drone photography of the city which we used as a target to finalize the look development and photorealism of the city. We used it to dial in the temperature of the lights, their exposures, the amount of atmospherics and fog in the city, as well as fine tune the animated loops and patterns of the lights animations on the buildings and casinos
We used this study to modify some shaders and settings at the asset level and kept other controls for compositors to dial in values in shot-context. At this point, everyone involved was very satisfied with the look and photorealism of the city and we were ready to transition into shot work.
An important aspect to this build was the photorealism, of course, as well as the amount of life and animated components in it. To achieve that, we treated the Macau asset as if it were a plate shoot, ie: make it look good, include all the simulations of crowd and traffic, as well as all the lighting animations in it at the asset assembly level, let all these elements “just be” in the shots, and only modify elements in them if they hurt the shot design. We knew that we had a lot of work to do in shots so we opted to front-load as much as possible into the asset and try to treat the full CG renders of the asset almost like BG plates in the shots.
The main skyscraper that houses the Golden Daggers Nightclub and onto which the fight sequence takes place is, of course, a fully digital asset as well. We built the skyscraper in a very modular way to give all the control we needed across the sequence. Knowing that the height and width of the building would potentially be modified to suit specific shots’ needs, we separated the build of the inner concrete structure and the outside mirrored glass panels and mullions to easily duplicate and modify certain proportions. The window mullions needed to have a notch in them where all the vertical bamboo in the scaffolding attached to the building. The bamboo scaffoldings themselves were built in an extremely modular fashion as well. We built individual high-resolution bamboo sticks matching the ones built on set. We proceeded with assembling sectional groups of scaffoldings and scattered them across the four sides of the buildings. Straps of cloth were built and distributed across all connection points of bamboo sticks.
The glass windows on the skyscraper were an extremely hero element in every single shot, so we spent a fair amount of time on them to develop the reflective shaders and how they affected the coloring and luminosity of the reflected renders as well as the dirt and smudges textures on them. We knew, from the start, that dialing in a tinted colour to the reflections was going to be a pivotal aspect of the look and one that ensured that the audience would always be able to quickly differentiate between looking at a reflection and looking directly at Macau from shot to shot.
For additional detailing as well as continuity, we built wooden planks onto which people can walk and run on and distributed them on all scaffolding sections. We also built and animated red and white plastic ribbons that flapped in the wind, Diagonal bamboo in X-patterns to solidify the scaffoldings, light enclosures, as well as the outside service elevators. All these props were carefully positioned in a very logical way in the master digital set but with the added controls to easily modify and override their positions and visibility on a per-shot basis. The bottom half of the skyscraper was also wrapped in a green tarp, as seen on a lot of construction sites in that part of the world. The tarp itself was built in position on the skyscraper and did not require any sim or motion.
One of the very complex aspects of this hero skyscraper build was assembling all these different sub-assets together, allowing for maximum flexibility for in-shot modifications, allowing for continuity from shot to shot, as well as to introduce different variants of these assets throughout the fight sequence. For example, adding broken windows, explosions, and post-explosion damage to scaffoldings and wooden planks.
To achieve this level of continuity, early on we analyzed the whole fight sequence and our layout team created a very rough and temp mockup of the entire scene by tagging every single shot to a specific location on the skyscraper. Doing this helped identify from which window they come out at the very start of the scene, at what location and height every single shot takes place, which window needs to explode, which sections of scaffoldings will need to have damage, where the scene ends, etc etc.
This also allowed us to determine over how many stories the action takes place. Once every single shot got camera-matchmoved, the layout team was able to easily place every single camera from all shots into a master layout scene with the entire skyscraper and manage continuity of all elements from first shot to last shot. This step was instrumental in ensuring the “static” asset evolved over time with different story points through time across the sequence.
Did you use procedural tools to create some buildings or skyscrapers in the city?
We did not use any procedural tools for this build.
The sequence happens at night—how did you handle the challenge of the lighting and the scale?
In terms of lighting, we “tried” to keep it simple 🙂 The Macau city itself was always rendered with the same lighting rig independent from the lighting rigs used to match to plate photography. The Scaffoldings and CG skyscraper were lit, matching the on-set lighting to be able to accurately execute the set-extensions of the bamboo scaffoldings and lock the CG renders to the plate photography lighting. Once we locked the FG lighting to the plates and BG Macau was rendered, we would dial-in the lighting in comp to “sculpt” the lighting better and selectively feather off brightnesses on the scaffoldings to taste. We would also, in comp, add life and animate lighting modulations, and flickering, using specific light-groups prepared by the lighting team.
How did you populate the streets with cars and people?
We built a large library of crowds and assorted vehicles, and simulated them using Golaem. Cars and vehicles were a nice challenge in terms of lighting, as each one of them had a light-rig for the headlights and taillights that needed to be constrained to the sim itself, and light their surroundings, as well as have the taillights get bigger and brighter red light when the cars needed to slow down or stop. It ended up working very well and added the necessary realism to all the shots.
A big part of the sequence is taking place in front of a lot of neon lights and reflections. How does that affect your work?
Conceptually, we knew from the start that we needed to have two specific billboards in the scaffolding fight scene. One on the left and one at the right of the skyscraper. These bright and flashy billboards would help with geographically understanding and following the continuity of the fight in the scene, help light the dark environment, allow to design some very graphic silhouettes and cutouts of the FG actions against them, as well as help with creating a very dynamic visual complexity to the whole sequence.
We knew these billboards would be a huge piece of the puzzle as they are very hero and contribute immensely visually in each shot, which meant that we had to find an efficient way to quickly iterate for contents and timings. We opted to use a simple grid in CG for their locations and composite the billboard contents using UV maps in comp. By doing this, we separated the renders of these billboards from the main Macau renders which were very CPU-intensive and long. Compositors could easily slip and slide timings of each billboard to suit the shots needs as well as apply the LCD panel look to them as a post-process step.
Shot-work for the Scaffolding fight scene was as, if not more, complex than the Macau build itself!
The whole action scene was beautifully choreographed and shot on a blue screen stage with a partial build of the bamboo scaffoldings. The scaffolding was built against a mirrored wall that would act as the main skyscraper’s facade and windows to capture all the reflections of the characters executing the fight. Even though it was all shot against bluescreen, we opted to do bluescreen extraction of the FG but also roto all the action that we see in reflections to have enough control and ability to layer in and adjust reflection of the CG city as well as to add our CG skyscraper and Bamboo scaffolding extensions.
Most of the action was shot @48fps with a shutter of 300deg which allowed to have ample frames for complex matchmoves, adding the CG, as well as retimes, while ensuring proper and natural motion blur once brought back to 24fps. Witness cameras were used to help with the complex camera matchmove as well as all object tracking of the partial Hooked-swords that the Ten Rings soldiers were holding. We replaced every single sword with a CG version that included a blue tinted energyFX that sparked with electricity and interacted with its surroundings.
We ran FX/simulations for all the window smashes and breaking, as well as the numerous explosions, and the electrocutions caused by the hooked-swords and ensured continuity of these elements across the sequence.
Numerous digidouble take-overs were used throughout the sequence to enhance some action beats as well as to add CG versions of the soldiers falling to their deaths from the terrifying height of the scene. These animated characters were handled by Grae Revell (Animation Supervisor) and his team.
A handful of full CG shots were also created for some of the wider views as well as to transition to and from specific choreographed fighting sections.
Adding the CG bamboo scaffolding extensions were very challenging as the set flexed, moved and bounced with the characters interacting with them so we had to hand animate the set extensions of the scaffoldings deforming with the built set.
As we were assembling all the elements in these shots and populating the edit, we went through numerous iterations for the billboards contents and timings. We kept an up to date edit of all the work in progress shots and added a PiP proposal for the latest and greatest billboards content to ensure continuity and quickly iterated until we all felt the content worked best for the entire scene.
The lighting team found a very clever way to preserve all the AOVs of the CG Macau in the reflections of the mirrored glass windows which was instrumental in controlling the look throughout the sequence. Comp had access to all the same typical CG AOVs and passes as if it were a normal render, even though they were being rendered as reflections getting deformed and influenced by the CG glass panes of the skyscraper. Each shot was lit using on-set lighting references to accurately extend the FG set into the CG env. And compositors used the provided AOVs from lighting to create additional breakup and falloffs to darkness on a shot by shot basis. Interactive lighting from the animated billboards was also added to shots and plate photography and lighting was dynamically enhanced to suit the shots and flaring / light-wraps were art-directed in every single shot. Macau city itself did not need any lighting adjustments in shot work and only exposure was adjusted on a shot by shot basis.
Compositors dialed-in lensing effects and dynamic flares to taste in almost every shot and adjusted defocus and exposure levels of the very dynamic city. If you look closely, you can even see some people inside some of the CG buildings in a handful of shots. These were Rodeo artists shot on greenscreen that were composited into the BGs
Executing a photoreal nighttime look is always a challenge in VFX especially when shooting a whole scene on stage against bluescreen. Since we had the whole scene from start to end, we opted with neutralizing all shots so they match one to the other and applied a sequence grade over all of it which locked the look and aesthetics of it without offloading too much of that work to the final DI of the film.
The same assets and elements were used for the scene that followed the bamboo fight where Shang-Chi and Death Dealer fight each other on another side of the building, against a bright animated billboard BG. That scene was enhanced with a CG helicopter, explosions, and CG knives.
During the fights, the director did some impossible camera movements. Can you tell us more about them?
Oh absolutely! The camera work on the bamboo scaffolding scene is mind-blowingly impressive. I think the combination of handheld cameras with wider lenses, shooting these insane choreographies without resorting to fast cuts is really what makes it an immersive experience for the audience. All credits goes to the director, cinematographer, stunt coordinator and editors! In terms of VFX, the only things we did was sometimes going a bit wider / further away from the action in some shots, and adding subtle imperfections and handheld motion to some of the setups shot with a techno-dolly to give them the same human imperfections as the handheld shots.
How did you create the various digital doubles and especially Shang-Chi and Katy?
Creating digi-doubles is something very common in our industry, especially for action sequences. We built the Death-Dealer digidouble, and the Ten Rings soldier(s) shared them with the other vendors and received the digi-doubles of Shang-Chi and Katy from other vendor(s) as well.
For Shang-Chi and Katy, we updated their wardrobe to match the ones in our sequence. They’re standard digidoubles, modeled and rigged in Maya, Textured in Mari, CFX / Groom in Houdini, and Lookdeved in Katana
Can you tell us more about the various CG weapons?
We had a lot of fun with the CG weapons. First of all, the Ten Rings soldiers all had hooked-swords that ended up being fully replaced in CG since we were adding a flickering blue electrified energy FX to them. Nathan Arbuckle (FX Supervisor) and his team created the effect and procedurally applied it to every single hooked-sword we see. One small detail that was added which helps a lot with their look was to alter their behaviour as the weapons would get close to objects to have them become brighter and more intense and even start sparking and creating electrical arcs. It made them look a bit more unstable and dangerous.
The other weapon we had the chance to work on was Death-Dealers explosive knife. We get to see it on 2 different occasions and in both they explode, creating a fully CG explosion sim as well glass and bamboo shattering.
Which shot or sequence was the most challenging?
The Bamboo scaffolding fight scene was an extremely challenging scene to build and put together. Every single department had to be at the top of their game.
Is there something specific that gives you some really sleepless nights?
I guess it would have to be some of the render-times of massive environments like this! But i still slept really well, thank you for asking
What is your favorite shot or sequence?
I’m really proud of the teams’ work on the Bamboo scaffolding escape as well as the fight between Shang-Chi and Death-Dealer. They look spectacular. It’s somewhat unfair to single out a specific shot from each sequence but.. If I had to, I would say the long shot where Shang-Chi runs from one end of the building to the other, in one uninterrupted long shot and grabs Katy’s hand as she falls down. It’s an insanely cool shot with jaw-dropping parkour-like vertigo-inducing choreography. I’m also a big fan of a full CG wide- shot of the helicopter approaching the building as the window blows-out with a massive explosion. Oh and also the other 153 shots we did ! 😉
What is your best memory on this show?
Honestly, seeing the progress of these assets and shots on a daily basis was a great experience. The dedication and talent of the artists, supervisors, and production team was very inspiring!
How long did you work on this show?
Well, with the insane lockdowns, shutdowns, schedule “stretching” and changes, I think we spent close to a year and a half on this show!
What’s the VFX shot count?
Our final shot-count was 155 shots.
What was the size of your team?
We had close to 130 artists who worked on this show.
The size of production team was about 12 people.
What is your next project?
My team and I are already working hard on our next shows, which I can’t disclose as of yet 😉
A big thanks for your time.
WANT TO KNOW MORE?
Christopher Townsend: Here is my interview of Overall VFX Supervisor Christopher Townsend about Shang-Chi.
Rodeo FX: Dedicated page about Shang-Chi on Rodeo FX website.
© Vincent Frei – The Art of VFX – 2021
