Mark Stetson has a long career in special and visual effects. He started working on the models of films such as BLADE RUNNER, GHOSTBUSTERS, 2010 or DIE HARD. THE FIFTH ELEMENT is his first film as Overall VFX Supervisor. In 2010, he joined the teams of Zoic Studios as Senior VFX Supervisor.
What is your background?
I studied industrial design in the 1970’s at University of Bridgeport and Art Center College of Design. Between those two college experiences I spent a year as an apprentice industrial design model maker for General Electric Company. While attending Art Center, in 1977, I went to see STAR WARS in 1977 on opening night at the Chinese Theater in Hollywood. I had my epiphany then – I recognized that the visual effects work in that movie was something I had been trained to do. Within a year (in May 1978) I was working for Jim Dow at Magicam as a model maker on STAR TREK: THE MOTION PICTURE. I made my reputation in miniature effects on such movies as BLADE RUNNER, THE RIGHT STUFF, 2010, GHOSTBUSTERS, DIE HARD, TOTAL RECALL and TRUE LIES. Then in 1995 I moved up to overall visual effects supervision starting with THE FIFTH ELEMENT at Digital Domain.
How did you get involved on this show?
I have been working with Zoic Studios since 2010. I met VFX Producer Monette Dubin before I started working at Zoic, sometime around 2009 when I was working with our mutual friend Mike Chambers on SURROGATES. Zoic contributed visual effects to two of Monette’s films prior to CROUCHING TIGER. Monette asked Zoic and me if we could take that creative relationship a step further, and with the approval of the Weinstein Company executives, Zoic Studios loaned me out to the production and I oversaw all the visual effects for the film, including the work by Pixomondo, Lola Visual Effects, and Zoic Studios of course.
How was your collaboration with director Yuen Wo Ping?
I love Master Yuen! He was focused, of course, on the martial arts and the spectacular wire stunts that have become his signature style. It was interesting to me to learn how he works and find ways to maximize the use of visual effects within that context. The production set up a dojo for Master Yuen’s stunt team on the studio lot, and he started choreographing, rehearsing and editing to create live-action pre-vis. Sometimes stunt previs led the CG previs, and sometimes CG previs led the stunt previs. Master Yuen was quite comfortable directing both the the stunt previs sequences and the CG animated previs.
What was his approach about the visual effects?
Master Yuen wanted this movie to be first and foremost a martial arts movie. We both agreed that with the wire stunt action and fight choreography, the visual effects would mostly serve to extend stunts into the realm just beyond the physically possible, and only when he couldn’t achieve a stunt within the context of time and money. Zoic and Pixomondo each had a handful of CG stunt double shots in their workload. We worked hard to emulate the weightless grace of the “light body qi” in our digital animation. Some of those shots transitioned from CG stunt double to live action wire stunt within the shot. In some ways those shots were easier to achieve because the reference to the wire-stunt style was so explicit. Beyond that, visual effects focused on creating the feeling of 19th century China, starting with our sets and locations in New Zealand and Beijing.
Can you describe one of your days on-set and then during post-production?
In prep, I worked with the art department to lay out green screens and help place sets. We also started up a pre-vis team from Proof, Inc. who focused on the Frozen Lake fight, the Pagoda Fight, and several smaller action sequences.
My on-set work was mostly spent with the stunt teams and the camera teams, working out the most efficient way to minimize VFX cleanup and make the most use of VFX to enhance the action where necessary. I worked with two data wranglers for the New Zealand shoot, Emily Roper and Ryan Harvey. When second unit was working, sometimes we would have a second VFX Supervisor on-set, Tim Capper. Data wranglers were tasked with grabbing all the pertinent camera data and also shooting HDRI sets for the various CG set extensions and stunts.
In China, we turned to Pixomondo Beijing, who were able to provide us with data wranglers, a VFX still photographer and a second on-set VFX Supervisor.
The movie features many beautiful landscapes and environments. How did you design and create them?
I worked primarily with Production Designer Grant Major to develop the look of the environments. For Ancient Peking, I researched old photographs and paintings of the Imperial Palace, the city walls, the rural countryside and landscapes. Grant’s art department had done likewise and between us we assembled a nice collection of research images. We found amazing photographs of Beijing from the beginning of the 20th century, only 100 years after our story took place. When I was in Beijing on the first scout and later during the shoot, I visited the Imperial Palace, the ancient City Gates and sections of the ancient city walls that remain standing. They are very beautiful and we tried to include a lot of that richness into the shots of the city. We also looked at the work in CROUCHING TIGER, HIDDEN DRAGON to find ways to maintain some visual resonance between the two.
For the landscapes, I asked the matte painting teams at Pixomondo and Zoic Studios to look at ancient Chinese watercolor landscapes, and try to keep that feeling in the wide scenes. We got a fair amount of that look into the environments, especially the opening scene, which Pixomondo created. Grant Major’s illustrators set the look of the Frozen Lake early on and we followed his lead through the development of the matte painted backgrounds. I wanted to get a sense of willow trees along the shore, again reflecting traditional Chinese art. Pixomondo did a great job of that environment. The art department built several sets of the Pagoda and also built the base of the Pagoda with the West Lotus encampment at Muriwai Beach along the western coast of New Zealand’s North Island. We also shot some aerial approaches to the Pagoda on the South Island. Zoic Studios had the task of creating an environment that merged all those conflicting elements together. Into those shots, the matte painting team at Zoic Studios also managed to work the theme of Chinese watercolors into the shots.
What kind of references and indications did you received from Yuen Wo-Ping for the environments?
In prep, Master Yuen worked a lot with Grant Major in development of Sir Te’s compound, the path of Yu Shu-Lien’s journey from seclusion into the city, and many other story issues. One of Master Yuen’s Associate Producers provided research images that really helped guide the look of Ancient Peking. Master Yuen also directed previs shots of the approach to Peking.
Can you explain in details about the creation of the huge City environment?
Modern Beijing is still dominated by the Imperial Palace, now a museum. In the time of our story, it was the center of all China. So working with the geography of Ancient Peking begins with the Palace. Many of the landmarks we chose to use in our paintings still exist. Grant Major’s art department spotted Sir Te’s Estate relative to the Imperial Palace and we looked at city landmarks to help guide our shots from there. Pixomondo Toronto’s matte painting team used a lot of CG modeling and modular building forms to create the city.
This city is seen in various light conditions. How did you handle this aspect?
Pixomondo created separate paintings for day scenes and night scenes. The use of modular CG building forms helped reduce the workload, but there was a lot of brute force involved.
How did you create the various CG weapons?
We built CG versions of the Green Destiny and the Hellfire Blade, as well as Mantis’ double headed spear and a few other spears, blades and arrows. Most of these started with scans of the hero prop weapons. The two hero swords were beautiful props, designed by Grant Major’s art department. Plowman Craven Limited provided scanning services on location in New Zealand.
Donnie Yen is fighting on a frozen lake. How was this sequence filmed?
The art department built a small set with an ice surface made of acrylic and poured polyester resin. It measured maybe 15 by 20 meters. We extended the set with VFX to become the lake we see in the movie. Master Yuen and I spoke about ways to take advantage of the low-friction surface in a martial arts fight, and he came up with many ideas that made up the final choreography. We brought a new previs team into the development, working at Proof’s home offices in Los Angeles under Supervisor Monty Granito. I had a funny facetime session with Monty one Saturday. To explain the storyboards, I was acting out some of Master Yuen’s ideas dancing spastically around my apartment 8,000 miles away from the previs team. Monty got it, however, and Proof’s first round of previs for the sequence pretty much convinced Master Yuen we could make something special out of the Frozen Lake Fight.
Master Yuen shoots his sequences in small bites, shot-by-shot, to make the wire stunts achievable. He had a list of close to a hundred shots when we began filming. I think we had 11 days on that set across two units. The art department worked hard to create an ice surface suitable for stunts. The art department was totally on top of maintaining the ice surface with bags and bags of delicate snow dressing, renewed for every take. Then in post-production, Pixomondo’s VFX team replaced the surface with CG or subtle matte paintings where cracks appeared or where Wei-Fang breaks through the surface, or wherever the ice surface is moving or chunks are drifting, or wherever we had multiple light source reflections… you get the idea. Wire stunts took place on the ice surface, using heavy construction equipment to cantilever the wire rig points over the set. I really enjoyed the shots of Iron Crow and Wei-Fang pushing Silent Wolf back across the ice.
The fighters are using the ice as weapons. How did you created these elements?
This idea came from Master Yuen. I pitched the idea to the previs team at Proof, Inc. and they developed the shots into a preliminary sequence. Then Master Yuen made a few rounds of adjustments and started working with his stunt team to refine the choreography and figure out his stunt list. The stunt rehearsal previs for this sequence was my favorite part of the stunt teams work on the movie. The team used roller blades and cut foam pieces to simulate the low friction and the ice chunks. Ultimately, of course, the ice chunks were all CG, executed by Pixomondo.
There is a beautiful underwater shot. Can you tell us more about it?
There was a lot of fretting about the underwater shots during prep, for reasons of cost and safety. Ultimately the needs of the story were acknowledged – to pay off the imminent danger of the fight on the ice, one of the opponents must fall through. Ultimately the art department was only allowed to build a small tank under the ice surface, barely 16 inches deep. Harry Shum is so athletic and so physical that we managed to get shots of him as Wei-Fang submerging in that shallow tank, and then the wire stunt of Silent Wolf pulling him out. I really admired him those days. Other shots of the ice cracking underwater were executed by Pixomondo Toronto entirely in VFX, using CG, matte painting, and comp tricks.
How did you created the digi-doubles for the final sequences?
Digi-doubles of our heroes were created for Wei-Fang’s escape over the rooftops and in the Frozen Lake Fight, and for Silent Wolf, Yu Shu-Lien and Hades Dai for about ten shots in the Final Pagoda Fight. The digi-double of Yu Shu-Lien was also seen in two shots in the Frozen Lake Fight. Plowman Craven Limited performed scans for the characters on location in New Zealand. Zoic Studios completed the modeling of the four main characters and then shared them with Pixomondo Toronto, sans hair and cloth setup. Zoic and Pixomondo used different methodologies for hair and cloth sims, and both Pixomondo and Zoic had shots in which they executed transitions on screen from practical costumes and hair to digital doubles. Both did a great job in matching cloth and hair poses across the transitions.
Pixomondo Toronto also provided background digital doubles for several of the wide city views. We shot elements in China for the mid-ground market shoppers and the mounted soldiers, but beyond that, the deep background crowds were all digital. Pixomondo re-purposed some existing assets to create these and really gave us a lot in terms of variety, flexibility and editability.
During the final fight, we can see swords causing destructions in close-up. Can you tell us more about these shots?
The shots featuring the destruction of the Pagoda in close-up were all planned shots. Master Yuen wanted to show the power of the swords in a progression that led up to the Green Destiny slicing through solid rock. Starting in the Pagoda interior throne room, the shots of the Hellfire Blade slicing a chunk out of the beam was done with a prepared wooden post with the chunk already sliced out, to which Zoic Studios added the digital chunks of debris. Also, the shot of Wei-Fang slamming against the window frame, cracking the stone blocks, required Zoic Studios to add the cracks and fine debris to the stunt. In some cases, Master Yuen tried some practical tricks, and the shot of Hades Dai’s Hellfire Blade gouging the Pagoda wall remained a practical shot. The matching shot of the Green Destiny sword gouging the wall was finally executed as an entirely CG shot.
Late in the sequence, when Hades Dai and Silent Wolf crash down through several levels of the Pagoda Roofs, the shots were executed entirely in CG, with cloth and hair sims for the stunt doubles and particle sims for the Pagoda destruction all rendered together to get the most interaction.
Michelle Yeoh is fighting against the Blind Enchantress who uses fire and smoke during the fight. How did you create these FX elements?
This was a troublesome sequence for Master Yuen. He had only been given one day to shoot the fight between the Blind Enchantress and Yu Shu-Lien, and he was unhappy with the sequence being so truncated. He and the creative executives wanted a more important moment for Yu Shu-Lien at this point in the story, and Eugenia Yuan, the actress playing the Blind Enchantress privately expressed her disappointment with a fight that came down to one beat. So the visual effects team really helped elevate this scene.
First, a day of second unit work was scheduled to add a few more beats to the fight before the Enchantress’ final, failed, trick. At that time, the green flame elements were shot, provided by Special Effects Supervisor Brendon Durey’s team led on-set by Steve Yardley. We shot an element for the reflection in Yu Shu-Lien’s eye. The POV of the brazier with the flame burst is a practical effect. Then, the dropping curtains were prepared executed practically by on-set art director Mark Grenfell and texture specialist Sarah Harper-Bailey. This gag was important to Master Yuen, not only to effectively blind Yu Shu-Lien (“Welcome to my world,” whispers the Enchantress) but also as a carry-over from an idea that Master Yuen pitched but couldn’t fit into the budget and schedule, that the fight would take place in a magical void of swaying curtains.
Then, as the fight takes off, we see the Enchantress disappear and re-appear in different parts of the set, as she had earlier in the film in the gnarled woods when we Wei-Fang meets her. Master Yuen was always reluctant to let any magical effects overtake the fight choreography, but I felt like the Enchantress’s appearances and re-appearances were a bit hard to pick out in the dark set and needed some extra flair. We wanted to suggest that the Enchantress’ magic was rooted in stage magic, so the puffs of smoke were meant to evoke that stagecraft too. The smoke puffs were CG sims.
As the sequence was cut together, it didn’t seem too difficult for Yu Shu-Lien to pick out the “real” Enchantress from the three elements we shot. I wanted to clutter the frame to make it appear more confusing. We also wanted to acknowledge the idea that in Chinese numerology the number 4 stands for death. Lead Compositor Patrick Kavanaugh did a little Frankenstein compositing to add a fourth enchantress into three or four shots, while still maintaining her eye lines and her swords pointing in the right general direction. Then he used some additional CG smoke puffs to tie the look into the smoke puffs we used earlier in the fight.
What was the most complicated sequence to create and why?
Probably the Frozen Lake Fight by Pixomondo Toronto and the Final Pagoda Fight by Zoic Studios Vancouver were equally challenging. Each required full-spectrum visual effects solutions, from straightforward rig removals to full CG shots. Each required extensive design development to develop the look of the environments, and each had to integrate the subtle differences in lighting between multiple shots in the sequences
How did you split the work amongst the other VFX studios?
Of course, many factors go into those decisions, and it took a long time for Monette Dubin and me to finalize the work assignments. Simply put, Pixomondo Toronto did most of the VFX sequences in the first and second act, and Zoic Studios Vancouver did the VFX sequences in the third act. Pixomondo did the early establishing shots at Vulture’s Peak, the Carriage Attack, all the wide shots of Peking, the Tea House Fight, the Sir Te Compound Attack, Wei-Fang’s escape from Peking, and the Frozen Lake Fight. Lola VFX focused on cosmetic cleanup work throughout the movie. Zoic did all the sequences that involved the Pagoda environment, including the early scenes introducing Hades Dai, Mantis, Iron Crow and Snow Vase, and then they did all the shots from Wei-Fang’s arrival at the West Lotus camp and the commencement of the Final Pagoda Fight to the end of the movie.
Can you tell us in detail about your collaboration with the various VFX supervisors?
Before this project, I had not met nor worked with Pixomondo Toronto VFX Supervisor Mahmoud Rahnama or VFX Producer Paul King. My main connection to Pixomondo was with Executive Producer Joni Jacobsen, with whom I had worked in the early 2000’s at Sony Pictures Imageworks. Joni had also called me in to Pixomondo Los Angeles on a commercial project in early 2010. So our relationship gave me confidence with Pixomondo as a creative partner. As the job unfolded, I grew to respect both Paul and Mahmoud very much. They were a really hard-working, creative pair and they led a great team. After so many hours on Skype together over the months of VFX post-production, it seemed strange to end our relationship so suddenly, but that is the way of the film business. I still haven’t met them! But I look forward to renewing our relationship on another project.
I had met the team at Lola VFX on prior projects and have great respect for Edson Williams’ work. That assignment was very easy to make.
Because I was loaned out from Zoic Studios to the Weinstein Company for this project, and Zoic has been my home since 2010, I was intimately familiar with Zoic Vancouver’s VFX Supervisor Ralph Maiers and VFX Producers Christopher Elke and Chris Roff, as well as the department supervisors on the show, including Kenton Rannie (Compositing), Richard Patterson (CG) and Rich Mahon (Matte Painting). We have an easy shorthand communication style we have developed on many projects over the last five years.
How did you follow the work progress of the VFX studios?
In post-production, VFX Producer Monette Dubin was based in New York, and I was based in Los Angeles. The Pixomondo Toronto team supervised by Mahmoud Rahnama remained in Toronto and the Zoic Studios Vancouver team supervised by Ralph Maiers remained in Vancouver. During shot development, we communicated primarily using Skype and cineSync, quickly getting into a time zone-based rhythm of daily sessions. Later in the VFX post-production cycle, when Lola VFX came on board under Edson Williams, we continued to use cineSync and Skype with them, even though they are based in LA. It was easier to work them into our tight post-production schedule that way.
Later, as we began reviewing shots for final approval, we used Technicolor New York and Technicolor Hollywood, who provided synchronized 2K screening services between their two locations. That way I could watch the shots in LA, with LA-based Weinstein executives, while Monette and the Weinstein creative executives in New York reviewed the shots there. We had a live telephone connection between the two screening rooms. It worked remarkably smoothly.
What was the main challenge on this show?
For me, much of the challenge was in the design of the film. I wanted to be true to Grant Major’s work on the film, evoke the feeling of ancient Chinese watercolors, remain true to the design of the first film, and convey the romance of the story. It was important to think this way about the visual effects, because for many sequences the VFX shots were the ones that gave us the chance to show all those concepts. I was always thinking about how to nudge the wide shots away from New Zealand landscapes and towards Chinese landscapes.
Was there a shot or a sequence that prevented you from sleep?
I didn’t sleep much in China, that’s for sure.
What do you keep from this experience?
I really enjoyed re-connecting with old associates from my time in New Zealand 14 years earlier on THE LORD OF THE RINGS. I also was amazed by my experiences in Beijing for our short shoot there. I was glad to have been able to walk in the city and explore the ancient city walls as well as the Imperial Palace. Mostly, I enjoyed working with Master Yuen and his team, who really worked like a family.
How long have you worked on this show?
About 18 months, from May 2014 through November 2015. There was a hiatus in there while we waited for all the pieces to fall into place for the China shoot, but I pretty much worked through the hiatus on the show anyway.
How many shots have you done?
There were about 500 VFX shots in the finished picture. Lola VFX did about 65 shots, Pixomondo Toronto did about 335 shots and Zoic Studios Vancouver did about 150 shots.
What was the size of your team?
I don’t have the exact counts in front of me but the teams were not too large. The Proof Inc. pre-vis team in New Zealand was about 8, and in LA about 10. Pixomondo Toronto and Zoic Studios Vancouver each had about 65 people on their crews. Lola VFX and our Production team each had about 15.
What is your next project?
I am currently working in Vancouver for Zoic Studios on a mini-series for ABC Television.
What are the four movies that gave you the passion for cinema?
2001, JAWS, STAR WARS and CLOSE ENCOUNTERS OF THE THIRD KIND.
A big thanks for your time.
© Vincent Frei – The Art of VFX – 2016
