Alex Lemke started his career in visual effects in 1996. He works in many studios such as Cinesite, Framestore and Weta Digital. As a VFX Supervisor, he takes care of the effects of movies like A MOST VIOLENT YEAR, BILLY LYNN’S LONG HALFTIME WALK and PATRIOTS DAY.
Michael Huber started working in visual effects as a Digital Compositor in 2000 and works on many commercials and films. In 2013, he co-founded East Side Effects with Alex Lemke.
What is your background?
Michael initially studied literature and philosophy at the University of Munich / Germany and went to a digital media and film school before he started working as a Flame-Compositor. After several years in the German VFX industry as an Artist and VFX Supervisor, he moved to New York in 2012.
Alexander originally wanted to be a SFX Makeup Artist, but ended up interning at a small optical effects studio in Munich, working on an Oxberry printer and Crass animation stand. He started working at Arri when they opened their digital film department and went freelancing around the world after that. He then moved to NYC in 2011.
How did you get involved on this film?
Alex knew the Coens from being their VFX Supervisor on INSIDE LLEWYN DAVIS. We were first approached about THE BALLAD OF BUSTER SCRUGGS in January 2017. We went through the script with them and designed what we call a “VFX bible”, which outlines how we think certain effects can be achieved. After that, we started collecting references from other films or real life footage.
What was your feeling to work with the Coen Brothers?
Of course we felt honored being able to work with the brothers. We have been big fans of their work since BLOOD SIMPLE. We really like their story-based filmmaking approach. They don’t just add effects for effects sake, but to help tell the story. It also helps that Joel was editing the film himself, so communication was very efficient during this project.
What was their approaches and expectations about the visual effects?
They have been doing movies for so long in their own way that previs never really became an issue. For the Indian battles, we tried to interest them into the Ncam system in combination with pre generated assets, but that is not their way of doing a film. The whole project, like any other film of theirs, is still storyboarded by J. Todd Anderson, who has been their go-to storyboard guy since RAISING ARIZONA. These storyboards gave a pretty good indication of what to expect, but there were still a lot of changes due to the nature of the project such as weather, shooting with animals, and more.
How did you organize the work between you two and with your VFX Producer?
Alex was on-set for the length of the shoot, over a period of three months in New Mexico, Colorado, and Nebraska. We didn’t have rounds or dailies with the artists, but Alex and Mic would rather be available immediately when they needed feedback or had a question. RV, with its shotgun integration, helped us a lot there.
Ron Pogue, our VFX Producer, was mainly focused on keeping the database and budget up to date.
Buster Scruggs did a an impressive entrance with his dust. How did you design and create this shot?
This was one of the nice little visual gags the Coens likes to put into their films! We originally just planned to use 2D elements for this, but realized after a while that this wouldn’t give us the full effect we wanted. So, the actual simulation was achieved by matchmoving his body and creating CG dust in Houdini, which was created by Stephan Galleithner and Martin Matzeder of Keller I/O.
This segment has many gore elements. Can you tell us more about that?
Alex being a big horror fan was certainly in his element here! We added CG and 2D element blood hits – Surly Joe’s shot off face and the fingers being blown off Ike’s hand. Originally, we went really over the top sometimes. For instance, we made a version of Joe’s face just having a giant hole blown into it and blood gushing out of it! It looked really hilarious, and Joel and Ethan laughed when they first saw it, but then we went back to a more toned down version. When he is lying on the floor, we would create digital wounds and track open eyes to his face for a comical effect.
For the fingers being shot off, we painted out the actor’s fingers and added little stumps of bone. Mic – together with Martin Grassl from BigHugFX – developed a Digital Asset in Houdini for blood simulations, which we could then reuse and tweak on a per shot basis, adjusting volume, velocity, direction, and impact. Additionally, we used blood spatter elements and wound elements to enhance the practical make-up.
Later Buster lost his duel and is going literally to the sky. How was this sequence filmed?
Plates of actor Tim Blake Nelson were shot by the main unit, outdoors against blue screen, with him either hanging from wires, courtesy of SFX supervisor Steve Cronin, or by placing him on a moving slant board. Backgrounds were either plates or a mixture of cloud photography and 2.5D matte painting elements.
Can you explain in detail about your work on these shots and especially the wings?
The wings, created by BigHugFX, went through a number of iterations for size and shape. We finally settled on a “swan-like” shape in a medium size. The animation was timed to match the ¾ beat of the song. BigHug added little touches in the animation, like the stretching out of the wings at the beginning, as if they are waking up. The Lyre was added as a CGI element, which we transitioned into the real one from set seamlessly. CG dust and smoke elements helped with the ghost transition.
How did you extend the town and its environment?
We only needed to extend the town in two shots. One of them was when the shot goes high up, looking down on Buster dead in the street while his ghost flies towards us. We shot elements of him being dragged away by the undertakers, as well as the kid riding away on a cherry picker in its highest position. In addition, Alex took a lot of set photographs which we used for our 2.5 D Matte Painting. We played with the height and angle of our camera, a bit, until we found something that Joel and Ethan liked.
To bring the story full circle, we added Monument Valley mesas into the far distance.
In the second episode, James Franco finds himself at the end of a rope. Can you tell us more about your work?
In some shots, actor James Franco only had a noose around his neck, so we added a digital rope based on reference footage and photographs taken on set. We did this to help with the impression that he was strung tightly to a hanging tree.
Other work in that sequence included blue screens and bullet hits (“Pan Shot”!), as well as CG arrows and weapons, and blood effects and dust enhancement for the fight sequence.
One of the chapter is featuring a man without arms and legs. What was your approach about him?
In pre-production, we collected a lot of real life reference footage, as well as previous film examples from FORREST GUMP and RUST AND BONE, to get an idea of what we were going to deal with and how that would affect our approach. We wanted to avoid having to add on CG stumps as much as possible.
Can you explain in details about your work on this character?
The “wingless thrush” – as he is referred to on a poster in the film – was a combined effort of the art department, special effects, costume design, VFX, and, of course, Harry Melling’s incredible stamina. He was performing this poetry while standing in a hole in the ground with his hands behind his back, and went for it take after take, sometimes in the freezing cold. It was clear that 98% of all shots would be painting out his arms and legs, so SFX Supervisor, Steve Cremin, had to devise a way to cut holes into the set and his chair to make it appear he was resting on his stumps. Our costume designer, Mary Zophres, had the great idea of having him wear a regular shirt where the long sleeves were just folded up, which helped with hiding his arms. He wasn’t wearing any blue garment, just black – which helped with not getting any unnecessary color spill in the set. Alex was on set to make sure we would shoot clean plates after each setup. Luckily, the Coen brothers’ approach to these shots really focused on Harry’s performance in long locked off takes, so we didn’t have to deal with a lot of camera motion. We also helped Harry’s look by warping his shoulders closer to his body in some shots.
Later we discover a beautiful place with various animals. Can you explain in detail about the deer?
For the deer in “All Gold Canyon”, the original plan was to shoot a real deer against blue screen, but it became clear that we might not get the very specific actions Joel and Ethan wanted to see. They were referring to the opening of SHANE, which has this great shot of the titular character appearing through the antlers of a deer. So eventually we realized it had to be a digital solution, and we were very happy to get The Mill in New York to work on that for us.
Can you tell us more about the owl and the little fish?
The owl is real!! Just a great acting owl. We shot it against an outdoor blue screen and added a DMP based on a plate and Alex’s own on-set photography. For the wide shots of the owl landing and taking off, we filmed the owl with a witness camera and picked a loopable portion of its flying motion that we used as needed. The little fish and the butterflies were done in CG by The Mill.
How did you enhance this beautiful location?
Most of it is actually real. This was shot in a valley an hour away from Telluride. Production designer Jess Gonchor really found something fantastic there.
We added a mountain range in the background in some angles to make the valley seem even more isolated and removed every hint of civilisation to make it feel more remote and untouched by humans. Since production was on a tight schedule and weather was a big issue, we shot sometimes out of sequence, so a lot of fixes also included continuity issues.
How did you multiply the carriage for the long convoy?
We had 15 full scale Conestoga wagons on set, and Joel, Ethan, and Bruno were very smart about staging their shots. So, there is only a handful of very wide shots where multiplying the carriages was necessary. Since resetting the wagon train took a very long time, we just let the camera run until we had enough footage, and then mixed and matched in post.
Can you tell us how you choose the various VFX vendors and split the work amongst them?
We went with the ones we felt were best suited for the task at hand. For example, we knew Iloura (NOTE: now Method Studios) did the “Battle of the Bastards” horses in that GAME OF THRONES episode and we knew Joel and Ethan liked that work, so we just reached out to them and we were super happy to work with them.
How did you proceed to follow the work of the various vendors?
We used Shotgun to track all the I/O from the respective vendors. Also, we built a proprietary connection to Nuke that handles all our own artists’ interaction and versioning, including automatically applying the correct baselight grade when creating a version. This also allowed us to use the RV-Shotgun integration for reviewing.
Which sequence or shot was the most complicated to create and why?
The most challenging shots turned out to be Liam Neeson carrying Harry in a backpack up the stairs in a brothel, putting him on the ground, and eventually turning him away from the “action” that is about to happen. We talked about different approaches early on. At some point, a rig was considered to help with him being carried up the stairs, but this would have meant an enormous amount of paint work, and not to mention the setup time on a very tight shooting schedule. A CG head might have worked for the stairs, but for the long close up shots of Harry – both over a minute long and only with very subtle facial expressions – it would have been cost prohibitive and maybe not successful in the end.
So, a head replacement seemed like the best solution, but it comes with its own set of problems. In our case, shooting a head element of Harry that would match exactly what the dummy on Liam’s back and ground was doing in the production plates. We came up with a very elaborate set up, where we would track the backpack and dummy in the live action photography in 3D Equalizer to then re-engineer this data into kuper move files that would drive a motion control and motion base combo. Basically, Harry would sit on a computerized motion base that would do the turning motion, so he could react to being pushed around, while the Motion Control camera would take care of all the translations. This also meant that our DP, Bruno Delbonnel, had to create animated lighting for the staircase shot to make the head element really sit in the plate.
We worked with Pacific Motion for the motion control – Mike Leben was our operator – and NAC effects for the motion base – Nic Nicholson took care of this. Special thanks goes out to Christoph Gaudl for his Camera Object Tracking, Stefan Galleithner for taking on the task of converting all that data into something the camera and base would understand, and Kelly Chang and Mike Viscione for on-set Maya support.
Of course, you only get an element that works 80% of the way – the rest was laborious compositing work. Since we put the motion base to its speed limits on the staircase shot, we actually had to shoot half speed and then speed it up in post, which meant a lot of warping/tracking was need to make sure there was no slippage. The dummy we used for the live action photography also didn’t have any breathing movement in it, so we used parts of Harry’s blue screen plates as a guideline of how his chest should move. These tricky tasks were expertly performed by our artists Danica Parry, Euna Kho, and Sabrina Tenore.
Is there something specific that gives you some really short nights?
Just the usual – so many pixels, so little time. But, it’s all worth it if the end product is so great!
What is your favorite shot or sequence?
Mic: Harry’s head replacement in the brothel and CG-Horses in Gal and Mortal Remains.
Alex: Definitely Harry’s head replacements in the brothel sequence.
What is your best memory on this show?
Mic: Discussing blood and gore levels with the Coens in the review sessions was quite a bit of fun.
Alex: Lots of good ones. For me, probably shooting the photo plates for the opening shot in Monument Valley. I went with my family on a weekend from Santa Fe and we stayed overnight nearby in Valley of the Gods, in a very old homestead with no civilization around. The starry sky at night was unbelievable.
How long have you worked on this film?
Including Set-Supervision and Pre-Production, about a year total.
What’s the VFX shots count?
In the end, 704 shots had digital effects in them. This has to be a new record for the Coens. Joel at one point jokingly called it their “Marvel movie”.
What was the size of your team?
East Side Effects had a team of 16 artists in-house.
What is your next project?
Currently, we’re working on Ang Lee’s new film GEMINI MAN – 4K 120fps Stereo – and we’re doing some work on JC Chandor’s new film TRIPLE FRONTIER.
What are the four movies that gave you the passion for cinema?
Mic:
A CLOCKWORK ORANGE (Stanley Kubrick / 1971)
DOWN BY LAW (Jim Jarmusch / 1986)
BONNIE & CLYDE (Arthur Penn / 1967)
THE GODFATHER (Francis Ford Coppola / 1972)
Alex:
I have always been a big horror fan, so …
VIDEODROME (David Cronenberg / 1983)
DAWN OF THE DEAD (George Romero / 1978)
THE THING (John Carpenter / 1982)
INFERNO (Dario Argento / 1980)
A big thanks for your time.
WANT TO KNOW MORE?
East Side Effects: Dedicated page about THE BALLAD OF BUSTER SCRUGGS on East Side Effects website.
© Vincent Frei – The Art of VFX – 2018
