In 2013, Adam Rowland explained the work of NVIZ on CAPTAIN PHILLIPS to us. He then worked on various films such as LEGEND, LIFE, THE COMMUTER and JURASSIC WORLD: FALLEN KINGDOM.

How did you get involved on this show?
I was contacted by Nviz, who in turn had been contacted by Point Grey, Seth Rogen and Evan Goldberg’s production company. They were in pre-production for Pickle and were looking for someone to supervise the VFX on set and through post. They had seen and had liked LEGEND, and were keen on bringing someone onboard who had worked on a one-actor-playing-two-characters production and could guide them through the process.

How was the collaboration with Director Brandon Trost?
Brandon is an excellent collaborator. As a director, he was keen on locking down a plan for each scene before shooting, but was also not too precious with the material to prevent things from changing when required. As an ex-DOP, he obviously has a very keen eye, so had quite specific ideas about how certain sequences should look – I was there to facilitate that as best as I could, while steering the shoot gently through any VFX pitfalls I was able to foresee.

What was his expectations and approach about the visual effects?
There were extensive storyboards for the majority of the the film. I was able to assess these with him, explaining how best we could achieve the desired effects, and how we might go about shooting the various elements required. He was was very open to finding creative solutions to tricky problems, and some of the scenes were even rewritten with our VFX requirements in mind. His vision and style of film-making for Pickle was quite traditional, which fortunately lends itself to facilitating the kinds of effects we were attempting within the budget available to us.

Seth Rogen is playing the two main characters. How did you use your experience on LEGEND for this movie?
I learned most of what I know about this kind of VFX work on LEGEND, a lot of which I learned after the shoot – deep into post, thinking how much easier it would have been had we done things slightly differently on set. I remember some very challenging shots, some of which needn’t have been so challenging, and was keen on bringing this experience to Pickle to avoid any such instances, if possible.

It would therefore be tempting to say that Pickle was easier – but that’s obviously not true. Ultimately you learn a set of tools on one production, and over the course of the project, through shooting and post, you learn which tools are best suited to what kind of situation. You establish what to look out for in each scenario, what material you need to acquire for each particular technique, and what the limitations are. However, there is always something surprising on the day – so the experience of having been through a similar process before is mostly beneficial in providing the confidence to make quick, concise decisions.

What are the main changes you did from LEGEND for the shots with the two Seth Rogen?
In LEGEND, aside from any face-replacements (which were either distant or very brief action shots), in almost all instances where Tom Hardy appears as both of the Kray twins, we were able to shoot the two sides of the split consecutively. We would shoot Tom as one twin, he would go to change and prepare, then we would shoot the second half – often without having moved the camera – with Tom responding to his previous performance in an earpiece. The blocking of the scene, and who was interacting with what or whom, would determine which twin was shot first. All of these scenes were shot in controllable conditions indoors.

With Pickle, Seth played Herschel as bearded and Ben as clean-shaven. False beards, even the expensive ones, are often not very convincing, so it was decided to shoot all the Herschel scenes first with a real beard, shave Seth, and then shoot all the Ben scenes afterwards. This meant that not only were we unable to lock-off cameras, but we were often scheduled to shoot the second half of a split over a month after the first. We essentially shot an entire version of the film with Herschel, and then went back and shot it again with Ben. The interaction between the two characters was also more complicated than LEGEND, where we were often able to contain the twins on separate sides of the screen. In Pickle, they don’t just exchange glances, they clink glasses, slap each other on the back, pass things to one another etc. This necessitated an additional bleep-track for Seth to listen to in his earpiece, along with his previous dialogue, in order to sync certain movements and interactions. In these cases, the timing and choreography of his performance was integral.

An extra layer of complexity was that a lot of these scenes took place outdoors, sometimes on quite difficult terrain in quite changeable conditions. The seasons where we shot in Pennsylvania shift pretty quickly, so it’s a miracle of scheduling that we were even able to align the lighting, let alone the action.

Can you explain in detail about the shooting and creation of these kind of shots?
We had to be quite meticulous in planning all of the split-screens working very closely in collaboration with the camera crew. We used UV pens to mark the floor of the set, denoting where the camera had been for which setup – height, angle etc. On location, we would use more weather-proof methods, such as bolts screwed into pavements, spraypaint, and stakes in the ground. Upon establishing camera location, it would then be up to the camera team to get the position as accurate as possible, which they were able to do by using QTake to overlay a live feed with the previously shot split. It cannot be overstated how difficult, frustrating and time consuming this could often be – and it is entirely down to John Gulesarian (the DOP) and his crew that some of these work at all. Once camera position was established it was a case of re-blocking the scene to ensure no unwanted intersections would occur. Various armatures would often be built for eyelines and to lock other physical interactions to specific 3D locations in space.

Can you elaborates about the village and its environment creation?
The fictional village of Schlupsk was shot at a location called Hartwood Acres in rural Pennsylvania. It’s a collection of outhouses and stables designed in a faux-period style set in extensive forest and parkland. Bill Arnold, the production designer, was able to create additional facades to the buildings to allow for the creation of a more substantial main-street, market place and synagogue. VFX was only required on the wide establishers of the village and surrounding countryside – where we added in mountains, extra period dressing and removed any unwanted modern detail.

The village is seen in two versions. How did you create the most recent with the factory?
All versions of the village establisher are quite extensive digital matte-paintings. The main buildings and trees are real, but most of the foreground barns, outbuildings, mountains etc. were added in post. There are essentially 3 variations of the same establisher in the film: Pristine 1919 Schlupsk, Destroyed 1919 Schlupsk and 2019 Schlupsk. Christoph Unger, the concept artist and matte-painter at Nviz created the first one, leaning heavily into the aesthetics of films like Yentland Fiddler on the Roof. Once Brandon had approved this, Christoph and the team at Nviz were then able to build a destroyed version – adding smoke, fire, dead animals, and other post-Cossack-attack details.

It was important to Brandon that when Ben returns to Schlupsk in 2019 that the town be instantly recognisable, but also modern – and not necessarily in a good way. The establishers are all shot from very similar angles, so for Christoph, it was a case of creating an entirely new matte-painting based on the layout of old one, updating the early 20th-century shtetl to a more post-industrial, dilapidated Soviet look. We were keen to mirror as much as possible from the original 1919 version, so a barn becomes a gas station, a beautiful forest and mountain becomes a factory and a strip-mine, and a cart becomes a burnt-out car. We were even able to replace the classic rural idyll of a chicken crossing the road with a slowly drifting plastic bag. These are all small self-referential details, but they’re there for people to spot.

Can you explain in detail about the old New York shots?
Pickle is ultimately an immigrant story, so it was very important that the approach into New York, passing the Statue of Liberty and arrival at Ellis Island be included. It’s a significant symbol of America, but also of the freedoms America represented to the people fleeing persecution in war-torn Europe in the early 20th century. It’s about hope and rebirth and possibility, but also the American dream of prosperity and success – which are all recurrent themes in the film.

Brandon and John had decided to shoot the 1919-era material, including the scenes in Schlupsk, on an antique Petzval lens, which has a beautifully soft, dreamy vignetting quality. It was also decided that this section of the film would be displayed in the squarer academy-ratio. Because of these inherent stylistic impositions, we decided that much of the 1919-era VFX, including the arrival in New York, should also have a slightly fantastical fairytale look. The look of the Petzval also had to be faithfully mimicked in comp for all VFX shots, which were filmed on standard primes. The prow of the ship was a partial set-build shot against greenscreen, and Nviz created 2.5D matte-paintings for the backgrounds of Liberty Island and the views of the Hudson Bay.

Interiors of Ellis Island were shot on a set, which was extended upwards by Nviz to include the iconic windows of the immigration centre. Some of the reverse-angle shots inside the immigration centre were shot with people’s badges and signage reversed, then flipped in post. This was Bill’s neat solution to save on studio space and lighting time, and remove the need for more VFX set-extensions, as it only required two of the four walls of the room to be built.

What kind of references and indications did you received for the village and the old New York?
As mentioned, the 1919-era Schlupsk was very much inspired by 1970s films like YENTL and FIDDLER ON THE ROOF (with perhaps a touch of MONTY PYTHON AND THE HOLY GRAIL). These are both clear visual references to not only the buildings and costumes, but also the look of Herschel himself. For modern Schlupsk, we looked at less pleasant Soviet-era towns, and other areas in Eastern Europe that had transformed under Stalin quickly from rural agricultural societies to heavy industrial cities and then into high-unemployment, post-industrial wastelands. It was important that Schlupsk in 1919 be simple and charming, but in 2019 be a significantly less appealing place. The 1919 approach to NYC sequence is somewhat of an homage to similar scenes in films like THE GODFATHER: PART 2. Brandon had found a lot of turn-of-the-century black-and-white photographs of the New York skyline that he wanted to replicate. Nviz were able to draw from these as well as many others in order that the shots of Manhattan were not only period-specific, but also geographically accurate – while maintaining the fairytale aesthetic.

Can you elaborates about the timelapse shot of the Pickle factory?
This shot is the big moment early on where the film transitions from 1919 to 2019, and from 1.37:1 aspect ratio to full 1.85:1. It moves forward 100 years in time, presenting not only a time-lapse of the evolving Manhattan skyline in the background and the slow demise of the pickle factory in the foreground, but also illustrating 100 years in the technical evolution of photography and cinema. It starts in a somewhat grainy monochrome square format, and as time passes, moves through the advent of colour celluloid, mid-century technicolour, video, and into the crisp widescreen digital realm.

I pitched this idea to Brandon, and as a DOP he liked its visual integrity – which we further developed to include the passing seasons of one year. There is a deliberate stop-frame feel to it, rather than a slick series of smooth transitions – buildings don’t grow, they shift through various states. Brandon was quite keen on maintaining the photographic realness that you would get if you were able to simply leave a stills camera mounted on the side of a building for 100 years. It’s only really the organic aspects – the weather, the trees, the river and skies – that don’t obey this rule, but a few tests were done early on and we felt having some sense of continuity gently animating against all the stop-motion prevented it from being too stroboscopic and helped ground the shot.

The foundation of the shot is the exterior plate of the pickle factory, which was shot in Pittsburgh. Most of the foreground, at least in the 2019 section, is in camera. Nviz took this and worked backwards, blocking out the century into 8 year slices, which allowed for about 12 different stages, and months of the year. They used traditional matte-paintings to create the changing buildings and declining industry in the background, and Maya and Speedtree for some of the foreground elements. Once the shot was blocked out in terms of basic timing, the different seasons and skies were added with all the extra subtle layers of detail being dressed in also. It is definitely one of the biggest shots in the film with hundreds of separate elements, and was worked on for an enormous amount of time to get right. Kudos to the compositor, Agueda del Castillo, for sticking with it and doing such a great job.

How did you choose the VFX studios?
The job came to me via Nviz because of the work we had previously done together on LEGEND. We continued that partnership on this show, as I knew them well and knew that they were familiar and experienced with this kind of VFX work. I have always found them great to work with and especially resourceful with limited budgets.

How did you split the work amongst these vendors?
Nviz were the main vendor and were involved from very early on.
Mammal Studios and FuseFX, both in LA, were brought in over the last few months of post, to help with some of the overspill. The edit was based in the Point Grey offices in West Hollywood, so it was very useful to be able to quickly ship shots out to local vendors and be able to communicate with someone in the same timezone.

Can you tell us more about your collaboration with their VFX supervisors?
Nviz were able to send out one of their supervisors, Jason Evans, for the duration of the main shoot. This was very useful, as it meant that when he got back to Nviz, he had intimate knowledge of what was shot, and could help the production team structure the schedule of the show based on what he knew would be coming. He ran the team in London for the entire duration of the project, which was very helpful. It was a totally transparent process between us, which allowed for a much more collaborative experience.

When embedded with editorial in LA, I visited both FuseFX and Mammal, and communicated with them regularly. They were also available for some of the sessions in the DI when we were putting the finishing touches to the movie in the last few weeks, which was very useful.

Which sequence or shot was the most challenging?
There’s a few I can think of. One in particular was very tricky to choreograph, as it was an exterior spit-screen with a lot of interactive elements, which we shot in the forest in two sections over a month apart. In it, Herschel and Ben team up to push over a stack of logs that are blocking their way. The Herschel side was shot first, with the branches rigged to fall when Seth pushed them – he was acting alongside a tennisball on a C-stand for an eyeline. When we returned, the season had moved from autumn into winter and most of the leaves had changed colour. We had to establish the camera position in amongst the undergrowth, find and re-rig the branches, use a travelling greenscreen and have Seth hit all his marks – all the while the light was fading. Using QTake for setups like these, where you can immediately view a take in the context of a rough composited shot, was invaluable.

Did you want to reveal any other invisible effects?
It’s something that people may not notice, but there is an enormous amount of graphics work in the film. Almost everything that appears on a screen, iPad or phone, was done in post. The character of Ben is an app-developer, so a lot of the graphics are significant plot points, where important information is revealed or a narrative strand is set up. In comedy, it’s not unusual for the edit to change quite frequently, which often meant the story being told by each graphic was not always the same one cut to the next. This meant that we had to always be on our toes and it had to be quite a fluid process.
The Nviz graphics department, headed up by Chris Lunney, drove the design side of much of this, and were even given the control to create opportunities for extra moments of humour. Fortunately they’re a funny bunch!

Another shot I like isn’t really invisible, but I do think it’s low-key enough not to draw attention to itself too much. It’s a sequence within which both Herschel and Ben are walking through the cemetery. It’s a steadicam split-screen with the two of them walking towards camera. We shot a series of rehearsals and reference plates on location, then mapped out their trajectory, establishing the walking speed, and then later shot Seth in two sessions on a treadmill in front of a greenscreen. I think it’s a really cohesive scene – Nviz did a great job pulling all these elements together.

Is there something specific that gives you some really short nights?
All face-replacements stress me out. I have done enough of them to know when they will work well and when they will not. We were blessed on Pickle to be given ample time to shoot facial elements for these, with John Gulesarian being available to light each setup appropriately, and Seth finding the whole process quite fun and hitting all his marks well. I think with these, if you can get the lighting and the action right, you are 95% of the way there. So at the end of the element shoot, we were all happy we’d got what we needed.

Anyway, there are a few face-replacements in the film which were not planned for, and so we had no shot-specific elements. Snuck in by the editor and director after the shoot, these are never easy and can’t ever look as good as they would had you been able to plan properly and shoot bespoke material for them.

What is your favorite shot or sequence?
The rats, for sure – they’re so ridiculous! Jason found a Frank Herbert book from the 1970s which had a grotesquely evil rat on the cover and Brandon instantly fell in love with it. They were designed to look like that, halfway between photoreal and cartoonish. I pushed the design of the main rat to look like General Roundwort in Watership Down, a reference that nobody in the US understands apparently – but that Brandon approved of because of the mange and the cataract. I think it’s a really nice sequence, and it doesn’t take itself too seriously – like the film.

What is your best memory on this show?
It was a really nice crew, and a great opportunity to work in the States, both on location in Pittsburgh and in post in LA. We are very lucky to be able to travel with work. It was also a unique experience for me, so far, to work on a comedy and see first-hand how these things are constructed piece by piece. The shoot was a lot of fun.

How long have you worked on this show?
About 18 months, all in. The whole show went on hiatus for several months over the summer of 2019, as they were planning to shoot some additional material and had to wait for Seth’s beard to regrow.

What’s the VFX shots count?
Just under 400, I think

What is your next project?
I am currently Associate VFX Supervisor on a Netflix show called THE IRREGULARS.

A big thanks for your time.

WANT TO KNOW MORE?
NVIZ: Official website of NVIZ.
HBO Max: You can now watch AN AMERICAN PICKLE on HBO Max now.

© Vincent Frei – The Art of VFX – 2020