In 2022, Andreas Giesen explained to us the visual effects work created by RISE on The Matrix Resurrections. He talks to us today about his work on a very different project, The Last Voyage of the Demeter.
How did you and RISE get involved on this show?
We got involved quite early, basically even before the principal photography started. That’s why our Lidar-Team and me also had the chance to be on-set in Malta, when most of our scenes were shot. That was a great start to get into the project and we could make sure to get all data that we needed to recreate the Demeter and the harbor environment as close as possible to the set build.
What was your feeling to enter into this iconic franchise?
It was quite exciting as there are so many iconic movies out there already. I also liked the idea that the story in the movie was just focusing on this one chapter called “The Captains Log” in the book and the eventful passage.
How was the collaboration with Director André Øvredal and VFX Supervisor Brad Parker?
To work with Brad was a real pleasure as his notes were really on the spot and I always had the feeling that we were completely on the same page to push a shot to the next level. André had a clear vision for the shots but was also open to new ideas. This was great and resulted in a very creative process on our side, which I really enjoyed.
How did you organize the work with your VFX Producer?
Working with Iva Modrah was just great, as we supported each other as good as possible. I could be sure that she, Josephine Ross and the rest of the production team were taking care of every aspect like scheduling, resources and so on so that we could focus on the creative and technical challenges that this project offered.
How did you split the work amongst your and the RISE offices?
We worked on this project in our Berlin and Stuttgart offices, but I have to say it felt more like one big team. As we had two compositing supervisors, Patrick Kreuser in Berlin and Philipp Mekus in Stuttgart both had individual sequences assigned, but there was still a lot of exchange and we had just one big daily session, looking at shots from both offices. Also the CG and FX teams were not really splitted.
What are the sequences made by RISE?
In total we touched about 22 sequences in the movie from the very first shot until the last one. The most complex ones were the wrecked Demeter on Whitby beach, the city of Varna including many shots in the harbor and several full CG shots showing the Demeter on the open ocean sailing into the sunset or also in a heavy storm. In addition to those we also had three sequences with burning people, partly with full character replacements including fire and smoke effects.
How did you create the harbor and its mountains environment?
The harbor is based on actual old buildings that are part of the Malta Film Studios. As there are just a few real ones we build the rest of the harbor based on lidar scans and a lot of references that we collected of harbor cities in the end of the 19th century. For the layout we used a procedural approach in Houdini to populate the docks with thousands of props like crates or baskets. As they had nearly 100 extras on set we did photoscans of about 10 individual ones, that we used later to populate the city with a CG crowd, including cloth simulation.
Where was filmed the exterior sequences?
Those sequences were filmed in the Malta Film Studios. In the gigantic water tank which is close to the actual ocean they built a real scale Demeter which was even floating. Seeing this 70m long ship the first time was quite impressive. The Varna Habor Set, several ancient buildings, was located close by and was also looking great. For those scenes they also built a second Demeter, but just a part of it to film the loading sequence with the crates in the beginning of the movie. So we had to extend the Demeter in this sequence as well.
That’s the first time that RISE is doing so many water. What was your approach about it?
First we collected as many references as possible from crashing waves to boat wakes and ships caught in horrifying storms. Then the RnD process started quite early led by our FX Supervisor Kristian Kück and Lead FX TD Jonas Sorgenfrei, the same team that also created water effects for one of our previous projects Jim Button and the Wild 13. In addition to that Igor Zanic supported our team with his experience. We relied on the large-scale ocean simulation tools that Houdini provides but added several proprietary tools and techniques along the way.
How did you manage the various states of the ocean (from calm water to storm)?
Every one of those states required a different setup with its own challenges. Starting with a believable ocean spectrum (this defines the look of the ocean surface) for a calm ocean which can be even as tricky as a stormy one. For the pre-storm surface we added patches of whitewater floating on the surface. As the wide shots required hundreds of those patches we had a tool to scatter pre-simulated whitewater caches that being displaced at rendertime to match the ocean surface. For the closeup shots and crashing waves at the beach we used sliced simulations to get the detail we needed as it wasn’t achievable in one big simulation.
How can you art directed the ocean simulations?
To art direct an ocean simulation it takes a lot of patience, because mostly you can see the effect of the changes the next day if you are lucky. During the RnD phase it was quite uncontrollable, but from shot to shot that we finished we refined the setups and tool, so that in the end the level of control was astonishing. But still…it’s a simulation and that’s also what makes it so realistic. So the best is always to send 5 wedges with slightly different parameters to the farm and pick the best one in the morning. I am not a big fan of using many precached elements for those stormy shots or big waves, that often results in a painterly feeling, as you can notice that physics are not correct.
What was the biggest challenges with the water simulations?
Creating a believable surface is one thing, but the real challenge is the whitewater/foam. To get a realistic movement so that it really sells the scale and does not look like bath foam required a lot of RnD. Especially the moment when a wave collapses and creates this whitewater, how it gets advected by other waves, moves on the crest and flows down bigger waves. We created hundreds of wedges comparing the effect of individual parameters to find the right balance for a believable motion.
How did you handle the interactions between the water and the Demeter?
To sell the interaction of the ship cutting through the water we created a robust boat wake setup, which offered a lot of flexibility regarding the size of the boat wake including a wetmap and dripping water along the hull, which helped a lot to sell the connection in the closeup shots.
Especially in the storm scenes the animation of the ship was crucial to get a realistic result. Our animation supervisor Dastin Hoffmann and his team did a really great job here. But already while studying the references we noticed that the size of a wave often looks way smaller in the video than you would experience it while being on a boat. As this was also happening in the storm shots we had to exaggerate the wave size and steepness in comparison to the Demeter quite a bit. This resulted in huge waves that had to interact with the ship, which made it extremely challenging to keep it still controllable. In the end the only way was to modify the gravity based on the wave angle, so that the simulation was more stable again.
Speaking about the Demeter, can you elaborates about its design and creation?
The build of the Demeter asset started based on the LIDAR scans and references we took in Malta in the water tank of the real ship. In addition to that the exchange with the art department was quite helpful as they could also provide blueprints for the parts of the ship that was not built on set.
Jenny Leupold, our CG Supervisor, and her team crafted literally every plank so that we ended up with a super detailed asset that could be used for the harbor extension shots, including some closeup shots and also the various full CG shots of the Demeter we created. The modeling process was done in Maya, then we did some procedural setups like scattering of mussels and dirt on the hull in Houdini and the texturing happened in Mari. For the scenes on Whitby beach a destroyed version was needed as well.
How did you get all the data and texture materials for the ship?
Our Lidar-Team and me could join the principal photography in Studio Babelsberg and on Malta, where they shot all Varna Harbor scenes and the ones with the Demeter in the water tank. Aside from the shoot we could take thousands of reference photos, to cover every inch of the Demeter.
Additionally, we took several lidar scans and used a drone for photogrammetry. As Malta has some unique locations, we went out on the non-shooting days to get more coverage from the old town of Valetta, which you can find now as buildings in Varna Harbour and we also scanned some nice cliffs, which we used for Whitby beach.
How did you handle the lighting challenges for the night sequences?
Once we received the plates we noticed that in most night scenes they used to many individual lights quite close to the boat, which resulted in a quite bright illumination including extremely bright highlights as the boat was wet in the most of those scenes. To tie those plates together with a believable nighttime ocean and sky we had to apply a strong color grade or even replace full parts of the ship so that we could get continuous lighting. For the Full CG shots it was much easier, we could just place the moon so that all sails were nicely illuminated. For sure that’s a luxury that you do not have on set.
Did you use procedural tools for the sails animations?
Yes, we created a procedural Houdini setup that could deal with different sizes and shapes of sails, including all the rigging and ropes. In addition to that we could switch to a pre-cached version if no special movement was required.
Can you elaborate about the design and creation of the burning people?
Looking at the plates it was quite clear that we had to create a complete replacement for Olgaren (the one tied to the mast) as there was no makeup indicating the burning progress and no interactive light to tie it together with the fire effects we had to add.
So we started to build a burned version of Olgaren and effects setups for blistering skin and flames. Using those assets and the actor’s performance as a reference we were able to start with just blistering skin and a bit of smoke until we see a fully burned corpse in the last scene, with different “burn-stages” in between. As everything was setup procedurally we could re-use those setups also for Toby and Anna in the end of the movie. Another complexity was added in the scenes with the burning boy, as his clothing also needed to catch fire. Akin Göcmenli, FX sup on the show, created a clever setup that provided an art directable cloth setup so that we had a proper emission geometry and movement to drive the flames simulation. Those CG flames could then be layered on top of the real cloth with additional burn masks.
What kind of references and influences did you receive from the Director?
In the beginning of the project we received great concept work Andre had created with concept artist during the pre-production, which helped us a lot to get an idea for which mood and details he was going. For example the shot with the wrecked Demeter on Whitby Beach is quite close to the concept.
Later during production we were also doing interactive sessions with Andre and Brad. Basically blocking out a simplified version of the shot in real-time in Houdini so that Andre could give us live feedback about camera angle, testing different focal lengths, boat and wave movements. That was very helpful and saved a lot of time, as there are endless options for a full CG shot. Once this blocking was done we could use it to refine the animations and start with the large scale water simulations.
Can you tell us more about the use of USD on this show?
The Last Voyage of the Demeter was the first start-to-end USD project at RISE so this alone made it quite a big challenge. But we also knew that we needed a project this size to put the pipeline to the test and push it to the next level. Simon Ohler, Luca Scheller, Kolja Hübschmann and the rest of the pipeline team worked already for quite some time on this completely new pipeline inside Houdini and Maya to make sure that it provided the most important features when we started. For sure everybody on the project knew that it would require a closer collaboration with pipeline than usual to provide feedback and to get into all the new features.
We also decided to switch from Mantra/Renderman to the native Houdini Solaris renderer Karma for this project, which was another challenge but was working out great in the end because of its seamless integration with USD and Solaris.
For more details about our new USD pipeline and how we used it on Demeter there is also a Siggraph Talk available online from last year:
Which shot or sequence was the most challenging?
Definitely the full CG shots with the ship in the storm, especially the ones with the camera quite close to the ocean surface and waves colliding with the Demeter. And last but not least the shot with the Demeter crashing into Whitby beach, just bursting wood and ripping sails, while in the background we see the angry sea.
Is there something specific that gives you some really short nights?
Before I was talking about the completely new USD pipeline and even though others might have considered this as a guarantee for short nights, I was actually pretty sure that this would work out. For me it was more the question if we could push our water simulations to a level so that they were really matching the references I took. But after seeing the results improving with every iteration and the final result I was really happy.
What is your favorite shot or sequence?
It’s hard to decide for one shot as all the sequences are so different. But one of my favorites is the one seeing the Demeter first time on the open ocean with a low sun coming towards camera. This full CG shot is showing the ship in all its beauty, has great sail sims and the best boat wake we have ever done.
What is your best memory on this show?
Around August 2021 we finished most of the shots and artists were moving on to other projects, just a small core team continued. So on the last day with the whole crew I put together a edit with all our shots and we watched it together. It was more than 10 minutes long and just great to see what we had achieved all together in the last 12 months.
How long have you worked on this show?
In total it was about 15 month starting with the shoot in Malta in August 2021 until delivering the last shot in autumn 2022.
What’s the VFX shots count?
We delivered about 250 finals but worked on more than 400 shots in total, as the movie was re-edited several times.
What was the size of your team?
The maximum team size at one time was about 60 artists including production and during the whole duration of the project probably more than 100 artists.
What is your next project?
After Demeter I worked on The Last of Us Series for HBO and currently we are working on undisclosed TV Series for Amazon Studios.
A big thanks for your time.
WANT TO KNOW MORE?
RISE: Dedicated page about The Last Voyage of the Demeter on RISE website.
© Vincent Frei – The Art of VFX – 2023
