Episode 323 — DUNE VFX Supervisor — Paul Lambert

Paul Lambert is a Visual Effects Supervisor. He is also the inventor of IBK Keyer in Nuke.  Over the course of his over-two-decade long career, he’s worked on such titles as Gone Girl, The Girl with the Dragon Tattoo, TRON: Legacy, The Curious Case of Benjamin Button, Pirates of the Caribbean: At World’s End, I, Robot, Mission Impossible II, Lost in Space and many, many others.

Paul has won two consecutive wins for Academy Awards for Best Visual Effects: one for the 2017 film Blade Runner 2049 at the 90th Academy Awards, and one for the 2018 film First Man at the 91st Academy Awards.

DNEG (www.dneg.com) is one of the world’s leading visual effects (VFX) and animation studios for the creation of feature film, television and multiplatform content. DNEG employs nearly 7,000 people with worldwide offices and studios across North America (Los Angeles, Montréal, Toronto, Vancouver), Europe (London) and Asia (Bangalore, Chandigarh, Chennai, Mumbai).

DNEG’s critically acclaimed work has earned the company six Academy Awards® for Best Visual Effects and numerous BAFTA and Primetime EMMY® Awards for its high-quality VFX work. Upcoming DNEG projects on behalf of its Hollywood and global studio and production company partners include Dune (October 2021), No Time To Die (October 2021), Ron’s Gone Wrong (October 2021), Last Night In Soho (October 2021), Ghostbusters: Afterlife (November 2021), The Matrix 4 (December 2021), Uncharted (2022), Borderlands (2022), Aquaman and the Lost Kingdom (2022), The Flash (2022), and Shazam! Fury of the Gods (2023).

DNEG was the lead vendor on Denis Villeneuve’s Dune, along with Rodeo (https://www.rodeofx.com/) and Wylie Co (https://wylie.co/.)

In this Podcast, Paul talks about Dune — from its conception to completion — the value and advancement of AI and virtual production and importance of creating authentic relationships for creatives.

DNEG Website: https://www.dneg.com

Rodeo FX Website: https://www.rodeofx.com/

Wylie Co Website: https://wylie.co/

DNEG at LinkedIn: ​​https://www.linkedin.com/company/dnegvfx

DNEG on Facebook: https://www.facebook.com/dnegvfx/

DNEG on IG: https://www.instagram.com/dneg/

Paul Lambert on IMDb: https://www.imdb.com/name/nm0995902/

Paul Lambert on LinkedIn: https://ca.linkedin.com/in/paul-lambert-4303341

HIGHLIGHTS:

[04:15] Paul Lambert Talks About His Beginnings as an Artist

[10:39] The Invention of IBK Keyer

[14:26] The Conception of Dune

[30:47] The Impact of Virtual Production on Filmmaking

[38:44] Creating the Sandworm and the Harvester Sequences

[49:17] Utilizing LiDAR Scanners on Set

[1:02:04] Importance of Relationships

EPISODE 323 — DUNE VFX SUPERVISOR — PAUL LAMBERT

Hi, everyone!

This is Allan McKay. Welcome to Episode 323! I’m speaking with Paul Lambert, the VFX Supervisor for Dune. We’ll be talking about DNEG’s massive contribution to film and where the technology is leading in visual effects in film.

I’m really excited for this Episode! Paul worked on so many films, like Bladerunner 2049, First Man, Dune (and the newly announced Dune Part 2). He is a two-time Oscar winner: for Bladerunner 2049 and First Man. He’s the inventor of IBK Keyer in Nuke, as well as has a massive history of working on films like TRON: Legacy, GI Joe, Pirates of the Caribbean, I, Robot, Day After Tomorrow, Tomb Raider, all the way back to films like Lost in Space.

During this Podcast, we cover some of the unique shooting methods for Dune, as well as sequences like the Sandworm. We talk about taking a sequence from the vision to its completion, and where AI and VR are going to play a major role.

Let’s dive in!

FIRST THINGS FIRST:

[01:16]  Have you ever sent in your reel and wondered why you didn’t get the callback or what the reason was you didn’t get the job? Over the past 20 years of working for studios like ILM, Blur Studio, Ubisoft, I’ve built hundreds of teams and hired hundreds of artists — and reviewed thousands of reels! That’s why I decided to write The Ultimate Demo Reel Guide from the perspective of someone who actually does the hiring. You can get this book for free right now at www.allanmckay.com/myreel!

[1:10:46] One of the biggest problems we face as artists is figuring out how much we’re worth. I’ve put together a website. Check it out: www.VFXRates.com! This is a chance for you to put in your level of experience, your discipline, your location — and it will give you an accurate idea what you and everyone else in your discipline should be charging. Check it out: www.VFXRates.com!

INTERVIEW WITH PAUL LAMBERT

[04:15] Allan: Paul, thank you so much for taking the time to chat! Do you want to quickly introduce yourself?

Paul: Yes! I’m Paul Lambert. I was the VFX Supervisor on Dune, working with Denis Villeneuve.

[04:28] Allan: For you going back to the beginning of your career, did you always imagine working in some kind of a creative field?

Paul: I totally fell into visual effects. A lot of people from my era have a similar story of falling into VFX. At the time, you couldn’t go to school for visual effects. You started off as a runner and worked your way up. That’s exactly what happened to me. I just finished my university (I did an aerospace program) and I finished it and [realized] it just wasn’t for me. I was in a class with students who pointed out any aircraft flying overhead and gave you what the tire pressure was on that plane. I didn’t have that passion. I got a bit lost. I started working as a currier and I was driving around London, making deliveries for a production company. I got to talk to the guys and got a job there. What they were doing was servicing mechanical machines that were editing machines for 35 mm film. We would take these machines to various places for post-production, on various movies. That’s when I realized: You can actually work in movies! Up until then, I loved watching movies but it never occurred to me that you could work in this field. When I joined them, that’s when Avid and Lightworks were just appearing. I got fully involved in that. You could do something digitally on your computer and connect it to the Avid. That’s where the first seed started.

At the time, CGI had a building in Soho Square. They were running a course for a week where you could experiment with this product called Flame. I think at the time they had Houdini as well. I took this course and loved it! Then I decided to quit my job and I spent 6 months trying to get a runner position at one of the post- houses in Soho. It took 6 months and it was getting to the point [where I thought of giving up]. And then I got a break: I got a temporary gig at Cinesite. At the time, you could still learn the Flame machines on your own time. So that’s exactly what I did. There were some great artists there. Then [I started] doing a bit of wire removal. I learned bit by bit, and I got better and better. Cinesite was a Kodak company at that time. They had a feature and a commercial department and those two never mixed. You never had any of the Flamers working on features. Because it was a Kodak place, Kodak decided they didn’t want to focus on commercials. I got kept on, however. We started working on tv. It was during that time that I got to learn the system in and out. I knew all the bits and pieces. I was on the Beta program. I was having a conversation with the head of RnD Simon Eves.

[10:39] Allan: I know Simon!

Paul: He explained to me how an image could go over one. How it could be brighter than one. It blew my mind! I knew the software but that blew my mind. That’s when I started to play with Shake. That’s when I started to think about how to get Flame to work on feature work, on 10 bit cineon files. I started scratching my brain about how it all worked. We had two Infernos and by the time I left — we had six! — all working on feature films. I left Cinesite to go to the U.S. to work for Digital Domain. I learned every aspect of how to produce high-end compositing. It was there that I invented IBK Keyer. Working with so many fantastic Supervisors: Eric Barber, Eric Nash, Bryan Grill. [08:36] It was there that I learned the drive to never give up. You can always push a shot to be better and better. That’s how I work now. After 13 years at Digital Domain, I got an opportunity to go to DNEG up in Canada. A friend of mine at Cinesite was running a place. I was made a VFX Supervisor on the Huntsman, and I’ve been on a crazy trip ever since!

[13:38] Allan: I grew up in Australia and we had one Flame at my first job. While the Mill had something like 16! I was humbled by the sheer workshop factory you can make. To dive into Dune, congratulations on Part 2!

Paul: Yes! That came out of the blue. That was awesome!

[14:26] Allan: In terms of when you got involved with Dune, it was early on. Can you talk about that, from going through the script treatment?

Paul: I had previously worked with Denis on Bladerunner. I had an absolutely fantastic time working with Denis and Roger [Deakins] and John Nelson! It was when we won a BAFTA for the Bladerunner that Denis said he was working on the script for Dune. Would I be interested? YES! Yes, I would! We carried on talking for a few months. We would chat every now and then. It was only when The Fast 9 came out that I got confirmed on Dune. We started out with a camera shoot with Greig Fraser and Denis in SoCal where there is this area of sand. At certain angles it looks like African dunes. It’s just near the border with Mexico. That’s where Greig was doing some camera tests with Alexa film, trying to figure out how we were going to shoot Dune. That was a real success.

We all flew out to Budapest. It was there that I got to read Denis’s script. Denis at that time was working with Patrice Vermette, on the visuals. He had this book of beautiful images. Concept work is a time for ideas, but he was really adamant that that’s what he wanted to build physically on the sets in Budapest — and what we were going to build out virtually. All of Patrice’s designs were huge. Reading the script, I purposefully hadn’t caught up on [the book]. I’d read it after seeing the David Lynch version. I didn’t know anything about it before then. I saw correlations between Dune and Star Wars. I did make a conscious decision to not reread the book. I just wanted to consume Denis’s version. I wanted to keep it fresh. Denis had a fantastic script, but he was still adapting it. We actually didn’t get greenlit until we were ready to shoot it. There was never an official go. It was during that pre-production phase that we came up with all the ideas.

The one good thing about having such deep pre-production work was: We knew we could set things up a particular way. We were never going to be in a position to figure something out in post. That was just a different attitude. Then we did our setups, working with Patrice and Greig. In VFX, we can take any piece of footage and put any background on it. But if the lighting is different, there is nothing I can do about it. It’s the luxury of Dune that we did know what the backgrounds would be. Coming from compositing, you have an innate sense. I was telling Greig that we could come up with different ideas. When Paul and the rest of the gang lands after the attack and they’re back in the hangar — and there is this bright light behind them — rather than putting a screen behind them, Greig put a silk angle there. You don’t see some of the background in it, and I would just deal with those edges. But you were never going to see the visual of the comp if I had a dark background. I can get that in camera and then blend it. Things like the slider car coming down in the Nexus, we made a conscious decision that we’d never try to replicate daylight inside a studio. For that sequence, it took forever. In the end, what we did was: We had two studios in Oregon and the space in between the two we connected with a material cutout. And there’d be a couple of hours in the day we could shoot there, with daylight coming in. And it would give us the right shadows and intensity. I could then extend out that world to make it bigger.

I had visited London for BGI Supplies. It wasn’t until I got there that I sat in the cockpit. I was seeing all the reflections. Instead, we found the tallest hill outside Budapest and we built the gimbal on top of the hill. We surrounded it with a 25-ft high ramp that was sand colored. On a bright day, you’d get the bounce from that brown. When you’re watching the footage, it looks like you’re in the desert already. We were planning to get some plate footage from a helicopter. We weren’t going to do any of the extractions: We kept all the glass and reflections. What the guys and girls did in Vancouver was do a blend between the aerial footage and the live action photography. It was a blend that kept all the reflections. And we got the correct lighting. The entire movie is peppered in with tiny techniques like that. We tried to avoid shooting against a blue- or greenscreen. We came up with an idea of a sand screen. If the background is of the same color, it’s a far more believable composite. I remember walking around the set with one of the producers. I took a picture of the set and I did a negative of it — it’s all bluescreen. It all worked out pretty well. It was all about coming up with the best techniques to give us the best effects. We didn’t have to push and pull the image too much. It worked out really well.

[30:47] Allan: For me, when I saw the film, I thought it was one of the most beautiful films I’ve ever seen. Knowing that you wanted to shoot on location, do you feel we are getting to a point — between being on set, virtual production and being able to extract mattes — where we are starting to move away and shoot things freely?

Paul: Oh, for sure! I think we’re on the cusp of nearly freeing ourselves from screens. I came up with these techniques, and it’s ironic that I try to avoid that kind of work now. Because you now have an option to put anything back there, the easiest composite is when the luminance you put behind the character is the equivalent of the luminance of a blue- or a greenscreen. That is when your comp goes really quickly. But because you’re dealing with a single color, it has to be a very specific set up to work that easily. With the advent of LED screens, we can have those backgrounds and not worry about that. LED screens have their own issues. We talked about using them on Dune. Greig used them on Mandalorian and I used them on First Man, the two productions to fully embrace that technology. We needed too much light and LED screens can’t give you that harsh daylight. For the past year, I’ve been heavily researching AI — and all that world — about being able to extract background from foreground. We’re so close! It’s not quite there to be consistent from frame to frame. I’ve seen ideas that are getting better and better. I think we’re a year or two away from being able to extract anything. You know how we shoot HDRI passes. I’m already planning to shoot AI passes: have actors do set-ups against a bluescreen and then train the AI. That’s getting super close! I’m going to shoot that just so that I have it. If you’re leaving the background for something in post-, there is a possibility your match won’t be right.

[36:20] Allan: There is a YouTube channel called Two Minute Papers (https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg). I look at it religiously. There are some websites that do a good job. I had Doug Roble from DD on the Podcast (www.allanmckay.com/290). What they’re doing with Virtual Humans, there are so many innovations. He made a claim that within 5 years that animators won’t be animating with keyframes anymore. It’s such a shift from what we’ve known! We’re now living in the innovation.

Paul: Totally! I read once that our understanding of what we’re doing is almost preatomial. It’s creeping up on us. I’ve been working with a company that works in the poster space. They have some tools now where you can click on a character and it does a rough matte around it. Every aspect of visual effects will be touched.

[38:44] Allan: I’m interested in that epic sequence of the Sandworm and the Harvester. Do you mind walking us through some of the challenges of that?

Paul: Yeah. There was a lot of planning for that sequence. It’s pretty much the only sequence that was pre-vized. Denis is not a big fan of pre-viz. He uses it more as an informational tool for other departments. He has this organic way of working. He has this saying, “Nature supersedes a storyboard.” We had done a rough pre-viz, MPC had three guys on set with us. We were going to have this huge space crawler in the desert. How are we going to have something physical out there. Patrice had this massive track built and Greig positioned it in a way the sun moved over the tracks, which then allowed up lots of hours of shooting. That allowed us a few days around that feature. Denis wanted a signature for the Worm; and Gerd Nefzer, our Special Effects Supervisor, came up with this idea of an 8 X 8 foot plate which we buried deep in the sand. When he cranked it up the sand would almost become water. You can see those patterns in the film, they were practical. Where you see Paul and Gurney and they’re on their hands and knees, they’re actually sinking into the sand. That was the signature for when the Worm was approaching. The Worm would be appearing in that sequence. You’d see the trail of destruction before you saw it. I was pretty concerned that I couldn’t find any reference to be able to shift this amount of sand. I spoke to Gerd about some explosives under the sand, for reference. We were shooting on a military zone, close to the border of Israel.

[43:23] Allan: That would be a bad idea!

Paul: Needless to say, I didn’t get that footage. I started tests of simulations done. Computation gets expensive. I knew there was going to be a process. Having built these onifactors, we transported one to the Jordanian desert. These things were close to 12 tons. It had to be transported by one of those Russian Antonov An-225 planes.

What we did with that, we’d pick them up with cranes and simulate the take off and landing of these ships. We would blast them with sand. It took two showers to get all of that stuff out. It gave it such a fantastic base! It made it for a harder composite. Having all the swirling dust, we then put on CG wings. We had fully hydraulic ramps. Sometimes it would be replaced by CG. But having that in-camera on set is the best place because you can reference it. Denis was adamant about wanting to keep consistent with concepts. If the concept didn’t match, it’d go back. Denis said that on Bladerunner there were so many vendors. It was his first huge VFX film, with thousands of shots. And he always lamented that he let some things slide. Things do change and it goes down a certain path. He was adamant that these things had to match. It all comes back to the last days of the shoot for that sequence. There were dunes we had to replace. But we tried to get as much as possible in camera. That was always the goal. It gives you time to work on more creative things.

[49:17] Allan: I know you did LiDAR scanning. What’s your opinion on how to utilize it for production?

Paul: I’m a big believer in trying to get as much information as you can. I always shoot with witness cameras even if I know I’m not going to be using it. Going from the set to the artist who wasn’t on set, having a visual is so important. Yes, you can talk about things but it’s like Chinese whispers. Having a visual helps. When we were in Budapest, there was a company called OroBlade. There was a chap from that company LiDAR-ing at the same time we were shooting. We were moving screens around, we tried to catch as much information as possible. Shooting in Jordan, we did weeks of scanning in Wadi Rum. Arrakis is based on a specific area in Wadi Rum. Denis, Patrice and I flew around in a helicopter. We found these rock formations and we had guys scan those. But also, the drone worked. I always try to get as much LiDAR as possible, and as many witness cameras as possible.

[53:14] Allan: I just did that for a big set. I’m amazed at the amount of accuracy from LiDAR. If you could pick one shot and walk through the concept to the final delivery.

Paul: That’s a tricky one to pick one. I and Denis have a similar sensibility. I’m never going to propose a virtual camera, for example, that is not physically based. If he asks me a question and I can’t give him a confident answer, he’ll change the question. Sometimes you need to think about it. That’s what I got into a habit of doing: I need a minute. I need to think this process through. I think on Dune one of the trickier ones we had to shoot — which came out pretty well — was the Salusa Secundus. Basically, we decided quite late that it was going to be raining on set and I had to change all the colors of the screens the morning of. The ground was getting muddy and the set was starting to fall apart. Thinking on our feet, we were going to shoot Secundus in that sunny / rainy environment. When you see that army stand up, we shot that practical. Dealing with change in light and rain, some areas had sun and others were in shadow. It was a bit of a compositing feat to handle that, because of the challenge in lighting. It’s a small sequence but it shows another world. And it came out really well!

One sequence just occurred to me in terms of, “How are we going to do this?!” It was the hologram sequence where Paul hides inside it and he has the hunter seeker coming toward him. In keeping with the philosophy of the shoot, I wanted to avoid putting digital light on Paul’s face.

[58:07] Allan: I saw some pretty amazing AI paper coming out on lighting. But I don’t have faith in it yet.

Paul: Exactly! I think there is a quote from Paul Debevec when he was at Google. He did that paper on changing portrait lighting based on the background. It’s getting there and you can get great relighting but it’s not quite production ready. But it’s coming! For the hologram sequence, I wanted to avoid getting the digital version of Paul. It took a bit of thinking. One thing on Dune, I brought in-house artists with me from the beginning. The idea was that they’d do tests for me that could inform the shoot. To get a test done by a facility, it takes some time to schedule it. I wanted something immediate. One of the tests we wanted to do was about how to come up with technique to shine the light on Paul. We came up with a way to track Paul interactively on set. There were various sensors on Timothee’s body. We used a projector to project the bush onto him. The bush was sliced into different planes, and when Paul moved, that plane would change. It wasn’t set up for a complex move and it worked really well. You’d get proper light going onto Paul. In comp, we CG’d the bush around him. And it worked!

[1:02:04] Allan: For most people listening, what artists often undervalue is relationships. Knowing that you’ve worked with Denis before and you’ve learned to communicate well, can you talk about that?

Paul: When I first worked with him on Bladerunner, I could tell there was something similar about us. I have this habit of walking around the set, head down, deep in thought. I’d look up and Denis was doing the exact same thing! We both have a terrible sense of direction. There was a particular moment on Bladerunner when I was watching something on video and Denis, in his wandering, came up to me. He was talking to me about negative space and it blew me away how much thought goes into everything he sees. Working with him in post-, I think he has a photographic memory. I think I can forget what I did in the morning. But when it comes to a shot, I can remember. He just takes it to a whole other level! I thought I had it good. Going back to that similar sensibility: The work I want to do is complete photoreal. I don’t want you to know I’ve done something to the shot. That’s how Denis wants his movies to be portrayed. In Bladerunner, you have all these spaceships. There is obviously something done in miniatures or VFX. But you get to be immersed in these worlds. If he senses a bit of a hesitancy from me, he’ll change it up. It’s a fantastic way to work with someone! And the whole team on Dune was really collaborative.

[1:06:14] Allan: When I first started in the 90s, there were a couple of vendors. After a while, it ends up requiring multiple houses to get something done. Now, more studios work on a feature. But now studios like DNEG have several locations. Do you see more of a comeback where one studio will be working on a major project?

Paul: That’s a great question! I’m not a fan of having one facility do the entire show. On Dune, DNEG did a lion’s share. Rodeo did some beautiful establishing shots. I had Wylie VFX do hundreds of shots in in-house capacity. MPC did the pre-viz. I’m not a fan of one facility. You’ll have a certain number of people and a certain amount of time. Certain sequences will be done after others. Sometimes a director may not see a sequence months later. Breaking down a film into a minimum of two places, so you can have the push from both facilities to do good work, that’s better. It’s a tidy sum. It can have issues with certain sequences getting tackled at a later day. The director will have studio cuts and he still has post-viz in some. There isn’t a legitimate answer to give him. As an independent VFX Sup, that’s how I’m going to structure my shows. There are too many questions that need answers.

[1:10:26] Allan: Thank you, Paul, for taking the time to chat! This has been amazing! I appreciate your time.

Paul: Thank you for having me, Allan! It’s been a pleasure!

What did you think?

I want to thank Paul for coming on the Podcast. Warner Brothers invited me to an advanced screening of Dune. I couldn’t wait to sit down and talk to him about the film!

Next week, I’m chatting with Erika Burton, DNEG’s EVP Global Head of Studios, about her career and her advice to young artists.

Please take a few moments to share this Podcast with others. Thank you for listening!

I’ll be back next Episode. Until then —

Rock on!

Click here to listen on iTunes!

Get on the VIP insiders list!

Upload The Productive Artist e-book.

Allan McKay’s Facebook Fanpage.

Allan McKay’s YouTube Channel.

Allan McKay’s Instagram.