Episode 277 — JEFFREY — The Creative Team

Connie Siu is the Writer and Director of short film Jeffrey. She holds an MBA from the Kellogg School of Management at Northwestern University, graduated cum laude from SFSU with a BA in Fine Art, and attended UCLA’s Producers Workshop Program. Connie is the former Chairman and co-founder of the San Francisco Chapter of Siggraph, the former Vice-Chair of the U.S. Sports Film Festival, and served as Executive Director of the Pre-Visualization Society.

Jeffrey is a short film that was shot as a virtual production in 2020. It’s a love story whose main character Jeffrey repairs the Dead so they can move on to the Beyond. Due to a series of magical events, he meets a woman and for the first time experiences the innocence of first love.

In this Podcast, Connie Siu is joined by the creative team behind Jeffrey:

• Virtual Lighting Supervisor Paul Debevec
• VFX Supervisor Joel Hynek
• Line Producer Allison Vanore
• Associate Producer Gena Kay
• And Phil Galler and Poom Wattanapan of Lux Machina

In this Podcast, Allan McKay interviews Writer and Director Connie Siu and the creative team behind JEFFREY — a short film and a virtual production shot during 2020 — about the film’s innovations and challenges.

Jeffrey on IMDb: https://www.imdb.com/title/tt9669630/?ref_=nm_knf_i2

Jeffrey on Instagram: @JHCfilm

Jeffrey on Twitter: @JeffreyHotelCa

Jeffrey on Facebook: @JeffreyHotelCalifornia  

HIGHLIGHTS:

[02:00] The Creative Team Behind Jeffrey Introduces Itself

[03:10] Director Connie Siu Talks About the Story of Jeffrey

[15:23] The Team Discusses Elements of Virtual Production

[21:33] Assets and Where in the Pipeline They’re Created

[26:56] The Challenges and Lessons of Working on a Virtual Production

[31:51] How the Global Pandemic Affected Production

[39:26] Paul Debevec Discusses the Future of Virtual Production

[45:07] Connie Talks About the Future of This Project

[53:54] Building a Creative Team for a Virtual Production

[1:01:52] Connie Gives Advice to New Creators

EPISODE 277 — THE CREATIVE TEAM BEHIND “JEFFREY”

Hello, everyone! This is Allan McKay.

Welcome to Episode 277! I’m sitting down with Writer and Director Connie Siu, as well as other creatives behind the project Jeffrey. I also have VFX Supervisor Paul Debevec, Phil Galler and Poom Wattanapan from Lux Machina.

Let’s dive in!

FIRST THINGS FIRST:

[1:08:11] One of the biggest problems we face as artists is figuring out how much we’re worth. I’ve put together a website. Check it out: www.VFXRates.com! This is a chance for you to put in your level of experience, your discipline, your location — and it will give you an accurate idea what you and everyone else in your discipline should be charging. Check it out: www.VFXRates.com!

THE CREATIVE TEAM BEHIND “JEFFREY”

[02:00] Allan: Thanks again, guys, for taking the time to chat! Do you want to quickly introduce yourselves?

Connie: I’m Connie Siu. I’m the Writer and Director of Jeffrey which is a virtual production. On the call with me, I have our producing, virtual production and VFX team.

Allison: I’m Allison Vanore, I’m the Line Producer on Jeffrey. I’m a Producer out in the world.

Gena: I’m Gena Kay. I’m the Associate Producer on Jeffrey.

Joel: I’m Joel Hynek, the VFX Supervisor on Jeffrey. I’ve been in this business for a long time. It’s really exciting to be doing a virtual production.

Phil: I’m Phil Galler. I’m a producer at Lux Machina.

Poom: My name is Poom [Wattanapan]. I’m acting as a VR Supervisor at Lux.

[03:10] Allan: Connie, can you talk about the backstory for Jeffrey? And can you talk about your writing process?

Connie: Jeffrey is a fantasy love story. It’s a sci fi story about a character that lives in the afterlife. His role is to put people back together so they can move onto the next world. He is a master craftsman. The story takes place one magical night when something happens to him and he meets someone for the first time. It’s a moment he has a connection with someone. As for the writing process, someone originally came to me with locations and wardrobe. They were looking to do a horror film. But this film really pushed me to think about who I am as a writer and storyteller, and what I want to tell, in the world. It’s about the magic of life. It’s really about having second chances and pulling yourself back together. There are some magical aspects in the world, and it’s about beauty.

[05:16] Allan: With the initial screenplay, how did it come to be? You’ve won an award for it, correct? How did you take it to something bigger?

Connie: I worked for Dreamworks Animation originally. Dreamworks had an internal writing program where we could exchange scripts. That’s when I began writing. I wrote this short script after I left Dreamworks. I submitted to the Emergence Films writing competition. That’s a competition that’s sponsored by a lot of vendors in Los Angeles. It’s a writing competition to promote underrepresented people in the industry. My script won the competition. After that, we started putting our team together. In the meantime, I submitted for the Epic Megagrant. At the end of last year, we were notified that the project was awarded the Megagrant.

[06:55] Allan: That’s really awesome! It sounds like you were pretty clear about what media you were going to use. How early on did you have that spark of inspiration that the film would be in VR?

Connie: My background is in visual effects. I’ve also worked the set for viz art teams. For me, it was natural to create a virtual world. I started in the gaming industry by building assets for Realtime games. Then I was at Dreamworks. Working in virtual environments was second nature for me and it was more natural than building locations. I was always more comfortable working in virtual environments . Then we started working with Phil and his team, and Joel joined the group. They’re experts in creating those worlds, so the team really came together.

[08:24] Allan: I’m really fascinated by you gaining this traction and getting people on board. It’s the definition of hustling. What was that like, as a process? Did you have to overcome any friction? A lot of the people on the team have so many opportunities around them. For them to say yes, they must’ve been attracted to your passion and charisma. How did you bring together this A Team?

Connie: Oh, wow! That’s an interesting question! I think part of it is that I’ve worked in the industry for a long time. Andrew Schmidt and I go back a few years. It’s amazing to work with him, as well as everyone else on the team. I think it was a snowball. I had the script and I put together the lookbook, so people could understand the story. Even though it’s set in a horror genre, I wanted it to be more magical. And then it was just talking to people and telling them about the idea; and being able to talk about the new cutting edge technology we were able to use. I think this was an interesting way for people to be involved in a virtual production. There are more people on the call and we talk to them about why they came on board.

[11:58] Allan: A lot of people have great ideas but they don’t put together a strategy.

Connie: That’s true but it wasn’t easy and it wasn’t overnight. We built the team slowly, but at some point it was just snowballing. And people came on board because they thought it was an interesting project, or they have a prior relationship with me. So they were excited to work with each other. That’s what I love about this team: How collaborative and how creative and technical everyone is! The way everyone collaborated together was amazing. I feel really blessed to work with them!

[13:21] Allan: In terms of global structure, how did things start to come together? You mentioned when Andrew came on board. How early did you, guys, all come together? Were you all working remotely [because of COVID-19]?

Connie: We started out as a virtual team. We’ve used every video conference packaging out there. When COVID hit, we were already using a lot of these virtual communication devices. One of the things was that this is a passion project and people were working on their off times. The project has always been pretty flexible in terms of hours and locations. Everyone became accustomed with working at home and we were able to roll with all of that. We learned to communicate through all of these platforms.

[15:23] Allan: Joel, you would’ve come on early on. What were some of the challenges that came up at that time?

Joel: One of the first questions we had to ask is that the imagery would look good on LED. Phil has definitely risen to the occasion. The other thing was that Connie had us combine physical structures with the LED wall. During our first test, we discovered the pros and cons to doing that. It’s pretty cool to work in a light box. Connie is a great leader. Why I joined the project is because she had a great script and vision, and she is charismatic too. Once I learned that Paul Debevec was on the show, I thought I had to be a part of it.

[17:19] Allan: I’m always fascinated by that! When you look at a project that’s not backed commercially, it’s usually the story and the leadership that brings people on board. It’s pretty obvious that Connie has brought people together. Do you want to talk about the physical set that’s tied with VR? The idea of having an LED stage with physical props is really fascinating.

Joel: Take it away, Phil!

Phil: I will say that first: Thanks for having us! Connie definitely inspired us with her story. From the physical extension set, it was kind of fascinating. There was a need to mesh in linearity with the LED wall and some of the scenic pieces, which is the first time we’ve tried that. And we’ve learned a lot of lessons from that. It’s cool to break boundaries sometimes, and it’s cool to work with people who are willing to take those risks. Because it’s all part of storytelling. Everyone also worked hard on how to get these huge set pieces into the set. We don’t have a lot of space here. It was difficult to do in tight quarters. We worked with practical and virtual environment extensions. And it held up in post pretty well.

Poom: I think that was really interesting and fun to take on that challenge to take two different surfaces and see how we could combine the two. A lot of my role has been coordinating with our Unreal operators and making sure that people that sit on the more traditional side of film and the people working with engines can communicate. I’m doing a lot of translating to get groups of people that technically don’t speak the same language on the same page. And that was a really fun thing to see when it worked! Once we got there, it was really gratifying.

[21:33] Allan: I guess it’s like the chicken and the egg thing with creating the assets, and creating them accurately. In terms of the assets development, what was that like? I’ve seen the lookbook and it captures that. When it came to virtual production, was it based on actual locations? Now with virtual productions, I’m fascinated to see us create assets in pre-production (not post). What was the asset creating process like?

Connie: All the locations are digital so we worked with Adam Davis who is our Production Designer. He designed the original spaces and created the 3D models of his concept of the world. We had 3D models built. From there, we took the FBX files and those went out to Pi Squared and Barnstorm; and they took Adam’s 3D models and filled out the space. They refined the models, added textures and materials that gave it grime and dust and scratches. But there was a lot of back and forth with Phil’s team. We learned a lot just by taking a model and putting it on the set. We would do the geometry and bring it into the digital world. But once we put those models on the LED screen, they wouldn’t align. We had to expand the world so it would fit the stage. And because we were also working with practical, we were working with another company. We had physical doors so the characters could enter the stage. The digital team replicated those in a 3D model and we had renders of that. We also had wallpaper and carpet. So the digital team had practical references and the practical team had the actual physical objects. Also, there were a number of set pieces that we replicated digitally, like light fixtures. There were also a couple of props. It was interesting because in a traditional VFX pipeline, you would shoot everything with a green screen, and then the digital team would match that. But in our case, there had to be communication because of the digital, practical and lighting team; so our DP had to step in earlier and light the digital set and then replicate that with live action actors.

[26:56] Allan: Paul, can we talk a little bit about your involvement and where you were able to contribute the most?

Paul: I knew Connie because she had me come in to speak at Dreamworks, back when she was the head of their artistic development. I got to learn about some of the work she was doing, with her short stories and childrens books. When Jeffrey came along, she showed me the script. I thought, “Wow, this is great!” I also thought it was interesting that she was applying some of the latest technology. It’s an experimental platform to show what some of the new things in technology could be used for this format. I’ve been working with lighting CGI images and using LED panels and lights to illuminate them. Then I saw the demo that Lux Machina did for SIGGRAPH in 2019. Connie was interested in giving that a shot. And also that she was interested in bringing on Joel Hynek whom I knew from the work he’s done. The rest of the team she put together with Gena and Allison. I had a great conversation with Poom about the practical issues we could have. There was an enormous amount of excitement around virtual production and surrounding your actors with LED’s. But then there are these practical considerations about wielding this technology effectively. The LED panel is a physical gizmo thing that has its own peculiarities. There are issues with color renditions. Phil and his team have figured a lot of them out and how to apply them to virtual production. And then once you have an interesting vision in a script, then it’s about figuring out how to wield this technology to do this scene or that. So far, everything we’ve tried, there is great opportunity with technology but there is also a great challenge to apply it and work through it. I’m certainly learning a lot through this process.

[30:17] Allan: Yeah, I’m really excited to see it and how it came to life. Allison, how was this project different from your previous ones?

Allison: The exciting part of this was when I first moved to LA, my first job was in interactive design. I’ve had enough experience to at least understand some of the challenges, but I’ve been doing physical production for years. It’s about making those elements work together and understanding the needs from the technical side. I’m just happy to be a part of something that’s challenging and trying new things. The opportunity to be involved in something new and that has never been done before and pairing with what I know. That’s what I love about this industry is that every job is certainly different.

[31:51] Allan: For all of you, guys, I’m curious to know how this project came to light during the COVID pandemic. How did you pivot around it?

Connie: We were hoping to shoot in the beginning of the year, in March or April. But because everything was shut down, it gave us a chance to refine things and work on the backgrounds and the set designs. We had months to work on that and to bring it together. Since everyone was working from home, we started having Zoom calls on a regular basis, so the team got to know each other. We also got to address some of the issues in advance. A lot of the time, we would throw out ideas about how things could be done. Now that things are opening up a little bit, we’re able to test the theories we had. Some things that we thought would be challenging were not, and other things that we haven’t even considered became a challenge. It’s been an interesting process.

Allison: Because of the technology and how we have been communicating from the beginning, it made it right for the occasion. Because we’re used to checking in virtually and checking files, people are joining us from all over the world. We have witness cams on set and people are able to join and weigh in. Our team is global. Because we started that way, COVID just reinforced that. And everyone is able to participate.

Connie: On top of that, our workflow is virtual as well. We utilized screen sharing quite a bit. We reviewed previz virtually as well. We have a whole system of share drives and screen sharing. So we were able to smoothly work through COVID. And on set, we have live feeds.

[36:15] Allan: I’ve recently been chatting with DP’s who are based in LA but shooting something in Florida. It’s crazy how much innovation has happened because we’re having to deal with that right now.

Connie: A lot of the members on our team are interested in cutting edge technology and using the latest techniques, both in the way of production and in the way people work. This team is open to the new ways of working and thinking about production, and communication.

[37:05] Allan: Joel, I’m curious about what some of the biggest challenges have been so far?

Joel: I think the main challenge was combining the physical set pieces with the LED wall; and realizing what the challenges of the set geometry were. We had to push it beyond the LED wall. Now that it’s working pretty well. It’s interesting that we’re now doing a lot of our post-production in pre-production. Some of the CG things we have that are digital, but we have to represent them somehow during production. But the other cool thing about this approach: If for some reason there is an aspect that’s not working, we can put green around the actor and still benefit from the LED background, but put the actual background in post. So it’s a really interesting combination of pre- and post-production, or production; and you can decide if you maybe don’t want to [do something] now and you want to do it later. It gives you actual freedom. One other thing about COVID is that it allowed me to be here because I was actually on a big virtual production in China. That’s one good aspect of COVID.

[39:26] Allan: Paul, knowing this technology has been around for a long time but it’s been innovating much quicker and becoming a big trend recently: Is there anything you’re excited about, be it specific technology or where it’s going?

Paul: I think that what clearly works is that if you don’t want to put a green screen behind people, you can put an LED wall. And if the resolution of the LED wall is out of focus (which is often the case with cinema lenses), you can use that in camera. What’s been especially nice is that people are interested in surrounding actors with LED panels so they can do lighting on them as well. Where I think it will be interesting — and I’m hoping to be helping with that here and within the industry — is getting the result with the physical LED’s just as effectively as one could’ve gotten with image based lighting and CGI.

There are some challenges with the color rendition and the spectrum of LED is weird. You don’t have that problem with CGI when your RGB channels fill out the spectrum using approximations. You were kind of sampling little specs of the spectrum with the RGB LED’s missing all the yellow or the cyan light. And you have to color correct for that. Dealing with dynamic ranges is huge and a big outstanding issue is how to combine in an automated, managed, verifiable way LED panel instruments and more traditional lighting instruments. Things like ARRI sky panels have both phosphor based white spectrum LED’s, the warm and the cool, and then individual red, green and blue to help move it to a certain point in color space; and the ability to go considerably brighter. And then [you can] use these in combination with each other! That’s something we’re having to do on Jeffrey. The Lux Machina stage allows you to aim the camera anywhere, but there should be light coming from that area too. Connie’s scenes are set in front of windows that should have light coming from them, onto the actors. We’re going to put practical lights in there and Connie’s DP Peter is doing a great job of that. But we don’t have a way of automatically dialing those in to get the right answer. And I don’t think anyone has cracked that nut to get it so that image based lighting in the real world is just as predictable and accurate and reliable as the image based lighting in the CGI world. That’s where we need to go over the next couple of years to make it a truly reliable tool.

[43:13] Allan: Do you see the future of virtual production take place at studios built specifically for it? Or do you see productions that have the budget to be able to construct their stages?

Paul: Great question! I think that it’s another tool in the toolbox. There is this vision: Why not shoot every shot of every movie using this technology? You could totally drink the Kool-Aid and go down that route. But at the same time, once you’re dealing with cameras getting lined up for a couple of hours, you’re thinking, “Why didn’t we just build this frigging set and shoot it on it?” I think honestly, you’re going to shoot some shots one way, some shots the other way. The stages will be available. You’ll book some days in LED, you’ll go on location for others. It’s the right tool for the right shot, to tell the story. It will not go away, it’s just going to be another tool in the toolbox.

[44:37] Allan: Connie, a quick question: What was Habib Zargarpour’s job on Jeffrey?

Connie: Habib was our Previz Supervisor. We’ve been working with him to previz the film. His team came on early on.

[45:07] Allan: I’ve been following his career since The Mask. Looking at the big picture, moving forward do you see this as a medium for production and interactive experiences?

Connie: In terms of Jeffrey, this is really a proof of concept for a bigger series or VR production. I’ve written the series treatment and a VR companion. In an ideal situation, we can reuse all of the assets (created by Unreal Engine). We can bring them into VR or an augmented reality experience. We would reuse the assets since they’ve been built in advance. This will allow us to use them for a game or an interactive story. As for the production side, we’ll be able to use these sets that we’ve built for the pilot.

[47:04] Allan: I was watching The Matrix the other day. The helicopter that crashes into the building. That helicopter ended up being used in another film that I worked on. So I like the fact that you are going to have these massive props and sets already. You’re able to recycle them in an efficient way. Do you think you could go the George Lucas route — and leverage your tools?

Connie: I think we’re able to do that with the assets we have now. And we have all of our sets that are all final sets in Unreal. Additionally, if we wanted to create another experience. Our set is built to be projected on an LED panel so you could create these rooms and populate them with an interactive experience, especially with Jeffrey’s workshop. You could interact with all the objects (which were built in Unreal Engine). We could use it across the board. It could be a VR experience, or graphic novels, or interactive graphic novels. It would be great to reuse the assets across the board!

[50:14] Allan: That’s so cool! One big question: In terms of the timeline, from getting this up and running, can you walk us through the process?

Connie: Our process took a little longer because it’s a passion project for a lot of people. There was the writing process and the fundraising process which was a whole kind of production in itself. Once we had the funding, then it became about finding the right people to join the team who understood the vision and the story. Then it’s really about doing the set design up front and the character design. That took some time. Once we had the assets, then it was about bringing on the team to implement that. Phil’s team came onboard, Joel came onboard. Then we started talking about how we’d bring it to life. In ours, there was some level of complication because we were bringing physical things into the set. If we were doing a fully digital or fully practical world, that’s been done. But we were integrating doors and windows and we wanted our characters to interact with those. All the practical elements needed to be matched with the virtual. That’s where all of our technical challenges came from. And then on top of that, there were practical consideration:

• Integrating your costumes;
• Setting up practical lighting.

We needed to decide what was digital and what was practical. Which things are we lighting in the digital world? Which lights will be interactive on set? We had to think about it in advance.

[53:54] Allan: For anyone that wants chime in about the best way to approach this, please do. Now you have so many light sources that are both real and CG. What’s that experience been like that you had to collaborate and communicate with each other? Have you been able to build that synergy?

Connie: One of the things we have in place is doing a regular call with the whole team to talk about these issues with everyone in the room. All the teams building the set are on the call, Joel, Paul, our DP, our Production, Costume and Makeup Departments. All the teams need to be involved and have the information in advance. It’s not a situation where we have the art department, for example, doing their thing separately. The passing of information between practical and digital, then it goes to Phil’s team to put it onto the LED panels. Once we see it there, we need to see what’s working or not. Do we need to change colors or geometry? And there are certain things that need to be considered for projecting onto the panels. Once you load it, the digital team needs to look at it on the stage. And there is a lot of discussion and back and forth. That’s why you need to make sure all the right people are in the room.

Joel: To that point, what a typical DP does on a physical set, there is no common language between CG world and the DP world. I’m sure Paul has something to say about this. Also, to your question about what’s coming up: I see the day when we’ll have robot controlled lights and the lights on the LED that will combine perfectly.

Connie: I think in an ideal world, there would be floor to ceiling LED panels, ceiling and floor panels. You could just project all the environments onto them and the light would reflect onto characters and reflective projects.

[59:04] Allan: We have a company in Portland, OR right now and their rate is 40K per day. I can definitely see a lot of what’s going on set, rippling from previz. It’s all going back to previz controlling. I think previz people are the closest to the director, along with the editor. It’s a pretty unique time. One last question for Phil, knowing the test you’ve done with Barnstorm, how did you approach it and how has it been different?

Phil: I think it’s been a pretty great collaboration in terms of the content with a lot of different groups bringing their unique approach to building content. Barnstorm is an Unreal house. It’s an interesting approach for them. It was cool to work through some plugin issues that are usually a little more daunting to solve. We got to work a lot with Pi Square and understand their workflow. It was a positive experience. This production was a passion project so how do we find the time to solve these issues? We have to figure it out on our own. I’ve learned a lot by this point. It’s been really cool!

[1:01:52] Allan: Connie, do you have any advice for female filmmakers or in general up and coming filmmakers? A lot of people who have creative ideas tend to fall short when it comes to bringing them to reality.

Connie: I think the first step is just doing it. If you have an idea — and a lot of people have ideas — you have to put it down on paper. Just get it out! And then refine it. Whatever you’re writing is going to be just a start. The version of Jeffrey that we’re shooting is so much richer than the first version I’ve written. But it’s because along the way people [made suggestions]. In the process of making it, it made me fill in the gaps and the story became really rich. I think you can’t just think you have to write a masterpiece from the beginning. It becomes amazing after all the feedback. Don’t worry about whether people like it or not. And if it’s a passion project, you have to believe in it. I think that’s why people are attracted to my project because it represents the things I believe in the world. And you just need to follow through with what you say you’re going to do. In the beginning, I wanted to make the story. Now, the reason I keep going is because there are so many people who believe in it. I have to finish it for all the people on the team, [including] all the donors and sponsors. I have to finish it because I want them to be proud! And that’s the most important thing to me right now. I just try to make things easy for everyone else because people are putting in their time and energy into this. I try to be clear with my vision so people know what’s going on; so that the team knows what the next steps are. So we are on the same path.

[1:07:00] Allan: Where can people go to find out more about Jeffrey?

Connie: We have a Facebook page and an Instagram page where we give weekly updates. Follow Jeffrey on Instagram: @JHCfilm; Twitter: @JeffreyHotelCa; Facebook: @JeffreyHotelCalifornia  

[1:07:31] Allan: This has been so great! I could easily pick your brains for a couple of hours, with all of you. Looking at the bigger picture to see everyone involved, I cannot wait to see the project! Thanks for taking the time!

Everyone: Thank you, Allan!

I hope you enjoyed this Episode. I want to thank Connie and everyone involved for their time. I cannot wait to see this project and it’s been so inspiring!

Next Episode, I will be talking about 2020. I think it’ll be interesting to deconstruct this shit show of a year and share my insight.

Please share this Episode with others. Thanks so much!

Until next week —

Rock on!

Click here to listen on iTunes!

Get on the VIP insiders list!

Upload The Productive Artist e-book.

Allan McKay’s Facebook Fanpage.

Allan McKay’s YouTube Channel.

Allan McKay’s Instagram.