DP Russell Carpenter on the Virtual Reality of ‘Avatar: The Way of Water’

Welcome to World Builders, our ongoing series of conversations with the industry’s most productive and thoughtful behind-the-scenes craftspeople. In this entry, we chat with cinematographer Russell Carpenter about Avatar: The Way of Water and how modern cameras made the impossible possible.

Weeks before production began on Avatar: The Way of Water, Sony unveiled their Venice camera package, allowing unparalleled flexibility in which filmmakers could cram their shots. The optical block is separated from the camera and mounted on a lightweight 3D rig at a distance of up to twenty feet. James Cameron and Sony have a very tight relationship, and the company was eager for the director to test out their baby and push it as far as they knew he could.

Cinematographer Russell Carpenter suddenly found himself working a system that squeezed into tight areas that his 2D cameras on Titanic and True Lies would never fit. The camera could operate next to the actors’ faces, breaking the emotional distance often created in 3D ventures. Cameron, who frequently prefers to be the one slinging the shooter, was also able to remain near the performers.

In addition, Sony and Cameron wanted to shoot 48 frames per second, which is double what audiences normally experience cinematically. With early access to the Venice package, Carpenter could test, test, test. He fell hard for the camera’s mobility, but he was equally enamored with its dynamic range and color depth.

“We go back to Jim,” says Carpenter, “and we show him our test. He says, ‘Yeah, yeah, I get this.’ We’re working with a 3D system that is inherently going to, at least, take a stop off the light that finally gets to the sensor. But, also, we are now shooting at 48 frames per second. So, there’s another fairly significant loss of light.”

Their lighting plans had to change on Avatar: The Way of Water. Carpenter’s lighting technician, Len Levine, proposed a method of moving the lights via remote control. As such, they could make adjustments instantaneously, shifting not just their direction but their quality, color, and intensity. The precision and speed were a game-changer, and it came from Levine and Carpenter visiting Cameron while he was working in the virtual world.

“Jim’s in a system where he has an amazing amount of control,” he says. “He’s maybe got 23 people at computers. So, if he says, ‘I need that Na’vi just a little bit tinier,’ and the framing doesn’t work, ‘let’s move him just a little bit to the left and lower. Oh, hey, that waterfall isn’t quite working for me there; let’s move it two feet.’ We were watching this, and we were going, ‘Look at the flexibility he has! We have so many different scenarios to deal with; how can we give him something similar?'”

Considering the light source on Pandora is more complicated than your usual environment. The planet itself shines, smashing various colors together during nearly every sequence in the film. Without Levine’s idea of lighting, the production would be ten times slower, if not twenty times slower.

“For instance,” Carpenter continues, “the Na’vi forest is full of things that are bioluminescent, a lot of green. The light comes through, strikes those, and then it comes up to, say, an actor walking through the forest or running through the forest. It comes up, and you see a bit of green on the actor as he runs through this patch of light. Instead of putting more lighting down on the floor, we would actually just put bounce cards down there, maybe large bounce cards. We could get exactly the right color in the right place.”

Shooting the stages was only half of Carpenter’s job. The other bit existed in the digital domain. He would shoot a chunk in live-action, then go to his computer set, and then back to live-action.

“One of the things I was tasked with right away,” he says, “was supervising the virtual lighting. Jim was out on the Volume, and he was setting up and blocking his scenes; he was working with his cameras. I’m in another part of the building. I’m working with a wonderful team of lighting designers, people who know CGI. I had to learn to ride the bicycle very, very quickly, the CGI lighting thing.

Carpenter worked with Gazebo, Weta’s in-house rendering software that uses real-world physics to surmise how various surfaces would react to specific lights. The cinematographer credits Visual Effects Supervisor Dan Cox with making the visual shoot a one-for-one experience with the live shoot. Although in the virtual realm, he could manipulate the sources in ways he couldn’t in reality.

“Somebody would flip a switch,” says Carpenter, “and the light would disappear, basically. You wouldn’t see it. You’d just see the effect of the light. It was a lot of fun.”

For Carpenter, part of that fun was adjusting his Earthbound notions of cinematography. He was working in an alien world, inspired by terrestrial locations but pushed to an extremity. The last thing he wanted audience members to recognize was their everyday industrial encounters with very bright lights.

“The world of Pandora is definitely different from the world of the RDA human-made things,” he says. “RDA lighting is like walking into a big box store, a Home Depot. It’s abrasive, and it’s not a lot of fun to be around, where, of course, Pandora is just this wonderful, nuanced, multicolor vision that Jim had. His mandate was, there’s a lot of color here, and I want you to start pulling the light apart. He said, ‘When you get to your sunset scenes, remember just start to fracture the light. Think of the things that Maxfield Parrish did in his work.’ The thing was, I knew later on, especially in the scenes with Spider, I’d have to match what I did with those wonderful people on Gazebo.”

Continuity proved challenging, but maybe not as big a challenge as Carpenter initially feared. Those Weta geniuses had his back, keeping track of whatever mad wizardry he was cooking up in their labs. When changes on set didn’t align with what he’d already completed in the virtual stage, they could adjust fairly quickly.

“The wonderful thing about Gazebo,” says Carpenter, “as opposed to what they had to work with on the first Avatar, is it literally tracks everything you did. You know the color temperature; you know exactly where the light was and why you did it. You have all those breadcrumbs that lead you back to your original vision when you were lighting. All that information goes along through the scenes. If Jim decides to make a change in how he’s blocking a scene, well, the lighting information is still there. If he does a different shot, the lighting information’s still there. That was a major thing.”

With so much focus on Avatar‘s digital creation, Carpenter doesn’t want the conversation around the film to fall exclusively into the virtual. He took James Cameron’s marching orders to heart. Pandora is alien, and the Na’vi are alien, but they’re ultimately our reflections. Separating the real from the unreal misses the point and is totally impossible when you look at how they accomplished the movie.

“You have to remember that everything in this film is based on something real,” he says. “You actually had real people in sensor suits with underwater motion capture systems. They’re doing everything. There wasn’t one thing that was like, ‘Well, we’ll just invent this creature.’ You had the Tulkun, that’s based on watching how whales move. Any animal is not some crazy concoction that somebody came up with at a CGI workstation. It’s all based on something real.”

When the audience stumbles from the theater, the cinematographer wants them to think about their surroundings. He doesn’t mind them having a flight of fancy, letting their imagination drift upward to the far reaches of space, but he hopes that they recognize the foreign as the domestic when they hang out on those fictional shores.

“Pandora really does stand in for how things are on Earth,” says Carpenter. “Light and all kinds of physical things and gravity, they’re the same as they are on Earth. That’s the touchstone.”

Avatar: The Way of Water is now playing in theaters everywhere.