Megan McLachlan speaks with The Irishman visual effects supervisor Pablo Helman about taking years off Robert De Niro.
Talk about a game-changer.
Netflix’s The Irishman, from director Martin Scorsese, recalls the possible involvement of Jimmy Hoffa’s death by rolling back time on its actors through state-of-the-art “de-aging” technology.
The film jumps around at different ages for characters, with Robert De Niro’s Frank Sheeran leaping around the most in the timeline, going from his 20s to his 80s, which is quite the visual achievement.
I talked with visual effects supervisor Pablo Helman about what makes this technology so advanced, what it says about themes in the film, and what it means for the future of visual effects.
Awards Daily: You’ve worked on plenty of special effects, but nothing quite like this. How did this project come about for you?
Pablo Helman: I was working with Marty on Silence and we were in Taiwan working, and Marty loves talking about film. So we started talking about technology, and I knew that at the time I thought he was going to do a Sinatra project, and I kinda pitched to him the idea of doing a younger actor [making an actor look younger]. He said that sounded good, and we started talking about technology, and then he said it sounded like a project he was trying to make for a long time called The Irishman. He sent me the script overnight, and I read it overnight, and in the morning, I told him I was in. It was a great project. Great way to not only further technology, but it’s a project that’s about content and performances. I’ve been doing visual effects for about 30 years, and it’s very difficult for a project to come with content, with performances. So then I called ILM [Industrial Light and Magic], and I told them about the project, and at the time it was only De Niro that we were going to do. The first answer was, ‘That sounds great, but it’s pretty risky.’ I talked to Dennis Muren, who has eight Oscars. He did Jurassic Park. His office is right next to mine. He said, ‘It’s a little risky.’ And I said, ‘Do you remember how you felt when you did Jurassic Park?’ And he was quiet for a little bit, and then he said, ‘Ahh, you’re right. You should do this.’ Then we did a test with a scene from Goodfellas, where De Niro came in, did the scene at 74 years old, and we brought him back to 40s and 50s. It was about 12 seconds, and it took about 10 weeks. After we showed it to Marty and Bob De Niro, basically the movie got greenlit.
AD: Did the production team ever think about using younger actors or makeup? Was it always just doing special effects?
PH: We talked about that and we also talked about maybe even to have a younger body doing what the actors did in different takes and replacing the faces. But it turns out the actors are so much into performances and Marty is so into the performances; they also ad-libbed a lot of the script. It was very difficult to bring other actors in. The way the movie was going to be edited, it was going to be so much back and forth. We knew Pesci was going to be in and we knew Al Pacino. That meant three different changes, plus all the edits, and that seemed like it was impossible [to include other actors]. Basically we decided we were in for it. Bob and Marty said the same thing, ‘We’re not going to wear any markers on our faces; we’re not going to wear the pajamas and the helmets; we’re not going to do a performance outside of the set.’ It took about two years for us to work out the marker-less system for the actors, so they don’t have to wear anything. They could be on set with theatrical lighting, and also together with the software system we created, we had to develop a three-camera rig to shoot the movie, so we could get all the information.
AD: The movie was delayed because of the de-aging technology. What kind of advancements did you have to wait for? Was it just being marker-less?
PH: Yeah, it was that. We were finishing Silence at the time, so I’d been working on this since 2015, so about four years. We developed the software, again its a marker-less software that works by sampling the lighting and the texture on the face of the actors and creates 3D geometry based on the lighting. At the same time, we developed a rig with a center camera for the director and then two cameras right next to the director camera on the same rig, and they are infrared cameras that throw infrared light onto the actors so it takes away the shadows, but in a different spectrum. With the infrared spectrum, the director and the DP don’t get bothered by it. The software takes a look at those three cameras and the lighting of the cameras and triangulates the expression in a particular frame and looks at the next frame and does the same thing and so forth until you finish the take.
AD: How do you think de-aging these actors plays into the themes of the film?
PH: The movie is about many things, but one of them is the passage of time. So it’s very fitting that Marty came up with this idea of making the actors younger, cause it has a lot to do with the theme of the movie. And also, the design of the characters. Marty didn’t just want to rewind 30 years and see Jimmy Conway from Goodfellas or Joe Pesci in Home Alone. He just wanted to see a younger version of Frank Sheeran and Russell Bufalino and Jimmy Hoffa. The idea was that even though these actors might not look exactly the way they looked 30 years ago, they look like the younger version of those characters. I know you have to get used to it a little bit, but I think it fits the movie, and it fits the characters.
AD: Yeah, it does. How many years did you take off? How far could you go? Which years did you focus on?
PH: De Niro starts at about late 20s, when he’s in the war. It wasn’t necessarily a rendition of Taxi Driver or Mean Streets. It’s basically a rendering of him as a younger actor. The original character that he’s portraying, Frank Sheeran, he was this 6’4″ man, 240 pounds, so [De Niro] didn’t have to be exactly the same. So De Niro starts at late 20s, then he goes to 36, then 41, 42, 47, 65, 62, then he goes into makeup. The first part, we did it CG, and then the rest was makeup, 60-something to 83 when he dies. Pesci starts at 53 in the movie and until he was like 65, and then after that, makeup picks it up. Then Al Pacino, we did all of it in CG. He didn’t want to go into prosthetic makeup, because he spans 44 to 62. We did the whole range of the performance in CG.
AD: Wow. I wasn’t sure how much you did with Pacino. It was so seamless. This is kind of a random question, but I know that De Niro’s eyes are blue in it. Did he wear contacts or did you do special effects for that?
PH: He wore contacts for the last part of the movie. We had to match him to that. Again, it was a Marty requirement, because the original character had blue eyes, very piercing blue eyes. And it’s not like anyone would know that, but for Marty, it’s about finding the truth, in the character and also the environment. We talked about the fact that Bob is such an iconic person and everyone knows him and knows he has greenish brown eyes. But he said it was really important to see a good version of Frank Sheeran.
AD: Whether your work would be good or bad, were you concerned about it being distracting?
PH: Yes, definitely. It’s always something you think about in terms of visual effects. But the software was something that we spent a lot of time on. My son always says, ‘Don’t read the reviews,’ but I do read them. One of the things that comes across in all of the reviews is that the performances are incredible. If we weren’t doing our job, the performances wouldn’t come through. It’s very easy for us to make a change and take something away from the performance. That was really important for Marty and Bob, that we kept all the performances and not change a blink. It’s a subtle difference from a smile to a wince. You can really change the emotion and the performances, but we didn’t want to do that, so because of that, there’s no keyframe animation. That’s another thing that’s different from any other project. There were no animators changing any of the performances, but the computer was sampling.
AD: You kind of touched upon this earlier, but were there discussions about physicality that goes with the effects? Did De Niro try to change his body language to appear younger?
PH: Yeah, we talked about that. At first we talked about how playful a younger person could be. You can see in one of the shots at the beginning of the movie, he puts his hand on a wall or something, waiting there playfully. But we also knew that it was going to be difficult to everyone to pay attention to the physicality, so we brought in a person, we called him a movement analyst, that was there on set to work with the actors from the beginning, early in the morning all the way through the takes. He would be on top of what the characters should be doing at different ages. Then throughout the movie, we did some work, we retouched some of the bodies—not necessarily the performances, but the way they look. We let them gain weight throughout the movie and thinned them out in the beginning. That was part of the physicality aspect of it. We were thinking about that. There’s also the fact that this movie is made from a different point of view. The whole movie plays like a memory from an early time in his life.
AD: How do you see this impacting other movies in the future?
PH: I think we all knew this marker-less technology is a natural progression in visual effects. Visual effects, we stand on each other’s shoulders to get to different levels. But like any achievement, it’s measured against what it will do to the industry later on. The achievement is having the actors do what they do and perform what they need to do without interference.
The Irishman is now streaming on Netflix and is in theaters nationwide.