First, Let Me Build a System
When I decided to document the process of completing all the VFX from scratch on Brothers’ Quarrel, I didn’t think about how much work the videos would be adding on top of everything. I also hit a massive snag as I started developing episodes, when I realized that one major step was… horribly tedious and not ripe for anything remotely educational.
Premiere is not designed for this
My method in recent years for exporting image sequences out of Premiere was straight-forward, albeit tedious as hell. But I figured it was just something you do once, so whatever. I would double-click a clip in the timeline, right-click it in the source monitor, and tell it to make a subclip. I’d paste a base name and change the shot number. Rinse and repeat.
Then I could grab all the subclips at once and export them with the same EXR settings… and a base output directory. But then I’d have to go down the line in Adobe Media Encoder and change the output of each clip so they’d render to their correct directories. Another tedious step.
Bending the Project Manager to my Will
There had to be a better way. And I found one… kind of. Turns out Premiere has a Project Manger that can do something I’ve been wanting, namely render handles of any length. There are a few “gotcha’s” though. For one, this tool is meant for archiving. It’s designed so that you can save the source media you absolutely need in order to keep this project renderable, while being able to trash all the original stuff to save space. It does this by re-exporting each clip in your timeline(s) to a format meant for editing, such as ProRes or DNxHD. No EXR or image sequence options. I mean, why would there be? You don’t edit in those formats.
BUT! I noticed you could import a preset, and these appeared to be the same presets Premiere uses in the exporter. Ah, think I, if one can save a preset to a file, then the existing presets in this list must also be files somewhere. Turns out they’re deep in this labyrinth of mysteriously named directories (see image). Spoiler: the EXR ones are at the end. Just start there.
So I was indeed able to load in the EXR preset into Project Manager, and we’re nearly in business. There’s this whole other issue with getting things named correctly, and a particular head-scratcher when using the same source clip multiple times in the timeline, but I got all that sorted and I’ll cover it in a YouTube video.
“That last fiber connection”
There’s a great anecdote on the appendices of The Two Towers about the remote workflow Weta built in 2002 so that Peter Jackson could review work being done in New Zealand while scoring in London. They managed to setup a high speed fiber connection going half-way around the globe, but for some reason, it stopped about a mile from where Peter was staying. (I’m sure there’s a fascinating story behind just that little tidbit).
The solution was to load whatever New Zealand sent onto an iPod, which at the time were just 2.5” external drives that could be side-loaded with anything, and then physically walk it down the street. We had the same sort of Sneakernet when I was on Captain America and Thor, but by design, since our main office had no internet connection for security reasons.
All this to say that often times we get a ton of pieces working together, and then end up with some break point right at the end, where there’s no clean way to finish the job. It happens all the time, especially with workflows in VFX. In my case, it should be expected. I’m using an archiving feature of an NLE that is not really a go-to for big budget, VFX-heavy projects, so no one at Adobe probably thinks this is a pressing need. And that’s fine with me. Premiere is what I have and know, so I’m going to make this work.
Enter Python!
For those that don’t know, I took a break from VFX for a couple years and got into web development. Specifically, I built WordPress themes so I spent a lot of time learning and working in PHP, JavaScript, jQuery, HTML, and oh so much CSS. I learned a lot from my Systems Administrator at the time, and he was a big fan of Python, so I wanted to learn that so I could talk shop with him more.
I fell in love with it. And when I came back to VFX, I lucked out in that Nuke and Silhouette both utilize Python. Oh the possibilities… I dabbled in customizing Nuke along the way, just sorta figuring stuff out as I went. I built some basic tools for myself that now probably litter the Los Angeles compositing departments (if you see something with Cow at the beginning, it was probably me). I went through Ben McEwan’s Python For Nuke 101 course, which I highly recommend. I’ve recently been getting far more obsessed with all this thanks to Ben, especially since he recommended Automate the Boring Stuff With Python (which is FREE!).
All this is driven by the fact that I need to make that last little fiber connection in my Premiere export workflow. Premiere will dump all the EXR files into one folder. Just thousands and thousands of files. So what I’ve been doing these past few weeks (months? What is time anymore?) is the first in what will probably turn into a full suite of tools. I need to look at all the files in the folder and move them into their respective shot directories.
Seriously. Not touching any of that without code.
This has involved learning several things, such as Regular Expressions to parse the filenames (basic string manipulation works too but is more prone to breaking, and harder to read), learning how to navigate and interact with the file system and using the os and pathlib modules, and how to actually screw with files. And between all this has been just learning how VS Code works, setting up virtual environments, learning how to properly install modules and making sure things run every time… you know, all the intimidating stuff that usually drives you away from getting too far into this in the first place. Or was that just me?
But I have some really specific things I need to do, and I need to do it to show it on my YouTube series. And I’m getting closer and closer to needing to put that episode out. So hooray for actual motivation! It may seem like I’m going overboard with all this. And I probably am. But I think going overboard with things is exactly how all my saber duels came about in the first place. It’s how I’ve ended up where I am now.
There is a certain hilarity in my learning like 15 other things so that I can avoid just having to shift-click 116 different groups of files and drag them into folders, a task that I could have done in one mindless day. I think Randall Munroe put it succinctly…
