image sequences to movie... best results?

[RRIS]

Hi all, this is technically not a Keyshot question, but I'm struggling to get smooth crisp video from various image sequences I've rendered with Keyshot.

I'm rendering at 1920x1080 / 25 fps. After assembling these image sequences in Premiere Pro I get choppy videos in my final export. Frames get doubled, no matter if I export to 25 fps, or increase to 29,97 or 30 (using all 3 options for frame interpolation in Premiere Pro 2019).
Even importing an image sequence and changing the sequence settings to 30 fps, then rendering it at 30 fps gives me choppy video.

The only thing that has worked for me (and don't ask my why..) is to open the image sequence in photoshop, setting the frame rate to 30 and exporting it with Field Order set to Upper First. Progressive gives me choppy video as well.
I've tried this with 3 sequences, threw them into Premiere Pro, rendered out with the same settings as the sequences and finally got something decent. My concern is that exporting from Photoshop, I have very little control over stuff like bitrates and the resulting video has too much compression for my liking.

So, my question to those more experienced in video. What can I do to get a good result out of Premiere Pro?

[mattjgerard]

18 year video editor here-

That's odd that you are getting that. There are quite a few variables, so here we go-
What is the output codec you are exporting from Ppro, and what bitrate?
Is this a problem with playback chop or are there actually duplicated/skipped frames?
How are you loading the frames into Ppro, as a frame stack, or just loading individual frames into the timeline?
It sounds like you are getting what used to be called "judder" which happened when incorrectly converting 29.97 to 25fps, or 24p to 30p, and such. Its not as big of a deal anymore since very few playback systems are locked to 29.97 anymore. You should be able to load in any number of frames, set your sequence frame rate and it will playback those frames at that rate. The result is that the frames will play back faster or slower depending on the fps.

Ex- you render out your images at 30fp in KS, in premiere setting the seq to 30fps will play back at the same rate, setting it to 60fps will play it back at 2x speed, 15fps will play back at 0.5x speed. Makes sense? Where it gets confusing is where you render out at 25 fps and your seq is at 30fps. Now, if you just drag all those frames into the seq, no problem. If you import them as a frame stack ( you get one file in the bin that looks like a stack of frames) and drop that into the seq, it will try to conform that clip to 30fps. And now, since you already told Premiere that you want that clip at 25fps, it will try to keep that clip paying at the same speed on the 30fps timeline. This means that it has to fill in 5 frames every second with duplicate frames. Hence, "judder".

Be sure to load your frames in as a single clip (framestack) then, to create the sequence, drag that clip down to the bottom of the bin and drop ont he New Item doc icon, this will create a new sequence with that clip's specific settings (fps, frame size, codec, etc.) then it should playback perfectly and export perfect.

That is if you are leaving the default export options as they are. Don't change the frame rate in the export options. That will mess with it too. Nowadays there is little reason to not use the default templates for whatever target your video is going to. Long time ago There were no presets, and you had to set everything manually, but now just stick with Mp4's for 99% of stuff. Youtube, Vimeo etc are all going to recompress your video anyway, so just feed them high bitrate high quality source files. Unless you are delivering to someone that specifically requires a different codec, stick with Mp4's.

I can go into further detail after we try some trouble shooting. You can also shoot me the frame stack via google drive or dropbox or something and I can try it out if you want to do that too. Shoot me a message if you want to go that way.

Cheers!
Matt

[mattjgerard]

Oh, and no interlaced video. Everything is progressive. No upper field/lower field stuff. Unless you are displaying on an old scanline glass tube TV

[theAVator]

Ditto to what Matt said! I couldn't think of how to put it so eloquently. haha

[RRIS]

Hi Matt, thank you so much man. It's so easy to get lost in these settings, so I'm glad you can help.

18 year video editor here-

That's odd that you are getting that. There are quite a few variables, so here we go-
What is the output codec you are exporting from Ppro, and what bitrate?

I'm exporting to h264, 1920x1080.
High Quality 1080p preset.
Frame Rate 30 fps
Field Order progressive
Encoding settings:
Profile: High
Levels: 4.2
Keyframe Distance is set to the default 90
VBR 1-pass 20-24 (but I've tried CBR at 40, VBR at Target: 10 - Max: 20 and some other values as well)
Time Interpolation: Frame Sampling (I've tried Frame Blending and Optical Flow..  neither game me satisfactory results)

Is this a problem with playback chop or are there actually duplicated/skipped frames?

I tried to find out by exporting PNG's instead of rendering to video and indeed I found that there were duplicate frames. Both at 25 and at 30 fps.
When I opened a sequence in Photoshop and rendered to h264 there, I found the video much smoother. I had the video on repeat and oddly enough it would be a bit choppy on first playthrough, then second and third would be smooth, then choppy again on the fourth.. Consider the fact I can watch 2560x1440 video on this rig without hickups.

How are you loading the frames into Ppro, as a frame stack, or just loading individual frames into the timeline?

I import them by selecting the first image of the series, checking the image sequence checkbox in the import window and then letting Premiere Pro take care of the rest. In the clip library in Premiere I drag each clip (still named after the first image in that series) into the timeline and then proceed adding transitions and such.
By default Premiere creates 25fps clips from these image series.. no idea why. Photoshop actually asks me during the import what the frame rate should be.

My client has supplied two videos that I should use to open/close the movie with, these happen to be 30 fps. Sadly I ended up rendering my animations at 25 fps to gain some time. Ideally I would've rendered at 30 or 60 fps, but we didn't have the rendering capacity for that.
So, in my timeline I open and close with a movie clip that's 30 fps.

It sounds like you are getting what used to be called "judder" which happened when incorrectly converting 29.97 to 25fps, or 24p to 30p, and such. Its not as big of a deal anymore since very few playback systems are locked to 29.97 anymore. You should be able to load in any number of frames, set your sequence frame rate and it will playback those frames at that rate. The result is that the frames will play back faster or slower depending on the fps.

Ex- you render out your images at 30fp in KS, in premiere setting the seq to 30fps will play back at the same rate, setting it to 60fps will play it back at 2x speed, 15fps will play back at 0.5x speed. Makes sense?

Makes sense

Where it gets confusing is where you render out at 25 fps and your seq is at 30fps. Now, if you just drag all those frames into the seq, no problem. If you import them as a frame stack ( you get one file in the bin that looks like a stack of frames) and drop that into the seq, it will try to conform that clip to 30fps. And now, since you already told Premiere that you want that clip at 25fps, it will try to keep that clip paying at the same speed on the 30fps timeline. This means that it has to fill in 5 frames every second with duplicate frames. Hence, "judder".

Be sure to load your frames in as a single clip (framestack) then, to create the sequence, drag that clip down to the bottom of the bin and drop ont he New Item doc icon, this will create a new sequence with that clip's specific settings (fps, frame size, codec, etc.) then it should playback perfectly and export perfect.

Now I'm getting confused indeed not sure what a framestack is? For example, I import by selecting the first image in the series (I'll call it series now instead of sequence, because sequence is something Premiere uses as well). That image is now shown in my project as a06.0001.png (shown as 25 fps). Is that the framestack you mean? Creating a new item from this by dragging it on the new item icon will add a sequence to my project, also 25 fps. And it will add that sequence in my timeline.

That is if you are leaving the default export options as they are. Don't change the frame rate in the export options. That will mess with it too. Nowadays there is little reason to not use the default templates for whatever target your video is going to. Long time ago There were no presets, and you had to set everything manually, but now just stick with Mp4's for 99% of stuff. Youtube, Vimeo etc are all going to recompress your video anyway, so just feed them high bitrate high quality source files. Unless you are delivering to someone that specifically requires a different codec, stick with Mp4's.

Presets it is, but like I said, I tried ones with 30fps and ones with 25fps and still got the juddery playback and duplicate frames. Somewhere I must be doing something wrong.

I can go into further detail after we try some trouble shooting. You can also shoot me the frame stack via google drive or dropbox or something and I can try it out if you want to do that too. Shoot me a message if you want to go that way.

That would be fantastic. I will create a dropbox folder for you with 3 image sequences and the opening/closing video clips that the client want to show. I have around 40 shots in the final video but that would be overkill
I'll PM you the download link. I won't bother with NDA's, but you know..

Cheers!
Matt

You the man!

[mattjgerard]

Ok, one of the worst answers to a question like this- It worked for me, but not sure why.

Couple issues-
1) There does seem to be something goofy with that first animation image sequence. I have no idea why Ppro plays it jumpy, no matter the frame rate. After Effects however, the meat grinder of image sequences, played it nicely after it cached the seq and was playing back from RAM. Might be Ppro doesn't like png seq, not sure. But when scrolling through the seq with the arrow keys in Premiere, there were no dupe or skipped frames. Only when played with the space bar did it go wonky. And always at the same spot. I dont' have another editor to try, but Ppro gets a bit pissy sometimes without reason.

2) Mixing frame rates. This is a tough one, you have to pick one, and deal with the issues it causes. In this case I chose to use 30fps for the seq settings. The intro animation matched this and the indicator bar in the timeline dutifully turned green indicating no rendering/proxy playback needed. For the _06 seq however, using the clip settings to interpret the seq as default 25 fps caused problems when dropped into the 30fps timeline. Indicator turned yellow and even after rendering it played back with a stutter. This also caused the clip to play back a tad faster than most likely intended.

3) When I interpreted the clip as 30fps and dropped it into the 30fps timeline, it played smoothly for the most part. There was still a random glitch at the same point. The clip played back a titch faster, but probably not noticeable. At this point I dropped the other animations in the same way, and after rendering the timeline it played back fine. Exported as h264 mp4 at default high bitrate, and played back in VLC player just fine.

So, mixing fps is dubious. You can drop a 25fps clip into a 30fps seq or you can change the interpretation of the clip to 30fps THEN drop it into the 30 fps seq, which seemed to work better in this case. IT does change the speed that the clip plays back, but avoids the problem of inserting/removing frames that the video editor has to do to play it back properly. IF you have to have mixed frame rates, its best to use the Interpret Footage tool in premiere and change the FPS to the seq fps that you are targeting. You might still have some issues, but its usually the easier way to go.

The other problem that can be had in lower frame rate animations, is motion speed. Move the camera too much and your eye wants to fill in the big jumps between the frames. This has been proven in studies where a dolly move of various speed with the same 24fps frame rate caused different visual differences between people. Some people had no problem, some people reported jumpy frames, skipping frames when there were none. Once motion blur was added those problems went away.  The first animation had a camera move that might be a bit to fast for a linear non-motion blurred move. Try to slow the move down if you can. You can always speed it up later

FYI- the Frame Blending modes are for trying to interpolate those frames that need to be filled in or removed. Optical Flow is the most intense and can give the best results, but doesn't always work.

Sorry this was a long post, its -25f here, and most of the office is gone so I have some extra time on my hands

[theAVator]

Is it really just previewing it in the timeline that it's jumpy? Or does the actual exported clip look the same?
If you drop your playback resolution in the program monitor from full to like 1/2 or 1/4, does that smooth things out at all while previewing?


-> Matt, where are you located? It's -27 here, so we must not be too many states away. lol

[RRIS]

Ok, one of the worst answers to a question like this- It worked for me, but not sure why.

Couple issues-
1) There does seem to be something goofy with that first animation image sequence. I have no idea why Ppro plays it jumpy, no matter the frame rate. After Effects however, the meat grinder of image sequences, played it nicely after it cached the seq and was playing back from RAM. Might be Ppro doesn't like png seq, not sure. But when scrolling through the seq with the arrow keys in Premiere, there were no dupe or skipped frames. Only when played with the space bar did it go wonky. And always at the same spot. I dont' have another editor to try, but Ppro gets a bit pissy sometimes without reason.

Hi Matt, thanks for giving this a shot man! The fact that it's not just me struggling with these files is at least some comfort!
I do have After Effects here, looks like I may have to give that a try. Otherwise I'll do a batch conversion from PNG to TIFF and try that in Premiere.

2) Mixing frame rates. This is a tough one, you have to pick one, and deal with the issues it causes. In this case I chose to use 30fps for the seq settings. The intro animation matched this and the indicator bar in the timeline dutifully turned green indicating no rendering/proxy playback needed. For the _06 seq however, using the clip settings to interpret the seq as default 25 fps caused problems when dropped into the 30fps timeline. Indicator turned yellow and even after rendering it played back with a stutter. This also caused the clip to play back a tad faster than most likely intended.

3) When I interpreted the clip as 30fps and dropped it into the 30fps timeline, it played smoothly for the most part. There was still a random glitch at the same point. The clip played back a titch faster, but probably not noticeable. At this point I dropped the other animations in the same way, and after rendering the timeline it played back fine. Exported as h264 mp4 at default high bitrate, and played back in VLC player just fine.

So, mixing fps is dubious. You can drop a 25fps clip into a 30fps seq or you can change the interpretation of the clip to 30fps THEN drop it into the 30 fps seq, which seemed to work better in this case. IT does change the speed that the clip plays back, but avoids the problem of inserting/removing frames that the video editor has to do to play it back properly. IF you have to have mixed frame rates, its best to use the Interpret Footage tool in premiere and change the FPS to the seq fps that you are targeting. You might still have some issues, but its usually the easier way to go.

The other problem that can be had in lower frame rate animations, is motion speed. Move the camera too much and your eye wants to fill in the big jumps between the frames. This has been proven in studies where a dolly move of various speed with the same 24fps frame rate caused different visual differences between people. Some people had no problem, some people reported jumpy frames, skipping frames when there were none. Once motion blur was added those problems went away.  The first animation had a camera move that might be a bit to fast for a linear non-motion blurred move. Try to slow the move down if you can. You can always speed it up later

That's some good reading! Thanks for your insights man. Really helping me out here.
That last paragraph might indeed be the main problem. I may just have too big of a movement there no matter what the framerate. I may have to render that sequence again with less movement :/
I'm going to try to set all the clips to 30 fps and see if I can get some high quality exports from After Effects. Then try to finish up the project in Premiere Pro.
I'll update this post later to tell if it worked

FYI- the Frame Blending modes are for trying to interpolate those frames that need to be filled in or removed. Optical Flow is the most intense and can give the best results, but doesn't always work.

Yeah, any kind of frame blending caused problems on transitions between sequences and ghosting in more severe movement (again I may have been taking too large of a step between frames)..

Sorry this was a long post, its -25f here, and most of the office is gone so I have some extra time on my hands

Oof, that's cold. Take care man, and be careful out there!

[RRIS]

I installed MPC-HC media player here with the CCCP codec pack, that made a huge difference in the playback of the videos you made. 25 fps was ok, but 30 fps was exactly what I want

However, the movies I made still exhibited the same issues, so I'll still go ahead with the above mentioned approach. Just thought I'd let you know though about your clips

[RRIS]

Well, it seems it's indeed a problem with Premiere and how it handles PNG files.
I imported each image sequence in After Effects and rendered them to separate clips with the build-in renderer, not Adobe Media Encoder. Saved each clip as a Quicktime file, 30fps. Then rebuilt the animation in Premiere Pro and exported it with default h264 settings. Works like a charm now.
I have to say this experience has put me off from using Premiere quite a bit.. maybe I should give Davinci Resolve a go for the next project.

[theAVator]

I wonder, now that you say it might have issues with PNG, you could try just a quick render test (doesn't have to be perfect rendering) and see if other formats produce differing results. I think more out of the sheer fact that I didn't actually need PNGs with transparency, the last few animations I did I went to JPEG frame outputs. The latest one where I thought I should use transparency, I actually used a green background color in the renders and in Premiere used the Ultra Key to key it out.

Could be another way to go about it, although if you're using ground shadows or anything, that tends to get a lil tougher (can't say I've had great success with outputting ground shadows and transparency regardless). 

In any case, is it tied specifically to the file format of the frames?

[mattjgerard]

PNG files are fantastic and terrible at the same time. They just don't always play well. Editors like premiere aren't built to playback image sequences very well, they are tuned to playback full self contained movie files, so its not surprising that this is happening. Resolve will most likely be able to handle the image seq better, as they designed it from the ground up to read high bit depth huge arse files from editors and cameras. Plus its pretty cheap for what you get, might be worth to look at.

Don't get too discouraged with Premiere, every application has its shortcomings. Now that there is ProRes encoding available for windows, it would be killer for KS to offer that as a video output option. Then you'd be all set. That being said, shooting movie files out of AE from an image seq is an extra step, but not a huge one. It was pretty common to have to transcode footage from the slew of different camera formats to find one that any editor app would play nicely with.

I'll bet that premiere would play other format image seq with spotty success, its just not one of its strong points, Adobe figures if you are working with image seq, you will most liekly head straight to AE which handles them very well. Would be interesting how it would work to send an AE comp to the premiere timeline without rendering out a movie file.....

Yes, I'm in the Twin Cities, MN, and it was -24f this morning. Forgot to plug the block heater in my truck in and it was not happy to be cranked up. Should be the last of it, supposed to be 40 (above zero) on sunday!

[RRIS]

I wonder, now that you say it might have issues with PNG, you could try just a quick render test (doesn't have to be perfect rendering) and see if other formats produce differing results. I think more out of the sheer fact that I didn't actually need PNGs with transparency, the last few animations I did I went to JPEG frame outputs. The latest one where I thought I should use transparency, I actually used a green background color in the renders and in Premiere used the Ultra Key to key it out.

Could be another way to go about it, although if you're using ground shadows or anything, that tends to get a lil tougher (can't say I've had great success with outputting ground shadows and transparency regardless). 

In any case, is it tied specifically to the file format of the frames?

I like PNG's because they seem to be higher quality than JPG's for me, plus the alpha capability and relatively small file sizes are nice. I might just stick to TIFF for the next project.

[RRIS]

PNG files are fantastic and terrible at the same time. They just don't always play well. Editors like premiere aren't built to playback image sequences very well, they are tuned to playback full self contained movie files, so its not surprising that this is happening. Resolve will most likely be able to handle the image seq better, as they designed it from the ground up to read high bit depth huge arse files from editors and cameras. Plus its pretty cheap for what you get, might be worth to look at.

Don't get too discouraged with Premiere, every application has its shortcomings. Now that there is ProRes encoding available for windows, it would be killer for KS to offer that as a video output option. Then you'd be all set. That being said, shooting movie files out of AE from an image seq is an extra step, but not a huge one. It was pretty common to have to transcode footage from the slew of different camera formats to find one that any editor app would play nicely with.

I'll bet that premiere would play other format image seq with spotty success, its just not one of its strong points, Adobe figures if you are working with image seq, you will most liekly head straight to AE which handles them very well. Would be interesting how it would work to send an AE comp to the premiere timeline without rendering out a movie file.....

Yes, I'm in the Twin Cities, MN, and it was -24f this morning. Forgot to plug the block heater in my truck in and it was not happy to be cranked up. Should be the last of it, supposed to be 40 (above zero) on sunday!

Yeah I was playing with the idea to put the AE comp straight into Premiere, but then again.. limited time and why play with the devil?
It's been a nice learning experience for me, and at least now I have something of a workflow

[mattjgerard]

Yep, I like PNG's as well, its the right combo of uncompressed quality and the alpha channel all in one. If you aren't glued to premiere, then I would certainly give Resolve a try. I've always wanted to give it a go from the Colorist standpoint when I was a video editor, but that was way back when the processing power required was way above our smaller shop's budget. Now with ultra cheap GPU's and the whole dang software package under $300, there is no reason to not try it. I'm stuck with Creative Cloud here at work, and frankly I know all my workarounds for all the little bugs therin, so I have to stick with that. Glad you have a workflow now, hope it all works out for you!