Recently, I accepted a Fiverr project to needed to sharpen and enhance a video. My Topaz Labs Video AI was unable to do anything useful, so I tried converting the entire video to photos and running through through their photo tools like Photo AI. Using the open source ffmpeg, I did it like this:

Movie to Images using ffmpeg

ffmpeg -i “input.mp4 frames-full/frame%d.jpg

That command creates jpg image files in a subfolder called “frames-full” for every frame of the original video. In my case, the original video was shot at 59.94 frames per second and created a ton of image files.

Processing 20,000+ images was going to take days, so I decided to just process it as 29.97 frames per second (standard US video rate) which would cut my processing time in half. To do this, I needed to delete every other image. I was able to use the “find” command to search for any matching filename that ended in 0, 2, 4, 6 or 8 (an even number):

find . -name "frame[0-9]*[02468].jpg"

One neat thing about the modern macOS file system is you can make a duplicate of a folder before doing something dangerous like command line deletes! The duplicate doesn’t take up extra space (beyond a bit of directory overhead) so that let me experiment over and over until I figured it out.

Now I had frame1.jpg, frame3.jpg, frame5.jpg and so on. After processing these files, I would need to re-assemble them using ffmpeg at the 29.97 frame rate. Here is the command for that:

ffmpeg -framerate 29.97 -I frames-full/frame%d.jpg -pix_fmt yuv420p output.m4v

Unfortunately, it expects all the files to be sequentially numbered (1, 2, 3, 4) and would not work with every other file missing.

To fix that, I needed a script that would rename files down from…

frame1.jpg
frame3.jpg
frame5.jpg
frame6.jpg
frame9.jpg

…to…

frame1.jpg
frame2.jpg (was 3)
frame3.jpg (was 5)
frame4.jpg (was 7)
frame5.jpg (was 9)

A bit of trail and error led me to this simple script that divides the number by 2, and since there is no floating point, all I needed to do was add one to the number and then do the division. 3 became 1+3 which is 4, which divided by 2 became 2:

1 -> 1+1 = 2 / 2 = 1
3 -> 3+1 = 4 / 2 = 2
5 -> 5+1 = 6 / 2 = 3
7 -> 7+1 = 8 / 2 = 4
9 -> 9+1 = 10 / 2 = 5

That let me simply rename the original number to the new number and get what I wanted. Here is the script, though it is hard coded to the number of files I needed. You’d have to change that since it’s not smart enough to figure this out (like, “stop when you find a file that doesn’t exist”). Also, it starts at 3 since frame1.jpg would be renamed to frame1.jpg which probably produces an error:

#!/bin/bash

# find . -name "frame[0-9]*[02468].jpg"

# Delete even-numbered files
for ((i=3;i<=26999;i=i+2))
do
    file="frame${i}.jpg"
    j=$((i/2+1))
    if [ -f "$file" ]; then
        echo "Renaming $file"
        mv "$file" "frame${j}.jpg"
    fi
done

I just wanted to post this here in case anyone else ever needs to do the same thing…

Until next time…

I recently had a request on Fiverr to try to shaper up an out-of-focus video. I thought I might be able to do it using Topaz Labs Video Enhance AI. After several attempts, I concluded that this was not the right tool for the job. While it did have the ability to sharpen video, even on the highest settings it was not sufficient.

Instead, Ida at Topaz Labs support suggested converting the video in to individual images and processing them through their Topaz Labs Sharpen AI product. After a number of attempts to convert the video to individual images, I ended up using the open source FFmpeg program.

Convert movie to images

Using the option “-i” to specify the input video file, and “-vf” for the video filter with the options “fps” for how many frames per second to process and finally the output filename format.

For fps, if you want to capture every frame of the video, set this to the frames per second of the video file. In this example, the original video was 25 frames per section:

ffmpeg.exe -i VIDEO.MP4 -vf fps=25 out%d.png

For the output filename, you can use C printf-style variables such as %d so it knows where to place the number in the filename. Above, that would produce “out1.png”, “out2.png”, and so on.

Once you have all the frame images, they can be imported in to Sharpen AI and processed. This will produce new output filenames adding “-SharpenAI-Focus” or similar to the filename. You can then reassemble those individual images back in to a movie.

Convert images to movie

The option “-framerate” is used to generate that many frames per second in the new movie file. It should match the -fps value used above.

“-i” is used to tell FFmpeg what filenames to look for. It will also use the C printf-style parameters. Since Sharpen AI takes “out1.png” and creates a new file called “out1-SharpenAI-Focus.png” (when using the focus mode), I needed to match that filename.

The first video I created would not play, and I found setting “-pix_fmt” would make a playable MP4 file.

ffmpeg.exe -framerate 25 -i out%d-SharpenAI-Focus.png -pix_fmt yuv420p NEWVIDEO.MP4

There are other options that can be used to set quality levels, so I may revise this article in the future when I use this technique the next time.

Until then…

When you slow down video by increasing the time between each frame, you get chunky video. To get smooth slow motion, the video is recorded at a high frame rate. For example, instead of a normal 30 frames per second like US TV, the video might be recorded at 120 frames per second. Now you can slow it down to 1/4 speed and still be displaying 30 frames per second.

But what if you have footage that was not shot at a high frame rate and you still want smooth slow motion?

AI video processing can take care of this. Below is a short clip I recorded.

On the left is the original video slowed down in a video editor. Notice the pauses between each frame.

On the right is the same video, processed by Topaz Labs Video Enhance AI, where it CREATES frames in between the ones in the original video! Check it out then I’ll add some more details…

What did you notice? Did it look good?

Watch it again, and this time, focus on the wheels of the car that passes on the left side. Also pay attention to the lettering on the side of the white van on the right.

Notice the defects? The AI does pretty amazing things, but still has artifacts that let us know it’s being processed.

Still, very neat.

I have Topaz Labs Video Enhance AI and have been converting hundreds of hours of my old Digital8/DV home videos. You can learn about this software here:

https://www.topazlabs.com/video-enhance-ai

If you can’t justify the cost of this software, I’d be glad to convert some videos for you. If the video is already high quality (like a 720p HD clip from a phone), converting it up to 1080p HD can look really good. If it’s older video (SD footage or old analog VHS stuff), it may not be quite as impressive, but still better than if you just scaled up the size in a video editor.

Drop me a note here, or you can check out samples at my Fiverr listing:

https://www.fiverr.com/allenhuffman/upscale-video-using-topaz-labs-video-enhance-ai