Automating Encoding for My Gameplay Videos on Steam Deck

Last week, when I updated OBS on my Steam Deck, it stopped working on Game Mode. After a while, I decided to play games on Desktop Mode from then on. For some reason, the performance drops A LOT while recording on Desktop Mode, so it took a lot of trial and error before arriving at the best settings for recording.

On Game Mode, I used to record videos using the H.264 software encoding, but on Desktop Mode that caused a performance hit, so I changed my method to with H.264 'Low CPU' Mode. The problem with that is the file size. If I want to upload videos after recording them, I must re-encode such videos.

Video Encoding Setup

Initially, I was using libx265 for video encoding:

ffmpeg -i input.mkv -c:v libx265 -crf 23 -c:a copy output.mp4

I made a whole command line file (a .sh file for the Linux machine I'm using) that automates video conversion in the Video folder I have the recordings in:

#!/bin/bash

# Get current date in DD-MM-YYYY format
output_folder=$(date +"%d-%m-%Y")

# Create a subfolder named after the current date
mkdir -p "$output_folder"

# Loop through all .mkv files in the current directory
for file in *.mkv; do
    # Extract filename without extension
    filename="${file%.*}"
    
    # Convert using ffmpeg with H.265 codec, preserving quality (-c:v libx265 -crf 23 is generally recommended)
    ffmpeg -i "$file" -c:v libx265 -crf 23 -c:a copy "$output_folder/${filename}.mp4"
done

echo "Conversion completed. Files saved in $output_folder/"

After a while, I found it too slow for my needs. An hour-long video can take up to 3 hours to encode. 5 videos will take a full day! I couldn't leave my machine running for that long. I had to find a better way.

To improve performance, I switched to various libraries before settling on hevc_vaapi. With this, I'm leveraging hardware acceleration to encode videos more quickly while maintaining high-quality output.

Here’s the updated encoding command:

ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i input.mkv -vf "format=nv12,hwupload" -c:v hevc_vaapi -cq 22 -preset quality -c:a copy "output.mp4"

---

---

Black Bar Detection

Some games I play on Steam Deck take full advantage of its 16x10 (800p) screen, while other games have 16x9 resolution (720p.) All of my videos are recorded at 800p, so, encoding the 720p game videos with unnecessary black bars wastes too much computation power.

So, I used ChatGPT to devise a script that analyzes the top and bottom segments of the video to check if they're black or not.

The code ChatGPT gave me checked for the top and bottom 10% of the screen for black. It didn't work as I intended, so I updated the code to check the top and bottom 40 pixels instead. Since I'm only dealing with 720p resolution, that was enough:

ffmpeg -t 90 -i "$file" -vf "crop=in_w:40:0:0,blackframe=99:32" -f null - 2>&1 | grep "blackframe" | wc -l

The end script:

In the end, I updated my .sh file to deal with my Videos folder that I record games on Steam Deck to. Of course, I made ChatGPT print all the steps for debugging.

#!/bin/bash

# Get current date in DD-MM-YYYY format
output_folder=$(date +"%d-%m-%Y")
echo "Output folder: $output_folder"

# Create a subfolder named after the current date
mkdir -p "$output_folder"
echo "Created folder: $output_folder"

# Loop through all .mkv files in the current directory
for file in *.mkv; do
    # Extract filename without extension
    filename="${file%.*}"
    echo "Processing file: $file"

    # Analyze the top 10% and bottom 10% of the frame for blackness in the first minute
    echo "Checking for blackness in the top region..."
    top_black_bars=$(ffmpeg -t 90 -i "$file" -vf "crop=in_w:40:0:0,blackframe=99:32" -f null - 2>&1 | grep -o "blackframe" | wc -l)

    echo "Checking for blackness in the bottom region..."
    bottom_black_bars=$(ffmpeg -t 90 -i "$file" -vf "crop=in_w:40:0:in_h-40,blackframe=99:32" -f null - 2>&1 | grep -o "blackframe" | wc -l)

    echo "Frames with blackness in the top region: $top_black_bars"
    echo "Frames with blackness in the bottom region: $bottom_black_bars"

    # Check if there are 600+ black frames at top and bottom
    if [ "$top_black_bars" -ge 1500 ] && [ "$bottom_black_bars" -ge 1500 ]; then
        echo "Sufficient black bars detected. Cropping 40 pixels from top and bottom."

        # Convert using ffmpeg with VAAPI and applying cropping
        ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i "$file" \
        -vf "crop=in_w:in_h-80:0:40,format=nv12,hwupload" \
        -c:v hevc_vaapi -qp 27 -preset quality -c:a copy "$output_folder/${filename}.mp4"
        echo "Cropped and converted $file to $output_folder/${filename}.mp4"
    else
        # Convert without cropping if insufficient black frames are detected
        echo "No sufficient black bars detected. Converting without crop."
        ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i "$file" \
        -vf "format=nv12,hwupload" \
        -c:v hevc_vaapi -qp 27 -preset quality -c:a copy "$output_folder/${filename}.mp4"
        echo "Converted without cropping: $file to $output_folder/${filename}.mp4"
    fi
done

echo "Conversion completed. Files saved in $output_folder/"

This automation lets me seamlessly handle different resolutions in my Steam Deck gameplay recordings, preserving the right aspect ratio for each game. Converting videos with hevc_vaapi significantly sped up the process while ensuring that black bars are only cropped when detected in the specific 40-pixel regions.

What do you think?

---

^{- The post's images are generated with Ideogram! The second image's link.}