Difference between revisions of "AV1 Codec Testing"

Revision as of 17:38, 11 March 2020

Last updated 3/11/2020

AV1 is the newest emerging video coding format currently under development by AOMedia. I was interested in trying it out and seeing how it compares to my current favorite x265, and also the older x264. Keep in mind I'm doing this for my own purposes, so the following benchmarks are not comprehensive and are limited to my own intended use cases.

The AV1 library libaom-av1 is not currently available in ffmpeg by default. I found a good script someone made for compiling ffmpeg with support for AV1: https://gist.github.com/sparrc/026ed9958502072dda749ba4e5879ee3

I downloaded the script:

wget https://gist.githubusercontent.com/sparrc/026ed9958502072dda749ba4e5879ee3/raw/e22698ead1984cd86b943f3473bd4bfb98591808/install-ffmpeg.sh

Note: As always, never download and run a script without reading it yourself to determine that it does not do anything malicious.

I ran the script:

sudo bash install-ffmpeg.sh

this script installs the custom ffmpeg into ~/bin/ffmpeg.

I chose a relatively small MTS file for testing:

du -h 00056.MTS

38M     00056.MTS

Runtime Benchmarks (Interlacing)

In the following benchmarks, the relevant time is the "real" time. The original MTS file is encoded with video:h264, audio:ac3, and has interlacing. Each encoding process I do below uses all defaults for the format, and encodes the audio to aac.

X264 Benchmark

time ~/bin/ffmpeg -loglevel -8 -i 00056.MTS -c:v libx264 out_x264.mp4

real    0m28.159s
user    1m43.388s
sys     0m0.176s

X265 Benchmark

time ~/bin/ffmpeg -loglevel -8 -i 00056.MTS -c:v libx265 out_x265.mp4

real    0m43.420s
user    2m40.852s
sys     0m0.304s

AV1 Benchmark

time ~/bin/ffmpeg -loglevel -8 -i 00056.MTS -c:v libaom-av1 -strict -2 out_av1.mp4

real    177m27.737s
user    413m16.620s
sys     0m6.688s

Note: The "-strict -2" parameter allows for the use of experimental codecs.

Conclusion

As you can see, the AV1 encoding took orders of magnitude longer.

• x264 = 28 seconds
• x265 = 43 seconds
• AV1 = 2 hours, 57 minutes, 27 seconds

Runtime Benchmarks (De-Interlacing)

In the following benchmarks, I decided to include de-interlacing.

X264 Benchmark (De-Interlacing)

time ~/bin/ffmpeg -loglevel -8 -i 00056.MTS -vf yadif -c:v libx264 outd_x264.mp4

real    0m29.251s
user    1m47.480s
sys     0m0.260s

X265 Benchmark (De-Interlacing)

time ~/bin/ffmpeg -loglevel -8 -i 00056.MTS -vf yadif -c:v libx265 outd_x265.mp4

real    0m42.207s
user    2m34.120s
sys     0m0.328s

AV1 Benchmark (De-Interlacing)

time ~/bin/ffmpeg -loglevel -8 -i 00056.MTS -vf yadif -c:v libaom-av1 -strict -2 outd_av1.mp4

real    162m54.203s
user    377m0.452s
sys     0m4.940s

Conclusion

The times are not significantly different from my first test that included interlacing, but interestingly choosing to de-interlace marginally reduced the runtime in every test.

• x264 = 29 seconds
• x265 = 42 seconds
• AV1 = 2 hours, 42 minutes, 54 seconds

File Size Comparison

The files starting with out_ are interlaced, and outd_ are de-interlaced.

du -h out_*;du -h outd*

11M     out_av1.mp4
15M     out_x264.mp4
5.6M    out_x265.mp4
8.2M    outd_av1.mp4
13M     outd_x264.mp4
4.1M    outd_x265.mp4

Conclusion

The best codec for file size, by a pretty large margin, is still x265. De-interlacing also resulted in smaller file sizes for all codecs.

Video Quality Comparison

In this test, I will take a cropped portion of the same frame from each file so that I can visually compare them.

I used ffmpeg to extract the 10th frame from each file:

for x in *.{MTS,mp4};do ffmpeg -i $x -vf "select=eq(n\,9)" -vframes 1 $x.png;done

Note: the "select" parameter takes a sequence starting at 0, so the 10th frame is 9.

I thought it was notable to look at the difference in file size of the images:

du -h *.png

2.9M    00056.MTS.png
2.1M    out_av1.mp4.png
2.0M    outd_av1.mp4.png
2.4M    outd_x264.mp4.png
2.1M    outd_x265.mp4.png
2.4M    out_x264.mp4.png
2.2M    out_x265.mp4.png

I then cropped a portion of each image starting at pixels X=0,y=250 with a size of 500x300. To do this automatically, it used the Imagemagick convert command:

for x in *.png;do convert $x -crop 500x300+0+250 ${x/\.png/_crop\.png};done

Image Comparison