Jason WeatherlyOh yeah, things are working as intended now. Unfortunately, I did crank the settings so these encodes are going to take about an hour each. However, the homelab seems to be up to the challenge.
After breaking out `strace` and just a ton of other stuff I finally figured out what was going on. A single encoder option was invalid which caused the workers to crash before even writing the file in the data directory. Fixing the setting cleared up the issue and made everything happy.
What caused this rabbithole initially was `av1an` eating itself when trying to put the video and audio stuff together at the end of an encode. Unfortunately the `audio.mkv` file it was trying to mux into the container kept causing `mkvmerge` to fail because the audio file was empty. _This_ was caused by bad audio encoder options being passed to `ffmpeg`.
I made sure I documented both of these and I'm currently running test videos before calling it 100% good.
I think this might be the most frustrating thing I've dealt with: the `av1an` encoder not being able to create files in the client container. Definitely has free space, the user is running in the container as `root` (also does the this as a regular user) but absolutely refuses to run.
The mystery continues.
So, got the server in place but discovered that the Docker image that I created that has the Sisyphus client and all of the binaries like `ffmpeg`, `av1an` was not very happy. Ffmpeg crashed because it couldn't find the `libSvtAvcEnc.so.4` library which was because I had a custom version of it installed (`svt-av1-psyex`).
Got the Dockerfile fixed by installing `svt-av1-psyex` and then compiling `ffmpeg` against those libraries, then installing both `svt-av1-psyex` and `ffmpeg` into the final container which makes `ffmpeg` happy. Also saw an issue where the Vapoursynth `lsmash` module wasn't being found, but that got tracked down to a stale Docker image (forgot to pull the latest from the repo).
I have two encodes going: one `ffmpeg` and one `av1an` which should be the real final test. If they turn out well, attachments where they're supposed to be, etc., etc. then I'll probably start working on documentation and get this out there.
#sisyphus #encoding #av1an #vapoursynth #ffmpeg #svtAv1Psyex
Having trouble with my container, seems like it's using the incorrect version of the `SvtAv1EncApp` which is really annoying. Went ahead and just slammed the `psyex` version into `/usr/bin` and now everything is happy.
After what can only be considered mean to the hardware, the Sisyphus encoding client version 1.6.3 is released with full support for Av1an. While I won't guarantee it won't eat your pets, I can say that I've encoded enough using the Av1an module that I'm confident you'll at least get some nice videos in return.
The documentation has also been updated as well on the main site along with the README.md with some better directions on getting it up and running.
Link: https://git.jamesthebard.net/jweatherly/sisyphus-client
I've got initial `av1an` support into the Sisyphus encoding system. Updated the Dockerfile for `av1an`, added `svt-av1-psyex` with HDR10+ and Dolby Vision support, and even installed the `lsmash` Vapoursynth plugin for fast scene detection/chunking. I still need to get `ffmpeg` compiled with support for `svt-av1-psyex`, but I'll deal with that in the future once I get encodes running and working.
Repo: https://git.jamesthebard.net/jweatherly/sisyphus-client/src/branch/av1an-psyex-support
Alright, started work on the `av1an` module for my distributed encoding server. First up, getting a proper JSON format to set the command-line options. The schema is about 320 lines long, but it does lend itself to some nice data formatting. A basic encode "job" block looks like this and actually generates the appropriate command-line arguments.
```json
{
"source": "video.mkv",
"destination": "test.mkv",
"general": {
"proxy": "test.pyv",
"log-file": "/home/jweatherly/ffmpeg-test/output.txt"
},
"encoding": {
"video-parameters": [
"--preset 2",
"--crf 25",
"--noise-adaptive-filtering 4"
],
"audio-params": [
"-c:a libopus",
"-b:a 128k"
],
"chunk-method": "lsmash"
}
}
```
Final tests in and the quality is outstanding. Going from the stock `svt-av1` encoder to the `svt-av1-psyex` encoder is already a nice bump in quality, and the encodes are faster using `av1an` due to using every ounce of the CPU. The `lsmash` speedup is nice as well as the stock method for chunking took a bit longer post-detection.
Also, can't say enough about the Ryzen 9 HX 370: did a great job with the encodes and only burned through ~24W during the encodes along with the Thinkpad that kept the temps around 68C.
Doing a test with `av1an` and `SvtAv1EncApp` (`svt-av1-psyex`). This poor laptop is absolutely getting hammered at the moment...lol
