FFmpeg hevc_nvenc Rejects libx265 Preset Names, Causing Muxing Failure in udemy-downloader

Summary

When --use-h265 and --use-nvenc are combined in udemy-downloader, the application passes software-encoder (libx265) preset names such as slower directly to the hardware encoder hevc_nvenc. NVIDIA's NVENC encoder does not share libx265's preset vocabulary, causing FFmpeg to abort with Unable to parse "preset" option value "slower" and a non-zero mux exit code. Any video segments processed during that invocation are lost or incomplete.

Root-Cause Analysis

Confirmed Evidence

The log lines are definitive:

[hevc_nvenc @ 0000008001025800] [Eval @ 00000080001fd610] Undefined constant or missing '(' in 'slower'
[hevc_nvenc @ 0000008001025800] Unable to parse "preset" option value "slower"
[hevc_nvenc @ 0000008001025800] Error setting option preset to value slower.
[vost#0:0/hevc_nvenc @ 000000800062ed80] Error applying encoder options: Invalid argument

FFmpeg received -preset slower targeted at hevc_nvenc. The encoder rejects the value because slower is a valid preset only for the software encoder libx265, not for any NVENC codec.

Root Cause

The downloader code applies a shared preset mapping regardless of which encoder backend is selected. When --use-nvenc is active, the encoder changes from libx265 to hevc_nvenc, but the preset string passed to FFmpeg is not translated to the equivalent NVENC preset vocabulary.

libx265 preset scale (slowest → fastest): veryslow, slower, slow, medium, fast, faster, veryfast, superfast, ultrafast

hevc_nvenc preset scale (slowest → fastest, FFmpeg ≥ 5.x naming): p7, p6, p5, p4, p3, p2, p1 Legacy aliases still accepted by some driver versions: slow, medium, fast, hq, hp, bd, ll, llhq, llhp, lossless, losslesshp

slower does not exist in either the modern (p1–p7) or legacy NVENC alias sets; hence the parse failure.

Secondary Issue (No Compression Without --h265-preset)

When --h265-preset is omitted and --use-nvenc is used, the reporter observes a file produced "without compression." This is a reasonable inference that hevc_nvenc is running with its default quality/rate-control settings, which default to a Constant QP of 0 (lossless or near-lossless) unless a rate-control mode is explicitly specified. The file is technically HEVC-encoded but at an extremely large size. This is a separate but related configuration gap.

Resolution Steps

For End Users (Immediate Workaround)

1. Stop using libx265-style preset names with --use-nvenc.
2. Substitute a valid hevc_nvenc preset. The modern driver-agnostic names are p1 (fastest) through p7 (slowest/best quality). The approximate equivalents are:

| libx265 preset | hevc_nvenc equivalent | |—————–|————————| | ultrafast | p1 | | fast | p2 | | medium | p4 | | slow | p6 | | slower | p7 | | veryslow | p7 |

1. For meaningful compression (not near-lossless output), also supply rate-control options. Example FFmpeg arguments when NVENC + H.265:

   -c:v hevc_nvenc -preset p6 -rc:v vbr -cq:v 28 -b:v 0

Adjust -cq:v between 18 (visually lossless) and 35 (aggressive compression).

For Developers (Code Fix)

1. Locate the section of the downloader that constructs the FFmpeg argument list for encoding.
2. Add a branch that detects --use-nvenc and maps the user-supplied preset string to its hevc_nvenc equivalent before passing it to FFmpeg.
3. Also conditionally inject NVENC-appropriate rate-control flags (-rc:v vbr -cq:v <value>) when --use-nvenc is active and no explicit rate-control override is present.
4. Validate that the preset value is in the acceptable set for the selected encoder before constructing the subprocess call, and surface a clear error message to the user if it is not.

CLI Commands

Verify your NVENC encoder and its accepted options:

ffmpeg -h encoder=hevc_nvenc

Test a manual NVENC encode to confirm the GPU pipeline works independently of the downloader:

ffmpeg -i <INPUT_FILE> \
  -c:v hevc_nvenc \
  -preset p6 \
  -rc:v vbr \
  -cq:v 28 \
  -b:v 0 \
  -c:a copy \
  <OUTPUT_FILE>

Check which NVENC presets your driver version accepts:

ffmpeg -h encoder=hevc_nvenc 2>&1 | grep -A 30 "preset"

List available hardware encoders on the system:

ffmpeg -encoders 2>/dev/null | grep nvenc

Configuration Snippets

If the downloader exposes an FFmpeg argument passthrough or a configuration file, use the following as the NVENC H.265 block instead of a raw preset string:

# Example downloader config — field names will vary by project
encoder: hevc_nvenc
encoder_options:
  preset: p6          # valid hevc_nvenc preset (p1–p7)
  rc: vbr             # variable bitrate rate control
  cq: 28              # constant quality target (lower = better)
  b:v: 0              # disable bitrate cap, let CQ drive quality

For a software fallback (libx265), the equivalent is:

encoder: libx265
encoder_options:
  preset: slow
  crf: 28

Verification

After applying the correct preset and rate-control flags, confirm the fix:

# Inspect the output file's codec and encoding parameters
ffprobe -v error \
  -select_streams v:0 \
  -show_entries stream=codec_name,codec_long_name,bit_rate,profile \
  -of default=noprint_wrappers=1 \
  <OUTPUT_FILE>

Expected healthy output:

codec_name=hevc
codec_long_name=H.265 / HEVC (High Efficiency Video Coding)
profile=Main
bit_rate=<a non-zero, reasonable bitrate — not near source bitrate>

Also verify that the file size is significantly smaller than a comparable uncompressed or H.264 source, confirming that compression is active and not near-lossless.

Check for the absence of NVENC error lines in the downloader log:

# No output from these greps is the success condition
grep -E "Unable to parse|Error setting option|non-zero exit code" <DOWNLOADER_LOG>

Rollback indicator: If the output file is absent or zero bytes, the preset value is still invalid. Revert to -preset p4 (a safe, universally accepted NVENC default) and retry.

Prevention

1. Encoder-aware preset validation: Add an allow-list check in the downloader's argument builder. Reject invalid preset/encoder combinations at startup with a descriptive error, before any segment download or FFmpeg subprocess is spawned.

1. Separate preset flags per encoder backend: Expose --nvenc-preset and --sw-preset as distinct CLI options rather than a single --h265-preset, making the distinction explicit to users.

1. Default rate-control for NVENC: When --use-nvenc is active and no rate-control mode is specified, automatically inject -rc:v vbr -cq:v 28 -b:v 0 so that the default output is compressed at a sane quality level rather than near-lossless.

1. CI smoke test: Add an integration test that invokes the downloader with --use-h265 --use-nvenc against a short synthetic clip (using a software NVENC stub or a CI runner with a GPU) and asserts a zero exit code and an output file whose bitrate is below a defined threshold.

1. Driver and FFmpeg version pinning: Document the minimum FFmpeg version and NVIDIA driver version required for p1–p7 preset support. Older drivers may only accept legacy aliases (slow, medium, fast), and the allow-list should account for that difference.