Skip to content

Instantly share code, notes, and snippets.

@DroidFreak32
Last active February 6, 2025 17:14
Show Gist options
  • Save DroidFreak32/740767f5123be8c02ab4a132ce6403ef to your computer and use it in GitHub Desktop.
Save DroidFreak32/740767f5123be8c02ab4a132ce6403ef to your computer and use it in GitHub Desktop.
ChatGPT NVENC + Openwrt FW Groovy

https://chatgpt.com/share/67a4b87c-a0fc-8007-b4c9-887145620c6b

Below are two example ffmpeg command lines—one using libx265 (CPU-based, software encode) and one using NVIDIA’s NVENC-based AV1 encoder—that meet your criteria. I’ve kept the original 708×480 resolution (with DAR 4:3) and used CRF/CQ settings in the visually lossless–to–near-lossless range (CRF around 18 for x265; and for AV1 NVENC we use a comparable constant‐quality “CQ” value).


1. Software Encoding with libx265

This command uses libx265 with a “slow” preset and CRF set to 18 (feel free to adjust between 18 and 20 if you’d like a tiny bit more compression versus quality). It also forces the pixel format to yuv420p (8‑bit) to match the source’s 8‑bit depth. (Since your source has a nonstandard resolution with a display aspect ratio of 4:3, ffmpeg should preserve the metadata—but if you ever need to enforce it you can add a video filter like -vf "setdar=4/3".)

ffmpeg -i "input.mkv" -c:v libx265 -preset slow -crf 18 -pix_fmt yuv420p "DragonBall_x265.mkv"

Explanation:

  • -c:v libx265 selects the x265 encoder.
  • -preset slow chooses a preset that balances encoding time with quality (and typically “slow” is a good choice for visually lossless results).
  • -crf 18 sets the constant rate factor; values between 18 and 20 should be visually lossless for this content.
  • -pix_fmt yuv420p ensures 8‑bit 4:2:0 output.

(See cite5†arch1t3cht for general encoding resources and discussion.)


2. Hardware-Accelerated AV1 Encoding with NVENC

Assuming your NVIDIA L4 GPU has the new AV1 NVENC support enabled, the following command uses the av1_nvenc encoder. Here we set:

  • -preset p7 (the slowest/most quality-oriented option for av1_nvenc as documented in NVENC AV1 guides),
  • -tune hq to optimize for high quality,
  • -cq:v 20 as a constant-quality parameter (roughly comparable in quality to x265 CRF 18–20; note that NVENC AV1 uses “cq” instead of CRF),
  • and a lookahead of 48 frames (which—for example, at 24 fps—amounts to about 2 seconds).
ffmpeg -i "input.mkv" -c:v av1_nvenc -preset p7 -tune hq -cq:v 20 -rc-lookahead 48 -pix_fmt yuv420p "DragonBall_av1.mkv"

Explanation:

  • -c:v av1_nvenc selects NVIDIA’s hardware AV1 encoder.
  • -preset p7 chooses the slowest preset (p7) for the best quality—if you find encoding time acceptable, this should yield high-quality output.
  • -tune hq tunes the encoder for high quality.
  • -cq:v 20 sets the constant-quality parameter; NVENC’s AV1 encoder does not support CRF, so –cq:v is used (a value of around 20 should be similar in quality to CRF 18–20 in x265).
  • -rc-lookahead 48 increases rate-control lookahead (helpful for better bit allocation).
  • -pix_fmt yuv420p ensures 8‑bit 4:2:0 output.

(For more on NVENC AV1 parameters, see the details in the GitHub gist by Nicklas373 and related discussions cite7†Nicklas373/av1_nvenc.)


Notes & Considerations

  • Nonstandard resolution & aspect ratio: Your source’s resolution (708×480) with a PAR of 177:160 yields a DAR of 4:3. In most cases ffmpeg will retain these details from the source. If you ever need to force the display aspect ratio, you can add a filter (for example, -vf "setdar=4/3").
  • Quality trade-offs: The suggested CRF of 18 in x265 is a common “visually lossless” setting for HD content. Similarly, the chosen cq value of 20 for av1_nvenc should produce a comparable quality output; you may wish to run a few tests if you want to fine‑tune the balance between file size and quality.
  • Encoding speed: Software x265 encoding on a 32‑core CPU will be considerably slower than NVENC AV1 (especially on a GPU like NVIDIA L4). If encoding time is a significant concern, the NVENC AV1 command may be preferred if its quality is acceptable.

Feel free to adjust the CRF/CQ values or presets as needed based on your subjective quality testing.

Do these commands match your intended workflow? Let me know if you have any further questions or need additional adjustments!


Deepseek

Let's break this down into key considerations and commands. I'll start with clarifying questions:

  1. Scaling Needs: The source is 708x480 with PAR 177:160. Do you want to maintain the original resolution/storage aspect ratio (SAR) or scale to square pixels (e.g. 640x480)? The current DAR is correctly 4:3 through PAR manipulation.

  2. Quality Priority: The source uses CRF 10 (very high quality). What's your target quality? We could use CRF 18-22 for x265/AV1 as a starting point (lower = better quality).

  3. Hardware vs Software: The L4 GPU supports both HEVC and AV1 encoding, but hardware encoding typically produces larger files than software encoding at same quality. Do you prefer:

    • Hardware encoding (faster but larger files)
    • Software encoding (slower but better compression)
    • Or a balance?
  4. Animation Tuning: Should we use animation-specific tuning? (x265 has tune animation, AV1 has specific film-grain handling)

  5. HDR/Color: The source appears to be 8-bit SDR. Any need for 10-bit encoding to prevent banding?

Once you answer these, I'll provide optimized commands. For now, here's a template for both codecs:

x265 Software Encoding (Conservative Settings):

ffmpeg -i input.mkv -map 0:v:0 -c:v libx265 \
-vf "setsar=1:1,scale=out_color_matrix=bt601" \
-x265-params \
"crf=18:deblock=-2,-2:psy-rd=2:psy-rdoq=2:aq-mode=3:aq-strength=1:no-sao=1:bframes=8:rc-lookahead=60:ref=6:min-keyint=23:keyint=240" \
-preset slower \
-tag:v hvc1 \
-color_range tv -colorspace bt470bg -color_primaries bt470bg -color_trc gamma28 \
-c:a copy \
output_x265.mkv

AV1 Hardware Encoding (NVENC):

ffmpeg -i input.mkv -map 0:v:0 -c:v av1_nvenc \
-vf "setsar=1:1,scale=out_color_matrix=bt601" \
-cq 24 \
-qmin 0 -qmax 63 \
-spatial_aq 1 -temporal_aq 1 \
-rc-lookahead 32 \
-g 240 \
-color_range tv -colorspace bt470bg -color_primaries bt470bg -color_trc gamma28 \
-c:a copy \
output_av1.mkv

Important Notes:

  1. The scale=out_color_matrix=bt601 ensures proper color matrix handling for SD content
  2. setsar=1:1 makes square pixels (common for modern displays) while maintaining 4:3 DAR
  3. The color metadata flags preserve the original TV color characteristics
  4. Both commands remove audio tracks via -map 0:v:0

Would you like me to:

  • Adjust the scaling approach?
  • Modify quality targets?
  • Optimize for specific hardware capabilities?
  • Add animation-specific tuning parameters?

Got it! Based on your clarifications, here are optimized commands for both codecs with explanations:


x265 (Software) - Recommended for Best Quality

ffmpeg -i input.mkv -map 0:v:0 \
-vf "scale=out_color_matrix=bt601,setparams=sar=177/160" \
-c:v libx265 \
-pix_fmt yuv420p10le \
-x265-params \
"crf=20:deblock=-3,-3:psy-rd=2:psy-rdoq=3:aq-mode=3:aq-strength=0.8:no-sao=1:bframes=8:rc-lookahead=60:ref=6:min-keyint=23:keyint=240:high-tier=1:limit-tu=4:tu-inter-depth=3:tu-intra-depth=3:me=3:subme=5:merange=48:no-strong-intra-smoothing=1:repeat-headers=1:colorprim=bt470bg:transfer=gamma28:colormatrix=bt470bg:master-display=G(8500,39850)B(6550,2300)R(35400,14600)WP(31270,32900)L(10000000,1):hdr10=1:repeat-headers=1" \
-preset slower \
-tune animation \
-tag:v hvc1 \
-color_range tv -colorspace bt470bg -color_primaries bt470bg -color_trc gamma28 \
-output.mp4

AV1 (Software via SVT-AV1) - Better Compression but Slower

ffmpeg -i input.mkv -map 0:v:0 \
-vf "scale=out_color_matrix=bt601,setparams=sar=177/160" \
-c:v libsvtav1 \
-pix_fmt yuv420p10le \
-svtav1-params \
"crf=22:film-grain=4:film-grain-denoise=0:enable-overlays=1:scd=1:keyint=240:min-keyint=23:tune=0:enable-tf=1:enable-qm=1:qm-min=0:aq-mode=3:enable-cdef=1:enable-restoration=1:input-depth=10:color-range=tv:color-primaries=bt470bg:transfer-characteristics=gamma28:matrix-coefficients=bt470bg:mastering-display=G(8500,39850)B(6550,2300)R(35400,14600)WP(31270,32900)L(10000000,1)" \
-preset 4 \
-g 240 \
-color_range tv -colorspace bt470bg -color_primaries bt470bg -color_trc gamma28 \
-output_av1.mkv

AV1 (Hardware via NVENC) - Faster but Less Efficient

ffmpeg -i input.mkv -map 0:v:0 \
-vf "scale=out_color_matrix=bt601,setparams=sar=177/160" \
-c:v av1_nvenc \
-pix_fmt yuv420p10le \
-rc vbr \
-cq 26 \
-qmin 0 -qmax 63 \
-spatial_aq 1 -temporal_aq 1 \
-rc-lookahead 32 \
-g 240 \
-tune hq \
-color_range tv -colorspace bt470bg -color_primaries bt470bg -color_trc gamma28 \
-output_av1_nvenc.mkv

Key Choices Explained

  1. No Scaling:
    setparams=sar=177/160 preserves original pixel aspect ratio (PAR 177:160 → DAR 4:3).
    scale=out_color_matrix=bt601 forces correct SD color matrix (avoids auto-guessing errors).

  2. CRF Targets:

    • x265: CRF 20 (excellent quality for animation)
    • SVT-AV1: CRF 22 (AV1 is more efficient)
    • AV1 NVENC: CQ 26 (hardware needs higher values for similar quality)
  3. Animation Tuning:

    • x265: tune animation, aggressive deblocking (deblock=-3,-3), and Psy-RDOQ
    • AV1: Film grain synthesis (film-grain=4), CDEF/restoration filters
    • Both disable SAO (Sample Adaptive Offset) to avoid over-smoothing line art
  4. 10-bit & Color:

    • yuv420p10le for 10-bit depth
    • Full HDR10 metadata (from your source’s mastering display data)
    • Explicit bt470bg (SD color primaries) and gamma28 (CRT transfer)
  5. Hardware vs Software:

    • SVT-AV1 will give best compression but is ~10x slower than NVENC
    • x265 is a good middle ground (quality/speed)
    • AV1 NVENC is fastest but requires higher bitrates

Recommendation

  1. Test with 1 episode: Compare file sizes and quality with:

    ffplay -i output.mp4 -vf "split=2[pre][post];[pre]scale=iw*4:ih*4:flags=neighbor[pre];[post]scale=iw*4:ih*4:flags=bicubic[post];[pre][post]hstack"

    (Zooms 400% to compare pixel-level details)

  2. Prefer SVT-AV1 if you can wait for encoding (better future-proofing).

  3. Use x265 if you need reasonable speed with great quality.

Let me know if you want to adjust the grain synthesis strength (AV1) or psy-RDOQ (x265)!

### EXAMPLE CONFIG SECTIONS
# do not allow a specific ip to access wan
#config rule
# option src lan
# option src_ip 192.168.45.2
# option dest wan
# option proto tcp
# option target REJECT
# block a specific mac on wan
#config rule
# option dest wan
# option src_mac 00:11:22:33:44:66
# option target REJECT
# block incoming ICMP traffic on a zone
#config rule
# option src lan
# option proto ICMP
# option target DROP
# port redirect port coming in on wan to lan
#config redirect
# option src wan
# option src_dport 80
# option dest lan
# option dest_ip 192.168.16.235
# option dest_port 80
# option proto tcp
# port redirect of remapped ssh port (22001) on wan
#config redirect
# option src wan
# option src_dport 22001
# option dest lan
# option dest_port 22
# option proto tcp
### FULL CONFIG SECTIONS
#config rule
# option src lan
# option src_ip 192.168.45.2
# option src_mac 00:11:22:33:44:55
# option src_port 80
# option dest wan
# option dest_ip 194.25.2.129
# option dest_port 120
# option proto tcp
# option target REJECT
#config redirect
# option src lan
# option src_ip 192.168.45.2
# option src_mac 00:11:22:33:44:55
# option src_port 1024
# option src_dport 80
# option dest_ip 194.25.2.129
# option dest_port 120
# option proto tcp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment