Skip to content

Instantly share code, notes, and snippets.

@thezoggy
Last active May 7, 2024 23:21
Show Gist options
  • Save thezoggy/3c243b712f0cc960fa4dd78ff1ab56e7 to your computer and use it in GitHub Desktop.
Save thezoggy/3c243b712f0cc960fa4dd78ff1ab56e7 to your computer and use it in GitHub Desktop.
testing multipar/par2cmdline/par2cmdline-turbo

Source is a 1800d old NZB of a typical ~4G movie that is broken.
Lets call it "The.Test.2018.iNTERNAL.1080p.WEB.x264-DAMAGED" ;)
This job needs repair before it can be extracted, using as typical user test case (although only focusing on par2's tasks).

Gutted sabnzbd 4.x newsunpacker so I could re-run over the same job (verification+repair) to record different times on various machines and have try to get some good real world benchmarks. With par we parsed out parts of the cli output so we know when to start/stop. I did add some logic so I can try out multipar/par2cmdline/par2cmdline-turbo and 32bit/64bit/arm variations but left everything else that sab calls the same. I do not give it any additional arguments, wanted to see what a pure drop in replacement would do.

For the laptops, all machines are fully charged + connected to power and up to date on updates.
A fresh copy of the job was created each time to try and reduce caching involved. Waited ~30s after job folder creation to allow cooldown of cpu/disk thermals and activity.

Due to limited space on the table with additional columns, par2cmdline-turbo was shorten to par2cmdline-t

Results

machine1

OS: Win10 Pro x64 (22H2) • CPU: Intel Core i7-4790K @ 4 GHz • RAM: 16G DDR3 • HDD: 1TB Samsung 860 EVO SSD

Arch App Verify Repair Hash Meth MD5 Meth INV Meth MUL Meth
x64 multipar 1.3.2.7 00:09 00:13 - - - -
x86 multipar 1.3.2.7 00:10 00:12 - - - -
x64 par2cmdline 0.8.1 00:12 00:21 - - - -
x86 par2cmdline 0.8.1 00:13 00:20 - - - -
x64 par2cmdline-t 0.9.0 00:06 00:08 - - - Xor-Jit (SSE2)
x86 par2cmdline-t 0.9.0 00:07 00:08 - - - Xor-Jit (SSE2)
x64 par2cmdline-t 1.0.1 00:05 00:06 - - - Shuffle (AVX2)
x86 par2cmdline-t 1.0.1 00:07 00:07 - - - Shuffle (AVX2)
x64 par2cmdline-t 1.1.1 00:06 00:06 BMI1+PCLMUL NoLEA Shuffle (AVX2) Shuffle (AVX2)
x86 par2cmdline-t 1.1.1 00:07 00:06 BMI1+PCLMUL PCLMUL Shuffle (AVX2) Shuffle (AVX2)

machine2 - Dell XPS 13 9350

OS: Win10 Pro x64 (22H2) • CPU: Intel Core i7-6560U @ 2.2 GHz • RAM: 8G LPDDR3 • HDD: 256G Samsung SM951 NVMe

Arch App Verify Repair Hash Meth MD5 Meth INV Meth MUL Meth
x64 multipar 1.3.2.7 00:22 00:16 - - - -
x86 multipar 1.3.2.7 00:18 00:16 - - - -
x64 par2cmdline 0.8.1 00:18 00:42 - - - -
x86 par2cmdline 0.8.1 00:19 00:38 - - - -
x64 par2cmdline-t 0.9.0 00:09 00:12 - - - Xor-Jit (SSE2)
x86 par2cmdline-t 0.9.0 00:11 00:14 - - - Xor-Jit (SSE2)
x64 par2cmdline-t 1.0.1 00:08 00:12 - - - Shuffle (AVX2)
x86 par2cmdline-t 1.0.1 00:11 00:14 - - - Shuffle (AVX2)
x64 par2cmdline-t 1.1.1 00:08 00:12 BMI1+PCLMUL NoLEA Shuffle (AVX2) Shuffle (AVX2)
x86 par2cmdline-t 1.1.1 00:11 00:15 BMI1+PCLMUL PCLMUL Shuffle (AVX2) Shuffle (AVX2)

machine3 - Samsung Galaxy Book Go

OS: Win11 Home ARM64 (23H2) • CPU: Snapdragon 7c Gen 2 @ 2.55 GHz • RAM: 4G DDR4 • HDD: 128G SSD

Arch App Verify Repair Hash Meth MD5 Meth INV Meth MUL Meth
x64 multipar 1.3.2.7 00:38 00:46 - - - -
x86 multipar 1.3.2.7 00:36 00:43 - - - -
x64 par2cmdline 0.8.1 00:33 01:26 - - - -
x86 par2cmdline 0.8.1 00:27 01:07 - - - -
x64 par2cmdline-t 0.9.0 00:15 02:49 - - - -
x86 par2cmdline-t 0.9.0 00:16 02:36 - - - -
arm64 par2cmdline-t 0.9.0 00:14 00:33 - - - -
x64 par2cmdline-t 1.0.1 00:13 00:32 - - - Shuffle (SSSE3)
x86 par2cmdline-t 1.0.1 00:15 00:34 - - - Shuffle (SSSE3)
arm64 par2cmdline-t 1.0.1 00:13 00:30 - - - CLMul (NEON)
x64 par2cmdline-t 1.1.1 00:14 00:33 Scalar+PCLMUL PCLMUL Shuffle (SSSE3) Shuffle (SSSE3)
x86 par2cmdline-t 1.1.1 00:15 00:34 Scalar+PCLMUL PCLMUL Shuffle (SSSE3) Shuffle (SSSE3)
arm64 par2cmdline-t 1.1.1 00:12 00:31 Scalar+ARMCRC ARMCRC CLMul (NEON) CLMul (NEON)

machine4 - MacBook Pro (15-inch, 2018)

OS: MacOS 13.6.6 • CPU: Intel Core i9 8950HK 6-core @ 2.9 GHz • RAM: 32G DDR4 • HDD: 512G SSD

Arch App Verify Repair Hash Meth MD5 Meth INV Meth MUL Meth
x64 par2-sl64 00:06 00:17 - - - -
x64 par2cmdline-t 0.9.0 00:04 00:08 - - - Xor-Jit (SSE2)
x64 par2cmdline-t 1.0.1 00:04 00:04 - - - Shuffle (AVX2)
x64 par2cmdline-t 1.1.1 00:04 00:04 BMI1+PCLMUL NoLEA Shuffle (AVX2) Shuffle (AVX2)

machine5 - MacBook Pro (14-inch, 2021)

OS: MacOS 12.7.2 • CPU: Apple M1 Max 10-core @ 3.2 GHz • RAM: 32G LPDDR5 • HDD: 1TB Apple (AP1024R) NVMe

Arch App Verify Repair Hash Meth MD5 Meth INV Meth MUL Meth
arm64 par2cmdline 0.8.1 00:15 00:23 - - - -
x64 par2-sl64 00:06 00:11 - - - -
arm64 par2cmdline-t 0.9.0 00:04 00:04 - - - CLMul (NEON)
x64 par2cmdline-t 0.9.0 00:05 01:18 - - - Shuffle (SSSE3)
arm64 par2cmdline-t 1.0.1 00:05 00:04 - - - CLMul (NEON)
x64 par2cmdline-t 1.0.1 00:07 00:05 - - - Shuffle (SSSE3)
arm64 par2cmdline-t 1.1.1 00:05 00:04 Scalar+ARMCRC ARMCRC CLMul (NEON) CLMul (NEON)
x64 par2cmdline-t 1.1.1 00:07 00:05 Scalar+PCLMUL PCLMUL Shuffle (SSSE3) Shuffle (SSSE3)

machine6

OS: Unraid 6.12.6 • CPU: AMD Ryzen 7 3700X @ 3.6 GHz • RAM: 64G DDR4 • HDD: 1TB Samsung 970 EVO Plus NVMe

Arch App Verify Repair Hash Meth MD5 Meth INV Meth MUL Meth
amd64 par2cmdline 0.8.1 00:15 00:13 - - - -
amd64 par2cmdline-t 0.9.0 00:04 00:03 - - - Xor-Jit (SSE2)
amd64 par2cmdline-t 1.0.1 00:05 00:04 - - - Xor-Jit (AVX2)
amd64 par2cmdline-t 1.1.1 00:05 00:04 BMI1+PCLMUL NoLEA Shuffle (AVX2) Xor-Jit (AVX2)

Notes

  • multipar by design is nicer on cpu (only uses ~85% of available cpu cores) but is heavier on memory.

  • par2cmdline-turbo by default can use upto half of system ram but I never saw it use too much.

  • For windows arm, there is now an arm based python however I could not use it due to various libs that wouldn't install due to missing support for build tools. So x64 python was used, which means wonky win64 arm detection - so had to hard code variable when to use it or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment