Skip to content

Instantly share code, notes, and snippets.

@Mic92
Last active December 15, 2023 19:35
Show Gist options
  • Save Mic92/2edb7d1afa861dffd2f601f1de78cb87 to your computer and use it in GitHub Desktop.
Save Mic92/2edb7d1afa861dffd2f601f1de78cb87 to your computer and use it in GitHub Desktop.
Tvix (79246855d1a0fd9b81be113b16a56379c7641aa1) vs nix (2.19.2) evaluation of the hello package

Tvix is a new implementation of the Nix language and package manager. In this benchmark we test it's performance when instantiating the hello package from nix. Note that at the time tvix does not have its own store implementation yet and it has to execute nix-store whenever it needs to copy files to the store.

Hello

Command Mean [s] Min [s] Max [s] Relative
./target/release/tvix -E 'with import <nixpkgs>{}; toString hello' 2.055 ± 0.006 2.048 2.066 5.73 ± 0.08
/nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString hello' 0.358 ± 0.005 0.353 0.369 1.00

Firefox

Command Mean [s] Min [s] Max [s] Relative
./target/release/tvix -E 'with import <nixpkgs>{}; toString firefox' 22.415 ± 0.233 22.191 23.041 11.42 ± 0.14
/nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString firefox' 1.962 ± 0.012 1.943 1.982 1.00
System: Host: ryan Kernel: 6.3.0-rc2 x86_64 bits: 64 compiler: gcc v: 12.2.0
parameters: initrd=\efi\nixos\g9rhaqalsxwa9nh78qr1nv0ai2h6f3pd-initrd-linux-6.3-initrd.efi
init=/nix/store/2cmg092i5ys0x7j5zxy69pd2kdprc01i-nixos-system-ryan-23.05.20230801.d7b628a/init console=tty0
console=ttyS0,115200 kvm_amd.sev=1 kvm_amd.sev_es=1 kvm_amd.sev_snp=1 sp5100_tco.blacklist=yes
default_hugepagesz=2MB hugepagesz=2MB hugepages=1000 nohibernate loglevel=4
Console: tty pts/0 DM: LightDM 1.32.0 Distro: NixOS 23.05 (Stoat)
Machine: Type: Server System: Dell product: PowerEdge R7515 v: N/A serial: 9P0PSH3 Chassis: type: 23 serial: 9P0PSH3
Mobo: Dell model: 07PXPY v: A04 serial: .9P0PSH3.CNCMS0016O01PE. UEFI: Dell v: 2.9.3 date: 08/05/2022
CPU: Info: 64-Core model: AMD EPYC 7713P bits: 64 type: MCP arch: Zen 3 family: 19 (25) model-id: 1 stepping: 1
microcode: A001173 cache: L2: 32 MiB
flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm bogomips: 255541
Speed: 2000 MHz min/max: 1500/2000 MHz boost: enabled Core speeds (MHz): 1: 2000 2: 2000 3: 3693 4: 2000 5: 2000
6: 2000 7: 1632 8: 2000 9: 2000 10: 2000 11: 2000 12: 2000 13: 2000 14: 2000 15: 2000 16: 2000 17: 2000 18: 2000
19: 2000 20: 2000 21: 2000 22: 2000 23: 2000 24: 2000 25: 2000 26: 2000 27: 2000 28: 2000 29: 2000 30: 2000
31: 2000 32: 2000 33: 2000 34: 2000 35: 2000 36: 2000 37: 2000 38: 2000 39: 2000 40: 2000 41: 2000 42: 2000
43: 2000 44: 2000 45: 2000 46: 2000 47: 2000 48: 2000 49: 2000 50: 2000 51: 2000 52: 2000 53: 2000 54: 2000
55: 2000 56: 2000 57: 2000 58: 2000 59: 2000 60: 2000 61: 2000 62: 2000 63: 2000 64: 2000
Vulnerabilities: Type: itlb_multihit status: Not affected
Type: l1tf status: Not affected
Type: mds status: Not affected
Type: meltdown status: Not affected
Type: mmio_stale_data status: Not affected
Type: retbleed status: Not affected
Type: spec_store_bypass mitigation: Speculative Store Bypass disabled via prctl
Type: spectre_v1 mitigation: usercopy/swapgs barriers and __user pointer sanitization
Type: spectre_v2
mitigation: Retpolines, IBPB: conditional, IBRS_FW, STIBP: disabled, RSB filling, PBRSB-eIBRS: Not affected
Type: srbds status: Not affected
Type: tsx_async_abort status: Not affected
Graphics: Device-1: Matrox Systems Integrated Matrox G200eW3 Graphics driver: mgag200 v: kernel bus-ID: c3:00.0
chip-ID: 102b:0536 class-ID: 0300
Display: server: X.org 1.21.1.8 driver: loaded: mgag200 note: n/a (using device driver) tty: 132x38
Message: Unable to show advanced data. Required tool glxinfo missing.
Audio: Message: No device data found.
Network: Device-1: Broadcom BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet vendor: Dell driver: bnxt_en v: kernel
port: 1000 bus-ID: 02:00.0 chip-ID: 14e4:16d8 class-ID: 0200
IF: enp2s0f0np0 state: up speed: 10000 Mbps duplex: full mac: 2c:ea:7f:af:56:78
Device-2: Broadcom BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet vendor: Dell driver: bnxt_en v: kernel
port: 1000 bus-ID: 02:00.1 chip-ID: 14e4:16d8 class-ID: 0200
IF: enp2s0f1np1 state: down mac: 2c:ea:7f:af:56:79
Device-3: Broadcom NetXtreme BCM5720 Gigabit Ethernet PCIe vendor: Dell PowerEdge R6515/R7515 LOM driver: tg3
v: kernel port: 1000 bus-ID: c1:00.0 chip-ID: 14e4:165f class-ID: 0200
IF: eno8303 state: down mac: d0:8e:79:ba:1a:28
Device-4: Broadcom NetXtreme BCM5720 Gigabit Ethernet PCIe vendor: Dell PowerEdge R6515/R7515 LOM driver: tg3
v: kernel port: 1000 bus-ID: c1:00.1 chip-ID: 14e4:165f class-ID: 0200
IF: eno8403 state: down mac: d0:8e:79:ba:1a:29
IF-ID-1: docker0 state: down mac: 02:42:76:7c:0b:c7
IF-ID-2: tinc.retiolum state: unknown speed: 10000 Mbps duplex: full mac: N/A
RAID: Hardware-1: Broadcom / LSI MegaRAID SAS-3 3008 [Fury] driver: megaraid_sas v: 07.725.01.00-rc1 port: 1000
bus-ID: 01:00.0 chip-ID: 1000.005f rev: 02 class-ID: 0104
Device-1: zroot type: zfs status: ONLINE level: linear size: 556 GiB free: 356 GiB allocated: 200 GiB
Components: Online: N/A
Drives: Local Storage: total: 2.39 TiB used: 1.88 TiB (78.6%)
ID-1: /dev/nvme0n1 maj-min: 259:2 vendor: Samsung model: SSD 970 EVO Plus 1TB size: 931.51 GiB block-size:
physical: 512 B logical: 512 B speed: 31.6 Gb/s lanes: 4 rotation: SSD serial: S4EWNX0R513894H rev: 2B2QEXM7
temp: 21 ° (294 K) C scheme: GPT
SMART: yes health: PASSED on: 74d 5h cycles: 6 read-units: 105,618,032 [54.0 TB]
written-units: 10,593,143 [5.42 TB]
ID-2: /dev/nvme2n1 maj-min: 259:0 vendor: SK Hynix model: PC801 NVMe 1TB size: 953.87 GiB block-size:
physical: 512 B logical: 512 B speed: 63.2 Gb/s lanes: 4 rotation: SSD serial: SNB5N744011305V1P rev: 51002141
temp: 25 ° (298 K) C scheme: MBR
SMART: yes health: PASSED on: 45d 1h cycles: 107 read-units: 1,416,392 [725 GB] written-units: 4,728,664 [2.42 TB]
ID-3: /dev/sda maj-min: 8:0 model: AL15SEB060NY size: 558.91 GiB block-size: physical: 512 B logical: 512 B
speed: <unknown> rotation: 10000 rpm serial: N/A rev: EF06 scheme: GPT
SMART: yes state: enabled
Partition: ID-1: / raw-size: N/A size: 453.47 GiB used: 114.42 GiB (25.2%) fs: zfs logical: zroot/root/nixos
ID-2: /boot raw-size: 1024 MiB size: 1021.9 MiB (99.80%) used: 77.9 MiB (7.6%) fs: vfat block-size: 512 B
dev: /dev/sda1 maj-min: 8:1
ID-3: /home raw-size: N/A size: 3.31 TiB used: 678.62 GiB (20.0%) fs: nfs4 remote: nfs:/export/home
ID-4: /tmp raw-size: N/A size: 339.05 GiB used: 128 KiB (0.0%) fs: zfs logical: zroot/root/tmp
Swap: Alert: No swap data was found.
Sensors: Message: No ipmi sensor data found.
Missing: Required tool sensors not installed. Check --recommends
Info: Processes: 862 Uptime: 20:38:20 up 2 days 7:19, 2 users, load average: 0.39, 1.52, 0.93 wakeups: 0
Memory: 501.21 GiB used: 53.49 GiB (10.7%) Init: systemd v: 253 target: graphical.target tool: systemctl Compilers:
gcc: 12.2.0 Packages: nix-default: 0 nix-sys: 899 lib: 219 nix-usr: 0 Shell: Zsh (sudo) v: 5.9 default: Bash
v: 5.2.15 running-in: tty pts/0 inxi: 3.3.04
hyperfine --export-markdown /tmp/report.md "./target/release/tvix -E 'with import <nixpkgs>{}; toString firefox'" "/nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString firefox'"
Benchmark 1: ./target/release/tvix -E 'with import <nixpkgs>{}; toString firefox'
Time (mean ± σ): 22.415 s ± 0.233 s [User: 20.626 s, System: 1.982 s]
Range (min … max): 22.191 s … 23.041 s 10 runs
Benchmark 2: /nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString firefox'
Time (mean ± σ): 1.962 s ± 0.012 s [User: 1.518 s, System: 0.432 s]
Range (min … max): 1.943 s … 1.982 s 10 runs
Summary
/nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString firefox' ran
11.42 ± 0.14 times faster than ./target/release/tvix -E 'with import <nixpkgs>{}; toString firefox'
hyperfine --export-markdown /tmp/report.md 221.63s user 24.58s system 100% cpu 4:04.31 total
hyperfine --export-markdown /tmp/report.md "./target/release/tvix -E 'with import <nixpkgs>{}; toString hello'" "/nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString hello'"
Benchmark 1: ./target/release/tvix -E 'with import <nixpkgs>{}; toString hello'
Time (mean ± σ): 2.055 s ± 0.006 s [User: 1.785 s, System: 0.316 s]
Range (min … max): 2.048 s … 2.066 s 10 runs
Benchmark 2: /nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString hello'
Time (mean ± σ): 358.4 ms ± 5.2 ms [User: 252.8 ms, System: 93.5 ms]
Range (min … max): 352.5 ms … 369.0 ms 10 runs
Summary
/nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString hello' ran
5.73 ± 0.08 times faster than ./target/release/tvix -E 'with import <nixpkgs>{}; toString hello'
hyperfine --export-markdown /tmp/report.md 20.54s user 4.49s system 101% cpu 24.670 total
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment