Tvix is a new implementation of the Nix language and package manager.
In this benchmark we test it's performance when instantiating the hello package from nix.
Note that at the time tvix does not have its own store implementation yet and it has to
execute nix-store whenever it needs to copy files to the store.
Last active
December 15, 2023 19:35
-
-
Save Mic92/2edb7d1afa861dffd2f601f1de78cb87 to your computer and use it in GitHub Desktop.
Tvix (79246855d1a0fd9b81be113b16a56379c7641aa1) vs nix (2.19.2) evaluation of the hello package
| Command | Mean [s] | Min [s] | Max [s] | Relative |
|---|---|---|---|---|
./target/release/tvix -E 'with import <nixpkgs>{}; toString hello' |
2.055 ± 0.006 | 2.048 | 2.066 | 5.73 ± 0.08 |
/nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString hello' |
0.358 ± 0.005 | 0.353 | 0.369 | 1.00 |
| Command | Mean [s] | Min [s] | Max [s] | Relative |
|---|---|---|---|---|
./target/release/tvix -E 'with import <nixpkgs>{}; toString firefox' |
22.415 ± 0.233 | 22.191 | 23.041 | 11.42 ± 0.14 |
/nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString firefox' |
1.962 ± 0.012 | 1.943 | 1.982 | 1.00 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| System: Host: ryan Kernel: 6.3.0-rc2 x86_64 bits: 64 compiler: gcc v: 12.2.0 | |
| parameters: initrd=\efi\nixos\g9rhaqalsxwa9nh78qr1nv0ai2h6f3pd-initrd-linux-6.3-initrd.efi | |
| init=/nix/store/2cmg092i5ys0x7j5zxy69pd2kdprc01i-nixos-system-ryan-23.05.20230801.d7b628a/init console=tty0 | |
| console=ttyS0,115200 kvm_amd.sev=1 kvm_amd.sev_es=1 kvm_amd.sev_snp=1 sp5100_tco.blacklist=yes | |
| default_hugepagesz=2MB hugepagesz=2MB hugepages=1000 nohibernate loglevel=4 | |
| Console: tty pts/0 DM: LightDM 1.32.0 Distro: NixOS 23.05 (Stoat) | |
| Machine: Type: Server System: Dell product: PowerEdge R7515 v: N/A serial: 9P0PSH3 Chassis: type: 23 serial: 9P0PSH3 | |
| Mobo: Dell model: 07PXPY v: A04 serial: .9P0PSH3.CNCMS0016O01PE. UEFI: Dell v: 2.9.3 date: 08/05/2022 | |
| CPU: Info: 64-Core model: AMD EPYC 7713P bits: 64 type: MCP arch: Zen 3 family: 19 (25) model-id: 1 stepping: 1 | |
| microcode: A001173 cache: L2: 32 MiB | |
| flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm bogomips: 255541 | |
| Speed: 2000 MHz min/max: 1500/2000 MHz boost: enabled Core speeds (MHz): 1: 2000 2: 2000 3: 3693 4: 2000 5: 2000 | |
| 6: 2000 7: 1632 8: 2000 9: 2000 10: 2000 11: 2000 12: 2000 13: 2000 14: 2000 15: 2000 16: 2000 17: 2000 18: 2000 | |
| 19: 2000 20: 2000 21: 2000 22: 2000 23: 2000 24: 2000 25: 2000 26: 2000 27: 2000 28: 2000 29: 2000 30: 2000 | |
| 31: 2000 32: 2000 33: 2000 34: 2000 35: 2000 36: 2000 37: 2000 38: 2000 39: 2000 40: 2000 41: 2000 42: 2000 | |
| 43: 2000 44: 2000 45: 2000 46: 2000 47: 2000 48: 2000 49: 2000 50: 2000 51: 2000 52: 2000 53: 2000 54: 2000 | |
| 55: 2000 56: 2000 57: 2000 58: 2000 59: 2000 60: 2000 61: 2000 62: 2000 63: 2000 64: 2000 | |
| Vulnerabilities: Type: itlb_multihit status: Not affected | |
| Type: l1tf status: Not affected | |
| Type: mds status: Not affected | |
| Type: meltdown status: Not affected | |
| Type: mmio_stale_data status: Not affected | |
| Type: retbleed status: Not affected | |
| Type: spec_store_bypass mitigation: Speculative Store Bypass disabled via prctl | |
| Type: spectre_v1 mitigation: usercopy/swapgs barriers and __user pointer sanitization | |
| Type: spectre_v2 | |
| mitigation: Retpolines, IBPB: conditional, IBRS_FW, STIBP: disabled, RSB filling, PBRSB-eIBRS: Not affected | |
| Type: srbds status: Not affected | |
| Type: tsx_async_abort status: Not affected | |
| Graphics: Device-1: Matrox Systems Integrated Matrox G200eW3 Graphics driver: mgag200 v: kernel bus-ID: c3:00.0 | |
| chip-ID: 102b:0536 class-ID: 0300 | |
| Display: server: X.org 1.21.1.8 driver: loaded: mgag200 note: n/a (using device driver) tty: 132x38 | |
| Message: Unable to show advanced data. Required tool glxinfo missing. | |
| Audio: Message: No device data found. | |
| Network: Device-1: Broadcom BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet vendor: Dell driver: bnxt_en v: kernel | |
| port: 1000 bus-ID: 02:00.0 chip-ID: 14e4:16d8 class-ID: 0200 | |
| IF: enp2s0f0np0 state: up speed: 10000 Mbps duplex: full mac: 2c:ea:7f:af:56:78 | |
| Device-2: Broadcom BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet vendor: Dell driver: bnxt_en v: kernel | |
| port: 1000 bus-ID: 02:00.1 chip-ID: 14e4:16d8 class-ID: 0200 | |
| IF: enp2s0f1np1 state: down mac: 2c:ea:7f:af:56:79 | |
| Device-3: Broadcom NetXtreme BCM5720 Gigabit Ethernet PCIe vendor: Dell PowerEdge R6515/R7515 LOM driver: tg3 | |
| v: kernel port: 1000 bus-ID: c1:00.0 chip-ID: 14e4:165f class-ID: 0200 | |
| IF: eno8303 state: down mac: d0:8e:79:ba:1a:28 | |
| Device-4: Broadcom NetXtreme BCM5720 Gigabit Ethernet PCIe vendor: Dell PowerEdge R6515/R7515 LOM driver: tg3 | |
| v: kernel port: 1000 bus-ID: c1:00.1 chip-ID: 14e4:165f class-ID: 0200 | |
| IF: eno8403 state: down mac: d0:8e:79:ba:1a:29 | |
| IF-ID-1: docker0 state: down mac: 02:42:76:7c:0b:c7 | |
| IF-ID-2: tinc.retiolum state: unknown speed: 10000 Mbps duplex: full mac: N/A | |
| RAID: Hardware-1: Broadcom / LSI MegaRAID SAS-3 3008 [Fury] driver: megaraid_sas v: 07.725.01.00-rc1 port: 1000 | |
| bus-ID: 01:00.0 chip-ID: 1000.005f rev: 02 class-ID: 0104 | |
| Device-1: zroot type: zfs status: ONLINE level: linear size: 556 GiB free: 356 GiB allocated: 200 GiB | |
| Components: Online: N/A | |
| Drives: Local Storage: total: 2.39 TiB used: 1.88 TiB (78.6%) | |
| ID-1: /dev/nvme0n1 maj-min: 259:2 vendor: Samsung model: SSD 970 EVO Plus 1TB size: 931.51 GiB block-size: | |
| physical: 512 B logical: 512 B speed: 31.6 Gb/s lanes: 4 rotation: SSD serial: S4EWNX0R513894H rev: 2B2QEXM7 | |
| temp: 21 ° (294 K) C scheme: GPT | |
| SMART: yes health: PASSED on: 74d 5h cycles: 6 read-units: 105,618,032 [54.0 TB] | |
| written-units: 10,593,143 [5.42 TB] | |
| ID-2: /dev/nvme2n1 maj-min: 259:0 vendor: SK Hynix model: PC801 NVMe 1TB size: 953.87 GiB block-size: | |
| physical: 512 B logical: 512 B speed: 63.2 Gb/s lanes: 4 rotation: SSD serial: SNB5N744011305V1P rev: 51002141 | |
| temp: 25 ° (298 K) C scheme: MBR | |
| SMART: yes health: PASSED on: 45d 1h cycles: 107 read-units: 1,416,392 [725 GB] written-units: 4,728,664 [2.42 TB] | |
| ID-3: /dev/sda maj-min: 8:0 model: AL15SEB060NY size: 558.91 GiB block-size: physical: 512 B logical: 512 B | |
| speed: <unknown> rotation: 10000 rpm serial: N/A rev: EF06 scheme: GPT | |
| SMART: yes state: enabled | |
| Partition: ID-1: / raw-size: N/A size: 453.47 GiB used: 114.42 GiB (25.2%) fs: zfs logical: zroot/root/nixos | |
| ID-2: /boot raw-size: 1024 MiB size: 1021.9 MiB (99.80%) used: 77.9 MiB (7.6%) fs: vfat block-size: 512 B | |
| dev: /dev/sda1 maj-min: 8:1 | |
| ID-3: /home raw-size: N/A size: 3.31 TiB used: 678.62 GiB (20.0%) fs: nfs4 remote: nfs:/export/home | |
| ID-4: /tmp raw-size: N/A size: 339.05 GiB used: 128 KiB (0.0%) fs: zfs logical: zroot/root/tmp | |
| Swap: Alert: No swap data was found. | |
| Sensors: Message: No ipmi sensor data found. | |
| Missing: Required tool sensors not installed. Check --recommends | |
| Info: Processes: 862 Uptime: 20:38:20 up 2 days 7:19, 2 users, load average: 0.39, 1.52, 0.93 wakeups: 0 | |
| Memory: 501.21 GiB used: 53.49 GiB (10.7%) Init: systemd v: 253 target: graphical.target tool: systemctl Compilers: | |
| gcc: 12.2.0 Packages: nix-default: 0 nix-sys: 899 lib: 219 nix-usr: 0 Shell: Zsh (sudo) v: 5.9 default: Bash | |
| v: 5.2.15 running-in: tty pts/0 inxi: 3.3.04 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| hyperfine --export-markdown /tmp/report.md "./target/release/tvix -E 'with import <nixpkgs>{}; toString firefox'" "/nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString firefox'" | |
| Benchmark 1: ./target/release/tvix -E 'with import <nixpkgs>{}; toString firefox' | |
| Time (mean ± σ): 22.415 s ± 0.233 s [User: 20.626 s, System: 1.982 s] | |
| Range (min … max): 22.191 s … 23.041 s 10 runs | |
| Benchmark 2: /nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString firefox' | |
| Time (mean ± σ): 1.962 s ± 0.012 s [User: 1.518 s, System: 0.432 s] | |
| Range (min … max): 1.943 s … 1.982 s 10 runs | |
| Summary | |
| /nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString firefox' ran | |
| 11.42 ± 0.14 times faster than ./target/release/tvix -E 'with import <nixpkgs>{}; toString firefox' | |
| hyperfine --export-markdown /tmp/report.md 221.63s user 24.58s system 100% cpu 4:04.31 total |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| hyperfine --export-markdown /tmp/report.md "./target/release/tvix -E 'with import <nixpkgs>{}; toString hello'" "/nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString hello'" | |
| Benchmark 1: ./target/release/tvix -E 'with import <nixpkgs>{}; toString hello' | |
| Time (mean ± σ): 2.055 s ± 0.006 s [User: 1.785 s, System: 0.316 s] | |
| Range (min … max): 2.048 s … 2.066 s 10 runs | |
| Benchmark 2: /nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString hello' | |
| Time (mean ± σ): 358.4 ms ± 5.2 ms [User: 252.8 ms, System: 93.5 ms] | |
| Range (min … max): 352.5 ms … 369.0 ms 10 runs | |
| Summary | |
| /nix/store/723r8wph94c630z0na6c7sjpg4z1qlw4-nix-2.19.2/bin/nix-instantiate --eval --json --expr 'with import <nixpkgs>{}; toString hello' ran | |
| 5.73 ± 0.08 times faster than ./target/release/tvix -E 'with import <nixpkgs>{}; toString hello' | |
| hyperfine --export-markdown /tmp/report.md 20.54s user 4.49s system 101% cpu 24.670 total |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment