Maybe to generate reports?
How are tests evaluated and the matrix/table populated? (It seems like that part is manual.)
Why is VirtualBox provider used? (Tom asked that in #developers then crossed it out - follow up)
What parameter(s) does
./result (i.e., the resulting test script) need?
answer: any arbitrary string that will be the name of a directory in the current dir, where the tests will be run; it can be an existing one. From the script below (which is generated from the template in
destdir=$(realpath "$1") mkdir -p "$destdir"
realpathreturns the absolute path of the given dir (or string, if dir does not exist)
When running any of the sample tests, I get
$ ./result + PATH=/nix/store/cyywvk6hlc9b0g449z8vcp155v2v2sys-coreutils-8.31/bin/:/run/wrappers/bin:/home/toraritte/.nix-profile/bin:/etc/profiles/per-user/toraritte/bin:/nix/var/nix/profiles/default/bin:/ run/current-system/sw/bin ./result: line 7: $1: unbound variable + destdir=
Makes sense as the contents of the shell script
./result points to has to have an argument (see below), and the first example calls it as
./result ./output-directory - yet the examples below it do not produce an
nix-build ./test-script.nix also does not produce an output dir, so what is exactly that
$ cat $(readlink -f result) #!/bin/sh set -eux PATH="/nix/store/cyywvk6hlc9b0g449z8vcp155v2v2sys-coreutils-8.31/bin/:$PATH" destdir=$(realpath "$1") mkdir -p "$destdir" shift set +e echo "Pre-fetching images" cat <<EOF | /nix/store/qsqiiy23dy7yjrnzazhivwp07z3784nj-findutils-4.7.0/bin/xargs -L 1 -P 4 bash /nix/store/1gk6y29w4kpianxzwrlgaf85na1nf565-shellchecked-fetch-image EOF echo "Running tests" cat <<EOF | /nix/store/qsqiiy23dy7yjrnzazhivwp07z3784nj-findutils-4.7.0/bin/xargs -L 1 -P 4 bash /nix/store/hxpjxa4lj9h7hckcqsaxlr6m7a5riy8v-shellchecked-run-install-default-alpine-3-8.sh "$destdir/install-default-alpine-3-
It is a shell script template, and the generated script in the Nix store will
- provision VMs (depending on the input arguments) to run the test on
Why is VirtualBox used as Vagrant provider instead of QEMU + KVM?
Maybe because the setup is easier this way? Also, "by default, Vagrant uses Oracle VirtualBox as provider."
Why are the out-of-the-box tests failing?
cp: cannot stat './nix.x86_64-linux.tar.xz': No such file or directory
solution: the script assumed that there is a specific Nix source tarball lying around in the project root which wasn't the case so just downloaded one
``` $ nix-build -v ./test-script.nix --argstr imageNameFilter arch --argstr installMethodFilter install-default ```
- VirtualBox VM ends up in
gurumeditationstate, and then it gets force-shutdown
The error is copied below, and it seems to be hitting the issue that KVM and VirtualBox don't play well with each other (and I have both enabled in configuration.nix on my NixOS)
solution: disable libvirt and KVM in
The SSH command responded with a non-zero exit status.followed by force shutdown
==> default: Waiting for machine to boot. This may take a few minutes... Bringing machine 'default' up with 'virtualbox' provider... \==> default: Importing base box 'generic/arch'... \==> default: Matching MAC address for NAT networking... \==> default: Setting the name of the VM: tmpXZBw5QPNkG\_default\_1629584121614\_84023 \==> default: Clearing any previously set network interfaces... \==> default: Preparing network interfaces based on configuration... default: Adapter 1: nat \==> default: Forwarding ports... default: 22 (guest) => 2222 (host) (adapter 1) \==> default: Running 'pre-boot' VM customizations... \==> default: Booting VM... \==> default: Waiting for machine to boot. This may take a few minutes... default: SSH address: 127.0.0.1:2222 default: SSH username: vagrant default: SSH auth method: private key default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's present... default: Key inserted! Disconnecting and reconnecting using new SSH key... \==> default: Machine booted and ready! \==> default: Checking for guest additions in VM... default: The guest additions on this VM do not match the installed version of default: VirtualBox! In most cases this is fine, but in rare cases it can default: prevent things such as shared folders from working properly. If you see default: shared folder errors, please make sure the guest additions within the default: virtual machine match the version of VirtualBox you have installed on default: your host and reload your VM. default: default: Guest Additions Version: 6.0.0 r127566 default: VirtualBox Version: 6.1 \==> default: Running provisioner: shell... default: Running: inline script default: /tmp/vagrant-shell: line 2: packman: command not found The SSH command responded with a non-zero exit status. Vagrant assumes that this means the command failed. The output for this command should be in the log above. Please read the output to determine what went wrong. \==> default: Forcing shutdown of VM... \==> default: Destroying VM and associated drives...