Skip to content

Instantly share code, notes, and snippets.

@bbpbuildbot
Created February 27, 2022 17:48
Show Gist options
  • Save bbpbuildbot/f384d61b727a3aeec8827ec252209b7a to your computer and use it in GitHub Desktop.
Save bbpbuildbot/f384d61b727a3aeec8827ec252209b7a to your computer and use it in GitHub Desktop.
Logfiles for GitLab pipeline https://bbpgitlab.epfl.ch/hpc/coreneuron/-/pipelines/40305 (:no_entry:) running on GitHub PR BlueBrain/CoreNeuron#772.
Running with gitlab-runner 14.3.2 (e0218c92)
 on BB5 map runner pnPo3yJy
section_start:1645983210:resolve_secrets Resolving secrets
section_end:1645983210:resolve_secrets section_start:1645983210:prepare_executor Preparing the "custom" executor
Using Custom executor with driver BB5 PROD runner v0.0.3...
BB5 PROD runner running on bbpv1.epfl.ch, version 14.3.2, user
TMPDIR is /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/tmp/custom-executor158865173, slurm job id
Runner ID 29, project root hpc, project name coreneuron
Pipeline ID 40305, build ref b78986445f9d60261c743bdbd9ff77689666ae2f, job ID 169149
Build dir /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P40305, optional exclusive flag , optional cpus per task flag --cpus-per-task=8
A slurm job will be created with name GL_J169149_PROD_P112_CP3_C3
Job parameters: memory=76G, cpus_per_task=8, duration=1:00:00, constraint=cpu ntasks=2 account=proj9998 user=bbpcihpcproj12 partition=prod
sbatch: INFO: Activating auto partition selection plugin, please report errors to HPC/CS
sbatch: INFO: Job specifies cpu constraint, setting --constraint=[skl|clx]
Submitted batch job 302161
job state: R
sbatch: sbatch -p prod -A proj9998 --ntasks=2 --cpus-per-task=8 --mem=76G --job-name=GL_J169149_PROD_P112_CP3_C3 -C cpu --no-requeue -D /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P40305 --time=1:00:00 --wrap="sleep infinity"
srun: srun --mpi=none --chdir=/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P40305 --ntasks=2 --jobid=302161 --cpus-per-task=8 --mem=76G
section_end:1645983243:prepare_executor section_start:1645983243:prepare_script Preparing environment
Running on r1i7n20 via bbpv1.epfl.ch...
section_end:1645983246:prepare_script section_start:1645983246:get_sources Getting source from Git repository
Skipping Git repository setup
Skipping Git checkout
Skipping Git submodules setup
section_end:1645983247:get_sources section_start:1645983247:restore_cache Restoring cache
Checking cache for build:coreneuron:mod2c:intel-8...
Runtime platform  arch=amd64 os=linux pid=186114 revision=58ba2b95 version=14.2.0
cache.zip is up to date 
Successfully extracted cache
section_end:1645983257:restore_cache section_start:1645983257:download_artifacts Downloading artifacts
Downloading artifacts for spack_setup (169142)...
Runtime platform  arch=amd64 os=linux pid=187216 revision=58ba2b95 version=14.2.0
Downloading artifacts from coordinator... ok  id=169142 responseStatus=200 OK token=zwpMrobA
section_end:1645983258:download_artifacts section_start:1645983258:step_script Executing "step_script" stage of the job script
WARNING: Starting with version 14.0 the 'build_script' stage will be replaced with 'step_script': https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26426
$ SPACK_BUILD="${PWD}/spack-build"
$ env -0 | sort -z | xargs -0 -L 1 echo > initial_environment.env
$ . ${SPACK_ROOT}/share/spack/setup-env.sh
$ export SPACK_USER_CONFIG_PATH=${PWD}/spack-config
$ mkdir ${SPACK_USER_CONFIG_PATH}
$ cat > ${SPACK_USER_CONFIG_PATH}/config.yaml << END_SCRIPT # collapsed multi-line command
$ spack config blame config
--- config:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169149/spack-config/config.yaml:2 build_stage:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169149/spack-config/config.yaml:3 - /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169149/spack-build
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169149/spack-config/config.yaml:4 source_cache: /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169149/spack-source-cache
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/config.yaml:2 ccache: True
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:2 install_tree:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:3 root: $user_cache_path/software
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:4 projections:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:5 all: install_{compiler.name}-{compiler.version}-{target}/{name}-{version}-{hash:6}
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:6 module_roots:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:7 tcl: $user_cache_path/modules
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:9 environments:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:10 root: $user_cache_path/environments
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:88 concretization: separately
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:12 build_jobs: 8
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:18 extensions:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:18 - $spack/bluebrain/spack-scripting
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:19 # This is the path to the root of the Spack install tree.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:19 # You can use $spack here to refer to the root of the spack instance.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:37 template_dirs:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:37 - $spack/share/spack/templates
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Temporary locations Spack can try to use for builds.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Recommended options are given below.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Builds can be faster in temporary directories on some (e.g., HPC) systems.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Specifying `$tempdir` will ensure use of the default temporary directory
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # (i.e., ``$TMP` or ``$TMPDIR``).
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Another option that prevents conflicts and potential permission issues is
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # to specify `$user_cache_path/stage`, which ensures each user builds in their
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # home directory.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # A more traditional path uses the value of `$spack/var/spack/stage`, which
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # builds directly inside Spack's instance without staging them in a
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # temporary space. Problems with specifying a path inside a Spack instance
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # are that it precludes its use as a system package and its ability to be
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # pip installable.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # In any case, if the username is not already in the path, Spack will append
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # the value of `$user` in an attempt to avoid potential conflicts between
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # users in shared temporary spaces.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # The build stage can be purged with `spack clean --stage` and
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # `spack clean -a`, so it is important that the specified directory uniquely
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # identifies Spack staging to avoid accidentally wiping out non-Spack work.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:73 test_stage: $user_cache_path/test
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:82 misc_cache: $user_cache_path/cache
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:94 connect_timeout: 10
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:99 verify_ssl: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:107 suppress_gpg_warnings: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:113 install_missing_compilers: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:118 checksum: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:123 deprecated: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:128 dirty: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:136 build_language: C
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:145 locks: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:150 url_fetch_method: urllib
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:176 concretizer: clingo
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:183 db_lock_timeout: 300
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:191 package_lock_timeout: null
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:197 shared_linking: rpath
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:202 allow_sgid: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:207 terminal_title: False
_builtin debug: False
$ export XDG_CONFIG_HOME=${PWD}/local_config
$ mkdir -p "${XDG_CONFIG_HOME}/git"
$ echo -e "[url \"https://gitlab-ci-token:${CI_JOB_TOKEN}@bbpgitlab.epfl.ch/\"]\n insteadOf = git@bbpgitlab.epfl.ch:" > "${XDG_CONFIG_HOME}/git/config"
$ cat "${XDG_CONFIG_HOME}/git/config"
[url "https://gitlab-ci-token:[MASKED]@bbpgitlab.epfl.ch/"]
insteadOf = git@bbpgitlab.epfl.ch:
$ SPACK_FULL_SPEC="${SPACK_PACKAGE}${SPACK_PACKAGE_COMPILER:+%}${SPACK_PACKAGE_COMPILER} ${SPACK_PACKAGE_SPEC} ${SPACK_PACKAGE_DEPENDENCIES} ${SPACK_PACKAGE_DEPENDENCY_ON_PREVIOUS_JOB}"
$ echo "Preparing to install ${SPACK_FULL_SPEC}"
Preparing to install coreneuron%intel +tests~legacy-unit build_type=Debug
$ JSON_SPEC=$(spack spec --json ${SPACK_FULL_SPEC})
$ SPACK_INSTALLED_HASH=$(module load unstable python; echo "${JSON_SPEC}" | python -c "import json, sys; print(json.loads(sys.stdin.read())[\"spec\"][\"nodes\"][0][\"hash\"])")
$ echo "Determined its hash will be ${SPACK_INSTALLED_HASH}"
Determined its hash will be vlyasz3tnx2qtt5jveu4ropwbkiqafni
$ SPACK_STAGE_DIR=${SPACK_BUILD}/spack-stage-${SPACK_PACKAGE}-develop-${SPACK_INSTALLED_HASH}
$ SPACK_BUILD_DIR=${SPACK_STAGE_DIR}/spack-build-${SPACK_INSTALLED_HASH:0:7}
$ SPACK_SOURCE_DIR=${SPACK_STAGE_DIR}/spack-src
$ if [ ${SPACK_USE_CCACHE+x} ]; then
$ module load unstable ccache
$ export CCACHE_BASEDIR=$(realpath -P ${CI_BUILDS_DIR})
$ echo CCACHE_BASEDIR=${CCACHE_BASEDIR}
CCACHE_BASEDIR=/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305
$ export CCACHE_MAXSIZE=512M
$ export CCACHE_DIR="${TMPDIR}/ccache"
$ mkdir -p ${CCACHE_DIR}
$ if [ -f ${CI_PROJECT_DIR}/ccache.tar ]; then
$ tar -C "${CCACHE_DIR}" -xf "${CI_PROJECT_DIR}/ccache.tar"
$ fi
$ ccache --zero-stats
Statistics zeroed
$ ccache --show-stats --verbose
Summary:
Cache directory: /nvme/bbpcihpcproj12/302161/ccache
Primary config: /nvme/bbpcihpcproj12/302161/ccache/ccache.conf
Secondary config: /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/ccache-4.4.2-6qbndl/etc/ccache.conf
Stats updated: Sun Feb 27 18:34:44 2022
Hits: 0 / 0
Direct: 0 / 0
Preprocessed: 0 / 0
Misses: 0
Direct: 0
Preprocessed: 0
Primary storage:
Hits: 0 / 0
Misses: 0
Cache size (GB): 0.23 / 0.51 (45.28 %)
Files: 4748
$ fi
$ module load unstable git ${SPACK_EXTRA_MODULES}
$ spack uninstall -y --dependents /${SPACK_INSTALLED_HASH} || true
==> Error: No installed spec matches the hash: 'vlyasz3tnx2qtt5jveu4ropwbkiqafni'
$ spack spec -Il ${SPACK_FULL_SPEC}
Input spec
--------------------------------
- coreneuron%intel~legacy-unit+tests build_type=Debug
Concretized
--------------------------------
- vlyasz3 coreneuron@1.0.0.20220218%intel@2021.4.0~caliper~codegenopt~gpu~ipo~ispc~knl~legacy-unit+mpi~nmodl~openmp~profile+report+shared~sympy~sympyopt+tests~unified build_type=Debug arch=linux-rhel7-skylake
- dwdch6b ^bison@3.8.2%intel@2021.4.0 arch=linux-rhel7-skylake
[^] dl7pfht ^boost@1.78.0%intel@2021.4.0+atomic+chrono~clanglibcpp~container~context~coroutine+date_time~debug+exception~fiber+filesystem+graph~icu+iostreams+locale+log+math~mpi+multithreaded~numpy+pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave cxxstd=98 visibility=hidden arch=linux-rhel7-skylake
- z3q5f3x ^cmake@3.21.4%intel@2021.4.0~doc+ncurses+openssl+ownlibs~qt build_type=Release arch=linux-rhel7-skylake
- s4ueg72 ^flex@2.6.3%intel@2021.4.0+lex~nls arch=linux-rhel7-skylake
[^] 3narjkw ^hpe-mpi@2.25.hmpt%intel@2021.4.0 arch=linux-rhel7-skylake
[^] ldqdrnx ^libsonata-report@1.0.0.20220218%intel@2021.4.0~ipo+mpi build_type=RelWithDebInfo arch=linux-rhel7-skylake
[^] ofveh3j ^hdf5@1.10.7%intel@2021.4.0+cxx~fortran+hl~ipo~java+mpi+shared~szip~threadsafe+tools api=default build_type=RelWithDebInfo arch=linux-rhel7-skylake
[^] xys6npz ^pkgconf@1.8.0%intel@2021.4.0 arch=linux-rhel7-skylake
[^] e7w5nez ^zlib@1.2.11%intel@2021.4.0+optimize+pic+shared arch=linux-rhel7-skylake
[^] wk3uenv ^spdlog@1.9.2%intel@2021.4.0~ipo+shared build_type=RelWithDebInfo arch=linux-rhel7-skylake
- 6ggc5yr ^ninja@1.10.2%intel@2021.4.0 arch=linux-rhel7-skylake
[^] f27a7nn ^python@3.9.7%intel@2021.4.0+bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93189bc278fbc37a50ed7f183bd8aaf249a8e1670a465f0db6bb4f8cf87,4c2457325f2b608b1b6a2c63087df8c26e07db3e3d493caf36a56f0ecf6fb768,f2fd060afc4b4618fe8104c4c5d771f36dc55b1db5a4623785a4ea707ec72fb4 arch=linux-rhel7-skylake
[^] gdhqypa ^reportinglib@2.5.6%intel@2021.4.0~ipo~profile+shared~tests build_type=RelWithDebInfo arch=linux-rhel7-skylake
$ spack ${SPACK_INSTALL_EXTRA_FLAGS} install -j${SLURM_CPUS_PER_TASK} --log-format=junit --log-file=${CI_PROJECT_DIR}/install.xml --keep-stage ${SPACK_FULL_SPEC} || install_failed=1
==> bison@3.8.2 : has external module in ['bison/3.8.2']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/bison-3.8.2-ej2cew (external bison-3.8.2-dwdch6bmdeclr2novthsywtrryotawwz)
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/boost-1.78.0-hiorc7 (external boost-1.78.0-dl7pfhtsvyaq3273gr2g3l5vr37eeydc)
==> cmake@3.21.4 : has external module in ['cmake/3.21.4']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/cmake-3.21.4-cdyb7k (external cmake-3.21.4-z3q5f3xwpuibd3qbgdscqmu3efarbu42)
==> flex@2.6.3 : has external module in ['flex/2.6.3']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/flex-2.6.3-3mtl7j (external flex-2.6.3-s4ueg72j7l6vkdyvfxj2tweo7v7s3otx)
==> hpe-mpi@2.25.hmpt : has external module in ['hpe-mpi/2.25.hmpt']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/hpe-mpi-2.25.hmpt-4ukyxt (external hpe-mpi-2.25.hmpt-3narjkwjh6i2jr3zl3g5wdjlqi52hkwh)
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_intel-2021.4.0-skylake/pkgconf-1.8.0-xys6np
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/zlib-1.2.11-mswjss (external zlib-1.2.11-e7w5nezpwf572epfhbosqfzboztysout)
==> ninja@1.10.2 : has external module in ['ninja/1.10.2']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/ninja-1.10.2-eh33rh (external ninja-1.10.2-6ggc5yre7qddwxdjmn7sfptpdoiy4dtp)
==> python@3.9.7 : has external module in ['python/3.9.7']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/python-3.9.7-yj5alh (external python-3.9.7-f27a7nnhppeztxijpispnaedcfo2vjxi)
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_intel-2021.4.0-skylake/spdlog-1.9.2-wk3uen
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_intel-2021.4.0-skylake/reportinglib-2.5.6-gdhqyp
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_intel-2021.4.0-skylake/hdf5-1.10.7-ofveh3
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_intel-2021.4.0-skylake/libsonata-report-1.0.0.20220218-ldqdrn
==> Installing coreneuron-1.0.0.20220218-vlyasz3tnx2qtt5jveu4ropwbkiqafni
==> No binary for coreneuron-1.0.0.20220218-vlyasz3tnx2qtt5jveu4ropwbkiqafni found: installing from source
==> No patches needed for coreneuron
==> coreneuron: Executing phase: 'cmake'
==> coreneuron: Executing phase: 'build'
==> coreneuron: Executing phase: 'install'
==> coreneuron: Successfully installed coreneuron-1.0.0.20220218-vlyasz3tnx2qtt5jveu4ropwbkiqafni
Fetch: 4.96s. Build: 31.92s. Total: 36.88s.
[+] /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/software/install_intel-2021.4.0-skylake/coreneuron-1.0.0.20220218-vlyasz
$ chmod -R g+rX "${SPACK_BUILD}"
$ if [[ ${install_failed} == 1 ]]; then exit 1; fi
$ if [ ${SPACK_USE_CCACHE+x} ]; then
$ ccache --cleanup
$ ccache --show-stats --verbose
Summary:
Cache directory: /nvme/bbpcihpcproj12/302161/ccache
Primary config: /nvme/bbpcihpcproj12/302161/ccache/ccache.conf
Secondary config: /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/ccache-4.4.2-6qbndl/etc/ccache.conf
Stats updated: Sun Feb 27 18:36:07 2022
Hits: 110 / 115 (95.65 %)
Direct: 92 / 116 (79.31 %)
Preprocessed: 18 / 24 (75.00 %)
Misses: 5
Direct: 24
Preprocessed: 6
Uncacheable: 18
Primary storage:
Hits: 202 / 232 (87.07 %)
Misses: 30
Cache size (GB): 0.24 / 0.51 (46.22 %)
Files: 4758
Uncacheable:
Called for linking: 14
Compiler produced no output: 1
No input file: 3
$ tar -C "${CCACHE_DIR}" -cf "${CI_PROJECT_DIR}/ccache.tar" .
$ fi
$ touch ${SPACK_STAGE_DIR}/spack-configure-args.txt
touch: cannot touch ‘/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169149/spack-build/spack-stage-coreneuron-develop-vlyasz3tnx2qtt5jveu4ropwbkiqafni/spack-configure-args.txt’: No such file or directory
srun: error: r1i7n20: task 0: Exited with exit code 1
section_end:1645983369:step_script section_start:1645983369:upload_artifacts_on_failure Uploading artifacts for failed job
Uploading artifacts...
Runtime platform  arch=amd64 os=linux pid=193080 revision=58ba2b95 version=14.2.0
install.xml: found 1 matching files and directories
WARNING: spack_build_info.env: no matching files 
spack_clone_variables.env: found 1 matching files and directories
initial_environment.env: found 1 matching files and directories
WARNING: spack-build-env.txt: no matching files 
WARNING: spack-build-out.txt: no matching files 
WARNING: spack-configure-args.txt: no matching files
Uploading artifacts as "archive" to coordinator... ok id=169149 responseStatus=201 Created token=WQxHzDuS
Uploading artifacts...
Runtime platform  arch=amd64 os=linux pid=193124 revision=58ba2b95 version=14.2.0
install.xml: found 1 matching files and directories
Uploading artifacts as "junit" to coordinator... ok id=169149 responseStatus=201 Created token=WQxHzDuS
Uploading artifacts...
Runtime platform  arch=amd64 os=linux pid=193168 revision=58ba2b95 version=14.2.0
WARNING: spack_build_info.env: no matching files 
spack_clone_variables.env: found 1 matching files and directories
Uploading artifacts as "dotenv" to coordinator... ok id=169149 responseStatus=201 Created token=WQxHzDuS
section_end:1645983371:upload_artifacts_on_failure section_start:1645983371:cleanup_file_variables Cleaning up project directory and file based variables
section_end:1645983371:cleanup_file_variables ERROR: Job failed: exit status 1

Running with gitlab-runner 14.3.2 (e0218c92)
 on BB5 map runner pnPo3yJy
section_start:1645983659:resolve_secrets Resolving secrets
section_end:1645983659:resolve_secrets section_start:1645983659:prepare_executor Preparing the "custom" executor
Using Custom executor with driver BB5 PROD runner v0.0.3...
BB5 PROD runner running on bbpv1.epfl.ch, version 14.3.2, user
TMPDIR is /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/tmp/custom-executor225657043, slurm job id
Runner ID 29, project root hpc, project name coreneuron
Pipeline ID 40305, build ref b78986445f9d60261c743bdbd9ff77689666ae2f, job ID 169150
Build dir /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P40305, optional exclusive flag , optional cpus per task flag --cpus-per-task=8
A slurm job will be created with name GL_J169150_PROD_P112_CP4_C4
Job parameters: memory=76G, cpus_per_task=8, duration=1:00:00, constraint=cpu ntasks=2 account=proj9998 user=bbpcihpcproj12 partition=prod
sbatch: INFO: Activating auto partition selection plugin, please report errors to HPC/CS
sbatch: INFO: Job specifies cpu constraint, setting --constraint=[skl|clx]
Submitted batch job 302165
job state: R
sbatch: sbatch -p prod -A proj9998 --ntasks=2 --cpus-per-task=8 --mem=76G --job-name=GL_J169150_PROD_P112_CP4_C4 -C cpu --no-requeue -D /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P40305 --time=1:00:00 --wrap="sleep infinity"
srun: srun --mpi=none --chdir=/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P40305 --ntasks=2 --jobid=302165 --cpus-per-task=8 --mem=76G
section_end:1645983675:prepare_executor section_start:1645983675:prepare_script Preparing environment
Running on r1i7n20 via bbpv1.epfl.ch...
section_end:1645983679:prepare_script section_start:1645983679:get_sources Getting source from Git repository
Skipping Git repository setup
Skipping Git checkout
Skipping Git submodules setup
section_end:1645983680:get_sources section_start:1645983680:restore_cache Restoring cache
Checking cache for build:coreneuron:nmodl:intel-8...
Runtime platform  arch=amd64 os=linux pid=203194 revision=58ba2b95 version=14.2.0
cache.zip is up to date 
Successfully extracted cache
section_end:1645983683:restore_cache section_start:1645983683:download_artifacts Downloading artifacts
Downloading artifacts for build:nmodl (169144)...
Runtime platform  arch=amd64 os=linux pid=203408 revision=58ba2b95 version=14.2.0
Downloading artifacts from coordinator... ok  id=169144 responseStatus=200 OK token=TRD26edM
section_end:1645983683:download_artifacts section_start:1645983683:step_script Executing "step_script" stage of the job script
WARNING: Starting with version 14.0 the 'build_script' stage will be replaced with 'step_script': https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26426
$ SPACK_PACKAGE_DEPENDENCIES="${SPACK_PACKAGE_DEPENDENCIES}^hpe-mpi%gcc"
$ SPACK_BUILD="${PWD}/spack-build"
$ env -0 | sort -z | xargs -0 -L 1 echo > initial_environment.env
$ . ${SPACK_ROOT}/share/spack/setup-env.sh
$ export SPACK_USER_CONFIG_PATH=${PWD}/spack-config
$ mkdir ${SPACK_USER_CONFIG_PATH}
$ cat > ${SPACK_USER_CONFIG_PATH}/config.yaml << END_SCRIPT # collapsed multi-line command
$ spack config blame config
--- config:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169150/spack-config/config.yaml:2 build_stage:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169150/spack-config/config.yaml:3 - /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169150/spack-build
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169150/spack-config/config.yaml:4 source_cache: /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169150/spack-source-cache
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/config.yaml:2 ccache: True
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:2 install_tree:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:3 root: $user_cache_path/software
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:4 projections:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:5 all: install_{compiler.name}-{compiler.version}-{target}/{name}-{version}-{hash:6}
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:6 module_roots:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:7 tcl: $user_cache_path/modules
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:9 environments:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:10 root: $user_cache_path/environments
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:88 concretization: separately
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:12 build_jobs: 8
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:18 extensions:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:18 - $spack/bluebrain/spack-scripting
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:19 # This is the path to the root of the Spack install tree.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:19 # You can use $spack here to refer to the root of the spack instance.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:37 template_dirs:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:37 - $spack/share/spack/templates
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Temporary locations Spack can try to use for builds.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Recommended options are given below.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Builds can be faster in temporary directories on some (e.g., HPC) systems.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Specifying `$tempdir` will ensure use of the default temporary directory
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # (i.e., ``$TMP` or ``$TMPDIR``).
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Another option that prevents conflicts and potential permission issues is
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # to specify `$user_cache_path/stage`, which ensures each user builds in their
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # home directory.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # A more traditional path uses the value of `$spack/var/spack/stage`, which
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # builds directly inside Spack's instance without staging them in a
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # temporary space. Problems with specifying a path inside a Spack instance
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # are that it precludes its use as a system package and its ability to be
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # pip installable.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # In any case, if the username is not already in the path, Spack will append
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # the value of `$user` in an attempt to avoid potential conflicts between
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # users in shared temporary spaces.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # The build stage can be purged with `spack clean --stage` and
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # `spack clean -a`, so it is important that the specified directory uniquely
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # identifies Spack staging to avoid accidentally wiping out non-Spack work.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:73 test_stage: $user_cache_path/test
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:82 misc_cache: $user_cache_path/cache
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:94 connect_timeout: 10
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:99 verify_ssl: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:107 suppress_gpg_warnings: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:113 install_missing_compilers: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:118 checksum: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:123 deprecated: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:128 dirty: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:136 build_language: C
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:145 locks: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:150 url_fetch_method: urllib
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:176 concretizer: clingo
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:183 db_lock_timeout: 300
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:191 package_lock_timeout: null
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:197 shared_linking: rpath
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:202 allow_sgid: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:207 terminal_title: False
_builtin debug: False
$ export XDG_CONFIG_HOME=${PWD}/local_config
$ mkdir -p "${XDG_CONFIG_HOME}/git"
$ echo -e "[url \"https://gitlab-ci-token:${CI_JOB_TOKEN}@bbpgitlab.epfl.ch/\"]\n insteadOf = git@bbpgitlab.epfl.ch:" > "${XDG_CONFIG_HOME}/git/config"
$ cat "${XDG_CONFIG_HOME}/git/config"
[url "https://gitlab-ci-token:[MASKED]@bbpgitlab.epfl.ch/"]
insteadOf = git@bbpgitlab.epfl.ch:
$ SPACK_FULL_SPEC="${SPACK_PACKAGE}${SPACK_PACKAGE_COMPILER:+%}${SPACK_PACKAGE_COMPILER} ${SPACK_PACKAGE_SPEC} ${SPACK_PACKAGE_DEPENDENCIES} ${SPACK_PACKAGE_DEPENDENCY_ON_PREVIOUS_JOB}"
$ echo "Preparing to install ${SPACK_FULL_SPEC}"
Preparing to install coreneuron%intel +nmodl+tests~legacy-unit build_type=Debug ^hpe-mpi%gcc ^/cnhmkfzsucqnmy6dfxyfikno2yj57grf
$ JSON_SPEC=$(spack spec --json ${SPACK_FULL_SPEC})
$ SPACK_INSTALLED_HASH=$(module load unstable python; echo "${JSON_SPEC}" | python -c "import json, sys; print(json.loads(sys.stdin.read())[\"spec\"][\"nodes\"][0][\"hash\"])")
$ echo "Determined its hash will be ${SPACK_INSTALLED_HASH}"
Determined its hash will be uvyo7araeayzbwzymt2uukci7mli7bqq
$ SPACK_STAGE_DIR=${SPACK_BUILD}/spack-stage-${SPACK_PACKAGE}-develop-${SPACK_INSTALLED_HASH}
$ SPACK_BUILD_DIR=${SPACK_STAGE_DIR}/spack-build-${SPACK_INSTALLED_HASH:0:7}
$ SPACK_SOURCE_DIR=${SPACK_STAGE_DIR}/spack-src
$ if [ ${SPACK_USE_CCACHE+x} ]; then
$ module load unstable ccache
$ export CCACHE_BASEDIR=$(realpath -P ${CI_BUILDS_DIR})
$ echo CCACHE_BASEDIR=${CCACHE_BASEDIR}
CCACHE_BASEDIR=/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305
$ export CCACHE_MAXSIZE=512M
$ export CCACHE_DIR="${TMPDIR}/ccache"
$ mkdir -p ${CCACHE_DIR}
$ if [ -f ${CI_PROJECT_DIR}/ccache.tar ]; then
$ tar -C "${CCACHE_DIR}" -xf "${CI_PROJECT_DIR}/ccache.tar"
$ fi
$ ccache --zero-stats
Statistics zeroed
$ ccache --show-stats --verbose
Summary:
Cache directory: /nvme/bbpcihpcproj12/302165/ccache
Primary config: /nvme/bbpcihpcproj12/302165/ccache/ccache.conf
Secondary config: /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/ccache-4.4.2-6qbndl/etc/ccache.conf
Stats updated: Sun Feb 27 18:41:47 2022
Hits: 0 / 0
Direct: 0 / 0
Preprocessed: 0 / 0
Misses: 0
Direct: 0
Preprocessed: 0
Primary storage:
Hits: 0 / 0
Misses: 0
Cache size (GB): 0.26 / 0.51 (51.67 %)
Files: 5906
$ fi
$ module load unstable git ${SPACK_EXTRA_MODULES}
$ spack uninstall -y --dependents /${SPACK_INSTALLED_HASH} || true
==> Error: No installed spec matches the hash: 'uvyo7araeayzbwzymt2uukci7mli7bqq'
$ spack spec -Il ${SPACK_FULL_SPEC}
Input spec
--------------------------------
- coreneuron%intel~legacy-unit+nmodl+tests build_type=Debug
- ^hpe-mpi%gcc
[+] ^nmodl@develop%gcc@11.2.0~ipo~legacy-unit~python build_type=RelWithDebInfo arch=linux-rhel7-skylake
[^] ^py-jinja2@3.0.1%gcc@11.2.0~i18n arch=linux-rhel7-skylake
[^] ^py-markupsafe@2.0.1%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^python@3.9.7%gcc@11.2.0+bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93189bc278fbc37a50ed7f183bd8aaf249a8e1670a465f0db6bb4f8cf87,4c2457325f2b608b1b6a2c63087df8c26e07db3e3d493caf36a56f0ecf6fb768,f2fd060afc4b4618fe8104c4c5d771f36dc55b1db5a4623785a4ea707ec72fb4 arch=linux-rhel7-skylake
[^] ^py-pytest@6.2.4%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-attrs@21.2.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-iniconfig@1.1.1%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-packaging@21.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-pyparsing@2.4.7%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-pluggy@0.13.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-py@1.9.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-setuptools@57.4.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-toml@0.10.2%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-pyyaml@5.3.1%gcc@11.2.0+libyaml arch=linux-rhel7-skylake
[^] ^libyaml@0.2.5%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-sympy@1.8%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-mpmath@1.1.0%gcc@11.2.0 arch=linux-rhel7-skylake
Concretized
--------------------------------
- uvyo7ar coreneuron@1.0.0.20220218%intel@2021.4.0~caliper~codegenopt~gpu~ipo~ispc~knl~legacy-unit+mpi+nmodl~openmp~profile+report+shared~sympy~sympyopt+tests~unified build_type=Debug arch=linux-rhel7-skylake
- dwdch6b ^bison@3.8.2%intel@2021.4.0 arch=linux-rhel7-skylake
[^] dl7pfht ^boost@1.78.0%intel@2021.4.0+atomic+chrono~clanglibcpp~container~context~coroutine+date_time~debug+exception~fiber+filesystem+graph~icu+iostreams+locale+log+math~mpi+multithreaded~numpy+pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave cxxstd=98 visibility=hidden arch=linux-rhel7-skylake
[^] 2qmvlfy ^cmake@3.21.4%gcc@11.2.0~doc+ncurses+openssl+ownlibs~qt build_type=Release arch=linux-rhel7-skylake
- s4ueg72 ^flex@2.6.3%intel@2021.4.0+lex~nls arch=linux-rhel7-skylake
[^] hyunzkn ^hpe-mpi@2.25.hmpt%gcc@11.2.0 arch=linux-rhel7-skylake
[^] btmx4zs ^libsonata-report@1.0.0.20220218%gcc@11.2.0~ipo+mpi build_type=RelWithDebInfo arch=linux-rhel7-skylake
[^] utj47fz ^hdf5@1.10.7%gcc@11.2.0+cxx~fortran+hl~ipo~java+mpi+shared~szip~threadsafe+tools api=default build_type=RelWithDebInfo arch=linux-rhel7-skylake
[^] ihxi5rl ^pkgconf@1.8.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] evtnqzd ^zlib@1.2.11%gcc@11.2.0+optimize+pic+shared arch=linux-rhel7-skylake
[^] x72vveu ^spdlog@1.9.2%gcc@11.2.0~ipo+shared build_type=RelWithDebInfo arch=linux-rhel7-skylake
- 6ggc5yr ^ninja@1.10.2%intel@2021.4.0 arch=linux-rhel7-skylake
[+] cnhmkfz ^nmodl@develop%gcc@11.2.0~ipo~legacy-unit~python build_type=RelWithDebInfo arch=linux-rhel7-skylake
[^] onq3mhg ^py-jinja2@3.0.1%gcc@11.2.0~i18n arch=linux-rhel7-skylake
[^] 4ectow5 ^py-markupsafe@2.0.1%gcc@11.2.0 arch=linux-rhel7-skylake
[^] 72xzp3v ^python@3.9.7%gcc@11.2.0+bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93189bc278fbc37a50ed7f183bd8aaf249a8e1670a465f0db6bb4f8cf87,4c2457325f2b608b1b6a2c63087df8c26e07db3e3d493caf36a56f0ecf6fb768,f2fd060afc4b4618fe8104c4c5d771f36dc55b1db5a4623785a4ea707ec72fb4 arch=linux-rhel7-skylake
[^] 22arfs4 ^py-pytest@6.2.4%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ascbeii ^py-attrs@21.2.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] kvw3vhm ^py-iniconfig@1.1.1%gcc@11.2.0 arch=linux-rhel7-skylake
[^] 7iyiygo ^py-packaging@21.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] c7qvw2q ^py-pyparsing@2.4.7%gcc@11.2.0 arch=linux-rhel7-skylake
[^] mazoiox ^py-pluggy@0.13.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] y7rfzdj ^py-py@1.9.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] v4z3s5e ^py-setuptools@57.4.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] w4gddqx ^py-toml@0.10.2%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ci5oe5b ^py-pyyaml@5.3.1%gcc@11.2.0+libyaml arch=linux-rhel7-skylake
[^] xj2wlac ^libyaml@0.2.5%gcc@11.2.0 arch=linux-rhel7-skylake
[^] xtett6q ^py-sympy@1.8%gcc@11.2.0 arch=linux-rhel7-skylake
[^] dzb2mfs ^py-mpmath@1.1.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] 5fkun4i ^reportinglib@2.5.6%gcc@11.2.0~ipo~profile+shared~tests build_type=RelWithDebInfo arch=linux-rhel7-skylake
$ spack ${SPACK_INSTALL_EXTRA_FLAGS} install -j${SLURM_CPUS_PER_TASK} --log-format=junit --log-file=${CI_PROJECT_DIR}/install.xml --keep-stage ${SPACK_FULL_SPEC} || install_failed=1
==> bison@3.8.2 : has external module in ['bison/3.8.2']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/bison-3.8.2-ej2cew (external bison-3.8.2-dwdch6bmdeclr2novthsywtrryotawwz)
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/boost-1.78.0-hiorc7 (external boost-1.78.0-dl7pfhtsvyaq3273gr2g3l5vr37eeydc)
==> cmake@3.21.4 : has external module in ['cmake/3.21.4']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/cmake-3.21.4-cdyb7k (external cmake-3.21.4-2qmvlfyylrv3t5ankluyr5cqey2nlfzd)
==> flex@2.6.3 : has external module in ['flex/2.6.3']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/flex-2.6.3-3mtl7j (external flex-2.6.3-s4ueg72j7l6vkdyvfxj2tweo7v7s3otx)
==> hpe-mpi@2.25.hmpt : has external module in ['hpe-mpi/2.25.hmpt']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/hpe-mpi-2.25.hmpt-4ukyxt (external hpe-mpi-2.25.hmpt-hyunzknktdzri34bx26bupkvapehonql)
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/pkgconf-1.8.0-ihxi5r
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/zlib-1.2.11-mswjss (external zlib-1.2.11-evtnqzdtuvprwnbd2nljimpzywrhw4uo)
==> ninja@1.10.2 : has external module in ['ninja/1.10.2']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/ninja-1.10.2-eh33rh (external ninja-1.10.2-6ggc5yre7qddwxdjmn7sfptpdoiy4dtp)
==> python@3.9.7 : has external module in ['python/3.9.7']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/python-3.9.7-yj5alh (external python-3.9.7-72xzp3vgzh5b424qevkfruhs3ajzqbiy)
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/libyaml-0.2.5-xj2wla
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/spdlog-1.9.2-x72vve
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/reportinglib-2.5.6-5fkun4
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/hdf5-1.10.7-utj47f
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-pluggy-0.13.0-mazoio
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-markupsafe-2.0.1-4ectow
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-setuptools-57.4.0-v4z3s5
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-pyparsing-2.4.7-c7qvw2
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-attrs-21.2.0-ascbei
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-iniconfig-1.1.1-kvw3vh
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-mpmath-1.1.0-dzb2mf
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-py-1.9.0-y7rfzd
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-toml-0.10.2-w4gddq
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-pyyaml-5.3.1-ci5oe5
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/libsonata-report-1.0.0.20220218-btmx4z
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-jinja2-3.0.1-onq3mh
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-packaging-21.0-7iyiyg
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-sympy-1.8-xtett6
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-pytest-6.2.4-22arfs
[+] /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/software/install_gcc-11.2.0-skylake/nmodl-develop-cnhmkf
==> Installing coreneuron-1.0.0.20220218-uvyo7araeayzbwzymt2uukci7mli7bqq
==> No binary for coreneuron-1.0.0.20220218-uvyo7araeayzbwzymt2uukci7mli7bqq found: installing from source
==> No patches needed for coreneuron
==> coreneuron: Executing phase: 'cmake'
==> coreneuron: Executing phase: 'build'
==> coreneuron: Executing phase: 'install'
==> coreneuron: Successfully installed coreneuron-1.0.0.20220218-uvyo7araeayzbwzymt2uukci7mli7bqq
Fetch: 4.84s. Build: 35.82s. Total: 40.65s.
[+] /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/software/install_intel-2021.4.0-skylake/coreneuron-1.0.0.20220218-uvyo7a
$ chmod -R g+rX "${SPACK_BUILD}"
$ if [[ ${install_failed} == 1 ]]; then exit 1; fi
$ if [ ${SPACK_USE_CCACHE+x} ]; then
$ ccache --cleanup
$ ccache --show-stats --verbose
Summary:
Cache directory: /nvme/bbpcihpcproj12/302165/ccache
Primary config: /nvme/bbpcihpcproj12/302165/ccache/ccache.conf
Secondary config: /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/ccache-4.4.2-6qbndl/etc/ccache.conf
Stats updated: Sun Feb 27 18:43:14 2022
Hits: 89 / 94 (94.68 %)
Direct: 80 / 95 (84.21 %)
Preprocessed: 9 / 15 (60.00 %)
Misses: 5
Direct: 15
Preprocessed: 6
Uncacheable: 16
Primary storage:
Hits: 169 / 190 (88.95 %)
Misses: 21
Cache size (GB): 0.26 / 0.51 (51.71 %)
Files: 5924
Uncacheable:
Called for linking: 12
Compiler produced no output: 1
No input file: 3
$ tar -C "${CCACHE_DIR}" -cf "${CI_PROJECT_DIR}/ccache.tar" .
$ fi
$ touch ${SPACK_STAGE_DIR}/spack-configure-args.txt
touch: cannot touch ‘/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169150/spack-build/spack-stage-coreneuron-develop-uvyo7araeayzbwzymt2uukci7mli7bqq/spack-configure-args.txt’: No such file or directory
srun: error: r1i7n20: task 0: Exited with exit code 1
section_end:1645983795:step_script section_start:1645983795:upload_artifacts_on_failure Uploading artifacts for failed job
Uploading artifacts...
Runtime platform  arch=amd64 os=linux pid=207109 revision=58ba2b95 version=14.2.0
install.xml: found 1 matching files and directories
spack_build_info.env: found 1 matching files and directories
spack_clone_variables.env: found 1 matching files and directories
initial_environment.env: found 1 matching files and directories
spack-build-env.txt: found 1 matching files and directories
spack-build-out.txt: found 1 matching files and directories
spack-configure-args.txt: found 1 matching files and directories
Uploading artifacts as "archive" to coordinator... ok id=169150 responseStatus=201 Created token=2bAv7ogL
Uploading artifacts...
Runtime platform  arch=amd64 os=linux pid=207154 revision=58ba2b95 version=14.2.0
install.xml: found 1 matching files and directories
Uploading artifacts as "junit" to coordinator... ok id=169150 responseStatus=201 Created token=2bAv7ogL
Uploading artifacts...
Runtime platform  arch=amd64 os=linux pid=207195 revision=58ba2b95 version=14.2.0
spack_build_info.env: found 1 matching files and directories
spack_clone_variables.env: found 1 matching files and directories
Uploading artifacts as "dotenv" to coordinator... ok id=169150 responseStatus=201 Created token=2bAv7ogL
section_end:1645983796:upload_artifacts_on_failure section_start:1645983796:cleanup_file_variables Cleaning up project directory and file based variables
section_end:1645983797:cleanup_file_variables ERROR: Job failed: exit status 1

Running with gitlab-runner 14.3.2 (e0218c92)
 on BB5 map runner pnPo3yJy
section_start:1645983659:resolve_secrets Resolving secrets
section_end:1645983659:resolve_secrets section_start:1645983659:prepare_executor Preparing the "custom" executor
Using Custom executor with driver BB5 PROD runner v0.0.3...
BB5 PROD runner running on bbpv1.epfl.ch, version 14.3.2, user
TMPDIR is /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/tmp/custom-executor268592697, slurm job id
Runner ID 29, project root hpc, project name coreneuron
Pipeline ID 40305, build ref b78986445f9d60261c743bdbd9ff77689666ae2f, job ID 169148
Build dir /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P40305, optional exclusive flag , optional cpus per task flag --cpus-per-task=8
A slurm job will be created with name GL_J169148_PROD_P112_CP3_C3
Job parameters: memory=76G, cpus_per_task=8, duration=1:00:00, constraint=cpu ntasks=2 account=proj9998 user=bbpcihpcproj12 partition=prod
sbatch: INFO: Activating auto partition selection plugin, please report errors to HPC/CS
sbatch: INFO: Job specifies cpu constraint, setting --constraint=[skl|clx]
Submitted batch job 302167
job state: R
sbatch: sbatch -p prod -A proj9998 --ntasks=2 --cpus-per-task=8 --mem=76G --job-name=GL_J169148_PROD_P112_CP3_C3 -C cpu --no-requeue -D /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P40305 --time=1:00:00 --wrap="sleep infinity"
srun: srun --mpi=none --chdir=/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P40305 --ntasks=2 --jobid=302167 --cpus-per-task=8 --mem=76G
section_end:1645983675:prepare_executor section_start:1645983675:prepare_script Preparing environment
Running on r1i7n21 via bbpv1.epfl.ch...
section_end:1645983678:prepare_script section_start:1645983678:get_sources Getting source from Git repository
Skipping Git repository setup
Skipping Git checkout
Skipping Git submodules setup
section_end:1645983678:get_sources section_start:1645983678:restore_cache Restoring cache
Checking cache for build:coreneuron:nmodl:nvhpc:acc-8...
Runtime platform  arch=amd64 os=linux pid=140210 revision=58ba2b95 version=14.2.0
cache.zip is up to date 
Successfully extracted cache
section_end:1645983681:restore_cache section_start:1645983681:download_artifacts Downloading artifacts
Downloading artifacts for build:nmodl (169144)...
Runtime platform  arch=amd64 os=linux pid=140275 revision=58ba2b95 version=14.2.0
Downloading artifacts from coordinator... ok  id=169144 responseStatus=200 OK token=TRD26edM
section_end:1645983682:download_artifacts section_start:1645983682:step_script Executing "step_script" stage of the job script
WARNING: Starting with version 14.0 the 'build_script' stage will be replaced with 'step_script': https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26426
$ SPACK_PACKAGE_DEPENDENCIES="${SPACK_PACKAGE_DEPENDENCIES}^hpe-mpi%gcc"
$ SPACK_BUILD="${PWD}/spack-build"
$ env -0 | sort -z | xargs -0 -L 1 echo > initial_environment.env
$ . ${SPACK_ROOT}/share/spack/setup-env.sh
$ export SPACK_USER_CONFIG_PATH=${PWD}/spack-config
$ mkdir ${SPACK_USER_CONFIG_PATH}
$ cat > ${SPACK_USER_CONFIG_PATH}/config.yaml << END_SCRIPT # collapsed multi-line command
$ spack config blame config
--- config:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169148/spack-config/config.yaml:2 build_stage:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169148/spack-config/config.yaml:3 - /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169148/spack-build
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169148/spack-config/config.yaml:4 source_cache: /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169148/spack-source-cache
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/config.yaml:2 ccache: True
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:2 install_tree:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:3 root: $user_cache_path/software
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:4 projections:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:5 all: install_{compiler.name}-{compiler.version}-{target}/{name}-{version}-{hash:6}
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:6 module_roots:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:7 tcl: $user_cache_path/modules
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:9 environments:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:10 root: $user_cache_path/environments
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:88 concretization: separately
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:12 build_jobs: 8
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:18 extensions:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:18 - $spack/bluebrain/spack-scripting
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:19 # This is the path to the root of the Spack install tree.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:19 # You can use $spack here to refer to the root of the spack instance.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:37 template_dirs:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:37 - $spack/share/spack/templates
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Temporary locations Spack can try to use for builds.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Recommended options are given below.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Builds can be faster in temporary directories on some (e.g., HPC) systems.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Specifying `$tempdir` will ensure use of the default temporary directory
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # (i.e., ``$TMP` or ``$TMPDIR``).
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Another option that prevents conflicts and potential permission issues is
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # to specify `$user_cache_path/stage`, which ensures each user builds in their
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # home directory.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # A more traditional path uses the value of `$spack/var/spack/stage`, which
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # builds directly inside Spack's instance without staging them in a
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # temporary space. Problems with specifying a path inside a Spack instance
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # are that it precludes its use as a system package and its ability to be
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # pip installable.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # In any case, if the username is not already in the path, Spack will append
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # the value of `$user` in an attempt to avoid potential conflicts between
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # users in shared temporary spaces.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # The build stage can be purged with `spack clean --stage` and
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # `spack clean -a`, so it is important that the specified directory uniquely
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # identifies Spack staging to avoid accidentally wiping out non-Spack work.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:73 test_stage: $user_cache_path/test
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:82 misc_cache: $user_cache_path/cache
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:94 connect_timeout: 10
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:99 verify_ssl: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:107 suppress_gpg_warnings: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:113 install_missing_compilers: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:118 checksum: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:123 deprecated: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:128 dirty: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:136 build_language: C
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:145 locks: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:150 url_fetch_method: urllib
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:176 concretizer: clingo
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:183 db_lock_timeout: 300
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:191 package_lock_timeout: null
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:197 shared_linking: rpath
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:202 allow_sgid: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:207 terminal_title: False
_builtin debug: False
$ export XDG_CONFIG_HOME=${PWD}/local_config
$ mkdir -p "${XDG_CONFIG_HOME}/git"
$ echo -e "[url \"https://gitlab-ci-token:${CI_JOB_TOKEN}@bbpgitlab.epfl.ch/\"]\n insteadOf = git@bbpgitlab.epfl.ch:" > "${XDG_CONFIG_HOME}/git/config"
$ cat "${XDG_CONFIG_HOME}/git/config"
[url "https://gitlab-ci-token:[MASKED]@bbpgitlab.epfl.ch/"]
insteadOf = git@bbpgitlab.epfl.ch:
$ SPACK_FULL_SPEC="${SPACK_PACKAGE}${SPACK_PACKAGE_COMPILER:+%}${SPACK_PACKAGE_COMPILER} ${SPACK_PACKAGE_SPEC} ${SPACK_PACKAGE_DEPENDENCIES} ${SPACK_PACKAGE_DEPENDENCY_ON_PREVIOUS_JOB}"
$ echo "Preparing to install ${SPACK_FULL_SPEC}"
Preparing to install coreneuron%nvhpc +nmodl~openmp+gpu+tests~legacy-unit~report+sympy build_type=RelWithDebInfo ^hpe-mpi%gcc ^/cnhmkfzsucqnmy6dfxyfikno2yj57grf
$ JSON_SPEC=$(spack spec --json ${SPACK_FULL_SPEC})
$ SPACK_INSTALLED_HASH=$(module load unstable python; echo "${JSON_SPEC}" | python -c "import json, sys; print(json.loads(sys.stdin.read())[\"spec\"][\"nodes\"][0][\"hash\"])")
$ echo "Determined its hash will be ${SPACK_INSTALLED_HASH}"
Determined its hash will be c4amjtoqlmcvk6kcsa63rseyyuib5y2u
$ SPACK_STAGE_DIR=${SPACK_BUILD}/spack-stage-${SPACK_PACKAGE}-develop-${SPACK_INSTALLED_HASH}
$ SPACK_BUILD_DIR=${SPACK_STAGE_DIR}/spack-build-${SPACK_INSTALLED_HASH:0:7}
$ SPACK_SOURCE_DIR=${SPACK_STAGE_DIR}/spack-src
$ if [ ${SPACK_USE_CCACHE+x} ]; then
$ module load unstable ccache
$ export CCACHE_BASEDIR=$(realpath -P ${CI_BUILDS_DIR})
$ echo CCACHE_BASEDIR=${CCACHE_BASEDIR}
CCACHE_BASEDIR=/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305
$ export CCACHE_MAXSIZE=512M
$ export CCACHE_DIR="${TMPDIR}/ccache"
$ mkdir -p ${CCACHE_DIR}
$ if [ -f ${CI_PROJECT_DIR}/ccache.tar ]; then
$ tar -C "${CCACHE_DIR}" -xf "${CI_PROJECT_DIR}/ccache.tar"
$ fi
$ ccache --zero-stats
Statistics zeroed
$ ccache --show-stats --verbose
Summary:
Cache directory: /nvme/bbpcihpcproj12/302167/ccache
Primary config: /nvme/bbpcihpcproj12/302167/ccache/ccache.conf
Secondary config: /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/ccache-4.4.2-6qbndl/etc/ccache.conf
Stats updated: Sun Feb 27 18:41:47 2022
Hits: 0 / 0
Direct: 0 / 0
Preprocessed: 0 / 0
Misses: 0
Direct: 0
Preprocessed: 0
Primary storage:
Hits: 0 / 0
Misses: 0
Cache size (GB): 0.19 / 0.51 (37.17 %)
Files: 6758
$ fi
$ module load unstable git ${SPACK_EXTRA_MODULES}
$ spack uninstall -y --dependents /${SPACK_INSTALLED_HASH} || true
==> Error: No installed spec matches the hash: 'c4amjtoqlmcvk6kcsa63rseyyuib5y2u'
$ spack spec -Il ${SPACK_FULL_SPEC}
Input spec
--------------------------------
- coreneuron%nvhpc+gpu~legacy-unit+nmodl~openmp~report+sympy+tests build_type=RelWithDebInfo
- ^hpe-mpi%gcc
[+] ^nmodl@develop%gcc@11.2.0~ipo~legacy-unit~python build_type=RelWithDebInfo arch=linux-rhel7-skylake
[^] ^py-jinja2@3.0.1%gcc@11.2.0~i18n arch=linux-rhel7-skylake
[^] ^py-markupsafe@2.0.1%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^python@3.9.7%gcc@11.2.0+bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93189bc278fbc37a50ed7f183bd8aaf249a8e1670a465f0db6bb4f8cf87,4c2457325f2b608b1b6a2c63087df8c26e07db3e3d493caf36a56f0ecf6fb768,f2fd060afc4b4618fe8104c4c5d771f36dc55b1db5a4623785a4ea707ec72fb4 arch=linux-rhel7-skylake
[^] ^py-pytest@6.2.4%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-attrs@21.2.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-iniconfig@1.1.1%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-packaging@21.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-pyparsing@2.4.7%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-pluggy@0.13.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-py@1.9.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-setuptools@57.4.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-toml@0.10.2%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-pyyaml@5.3.1%gcc@11.2.0+libyaml arch=linux-rhel7-skylake
[^] ^libyaml@0.2.5%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-sympy@1.8%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-mpmath@1.1.0%gcc@11.2.0 arch=linux-rhel7-skylake
Concretized
--------------------------------
- c4amjto coreneuron@1.0.0.20220218%nvhpc@22.2~caliper~codegenopt+gpu~ipo~ispc~knl~legacy-unit+mpi+nmodl~openmp~profile~report+shared+sympy~sympyopt+tests~unified build_type=RelWithDebInfo arch=linux-rhel7-skylake
- ocv5kkj ^bison@3.8.2%nvhpc@22.2 arch=linux-rhel7-skylake
[^] bfwiaoz ^boost@1.78.0%nvhpc@22.2+atomic+chrono~clanglibcpp~container~context~coroutine+date_time~debug+exception~fiber+filesystem+graph~icu+iostreams+locale+log+math~mpi+multithreaded~numpy+pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave cxxstd=98 visibility=hidden arch=linux-rhel7-skylake
- 55wtnz6 ^cmake@3.21.4%nvhpc@22.2~doc+ncurses+openssl+ownlibs~qt build_type=Release arch=linux-rhel7-skylake
[^] wd5k5wi ^cuda@11.6.0%nvhpc@22.2~dev arch=linux-rhel7-skylake
- xzdnug7 ^flex@2.6.3%nvhpc@22.2+lex~nls arch=linux-rhel7-skylake
[^] hyunzkn ^hpe-mpi@2.25.hmpt%gcc@11.2.0 arch=linux-rhel7-skylake
- ij6isvr ^ninja@1.10.2%nvhpc@22.2 arch=linux-rhel7-skylake
[+] cnhmkfz ^nmodl@develop%gcc@11.2.0~ipo~legacy-unit~python build_type=RelWithDebInfo arch=linux-rhel7-skylake
[^] onq3mhg ^py-jinja2@3.0.1%gcc@11.2.0~i18n arch=linux-rhel7-skylake
[^] 4ectow5 ^py-markupsafe@2.0.1%gcc@11.2.0 arch=linux-rhel7-skylake
[^] 72xzp3v ^python@3.9.7%gcc@11.2.0+bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93189bc278fbc37a50ed7f183bd8aaf249a8e1670a465f0db6bb4f8cf87,4c2457325f2b608b1b6a2c63087df8c26e07db3e3d493caf36a56f0ecf6fb768,f2fd060afc4b4618fe8104c4c5d771f36dc55b1db5a4623785a4ea707ec72fb4 arch=linux-rhel7-skylake
[^] 22arfs4 ^py-pytest@6.2.4%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ascbeii ^py-attrs@21.2.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] kvw3vhm ^py-iniconfig@1.1.1%gcc@11.2.0 arch=linux-rhel7-skylake
[^] 7iyiygo ^py-packaging@21.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] c7qvw2q ^py-pyparsing@2.4.7%gcc@11.2.0 arch=linux-rhel7-skylake
[^] mazoiox ^py-pluggy@0.13.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] y7rfzdj ^py-py@1.9.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] v4z3s5e ^py-setuptools@57.4.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] w4gddqx ^py-toml@0.10.2%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ci5oe5b ^py-pyyaml@5.3.1%gcc@11.2.0+libyaml arch=linux-rhel7-skylake
[^] xj2wlac ^libyaml@0.2.5%gcc@11.2.0 arch=linux-rhel7-skylake
[^] xtett6q ^py-sympy@1.8%gcc@11.2.0 arch=linux-rhel7-skylake
[^] dzb2mfs ^py-mpmath@1.1.0%gcc@11.2.0 arch=linux-rhel7-skylake
$ spack ${SPACK_INSTALL_EXTRA_FLAGS} install -j${SLURM_CPUS_PER_TASK} --log-format=junit --log-file=${CI_PROJECT_DIR}/install.xml --keep-stage ${SPACK_FULL_SPEC} || install_failed=1
==> bison@3.8.2 : has external module in ['bison/3.8.2']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/bison-3.8.2-ej2cew (external bison-3.8.2-ocv5kkjjvqshvbpp4gh3xrg4aj3xxh2k)
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/boost-1.78.0-hiorc7 (external boost-1.78.0-bfwiaozbrpjr7blzjkfcz67vabdnspxj)
==> cmake@3.21.4 : has external module in ['cmake/3.21.4']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/cmake-3.21.4-cdyb7k (external cmake-3.21.4-55wtnz6tfxjhgdgfbzzwzuhyl746wv7j)
==> cuda@11.6.0 : has external module in ['cuda/11.6.0']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/cuda-11.6.0-34qjta (external cuda-11.6.0-wd5k5wioelahwux72y3g3bytolebtkoh)
==> flex@2.6.3 : has external module in ['flex/2.6.3']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/flex-2.6.3-3mtl7j (external flex-2.6.3-xzdnug7pkmqiem4ll6p4chgbnwyt7epd)
==> hpe-mpi@2.25.hmpt : has external module in ['hpe-mpi/2.25.hmpt']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/hpe-mpi-2.25.hmpt-4ukyxt (external hpe-mpi-2.25.hmpt-hyunzknktdzri34bx26bupkvapehonql)
==> ninja@1.10.2 : has external module in ['ninja/1.10.2']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/ninja-1.10.2-eh33rh (external ninja-1.10.2-ij6isvr7fpc6nbooavdi3abtviuzpngb)
==> python@3.9.7 : has external module in ['python/3.9.7']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/python-3.9.7-yj5alh (external python-3.9.7-72xzp3vgzh5b424qevkfruhs3ajzqbiy)
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/libyaml-0.2.5-xj2wla
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-pluggy-0.13.0-mazoio
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-py-1.9.0-y7rfzd
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-setuptools-57.4.0-v4z3s5
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-attrs-21.2.0-ascbei
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-markupsafe-2.0.1-4ectow
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-iniconfig-1.1.1-kvw3vh
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-pyparsing-2.4.7-c7qvw2
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-mpmath-1.1.0-dzb2mf
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-toml-0.10.2-w4gddq
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-pyyaml-5.3.1-ci5oe5
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-jinja2-3.0.1-onq3mh
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-packaging-21.0-7iyiyg
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-sympy-1.8-xtett6
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-pytest-6.2.4-22arfs
[+] /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/software/install_gcc-11.2.0-skylake/nmodl-develop-cnhmkf
==> Installing coreneuron-1.0.0.20220218-c4amjtoqlmcvk6kcsa63rseyyuib5y2u
==> No binary for coreneuron-1.0.0.20220218-c4amjtoqlmcvk6kcsa63rseyyuib5y2u found: installing from source
==> No patches needed for coreneuron
==> coreneuron: Executing phase: 'cmake'
==> coreneuron: Executing phase: 'build'
==> coreneuron: Executing phase: 'install'
==> coreneuron: Successfully installed coreneuron-1.0.0.20220218-c4amjtoqlmcvk6kcsa63rseyyuib5y2u
Fetch: 4.55s. Build: 4m 45.75s. Total: 4m 50.30s.
[+] /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/software/install_nvhpc-22.2-skylake/coreneuron-1.0.0.20220218-c4amjt
$ chmod -R g+rX "${SPACK_BUILD}"
$ if [[ ${install_failed} == 1 ]]; then exit 1; fi
$ if [ ${SPACK_USE_CCACHE+x} ]; then
$ ccache --cleanup
$ ccache --show-stats --verbose
Summary:
Cache directory: /nvme/bbpcihpcproj12/302167/ccache
Primary config: /nvme/bbpcihpcproj12/302167/ccache/ccache.conf
Secondary config: /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/ccache-4.4.2-6qbndl/etc/ccache.conf
Stats updated: Sun Feb 27 18:47:25 2022
Hits: 5 / 96 (5.21 %)
Direct: 0 / 96 (0.00 %)
Preprocessed: 5 / 96 (5.21 %)
Misses: 91
Direct: 96
Preprocessed: 91
Uncacheable: 16
Primary storage:
Hits: 5 / 192 (2.60 %)
Misses: 187
Cache size (GB): 0.20 / 0.51 (38.87 %)
Files: 6945
Uncacheable:
Called for linking: 13
No input file: 3
$ tar -C "${CCACHE_DIR}" -cf "${CI_PROJECT_DIR}/ccache.tar" .
$ fi
$ touch ${SPACK_STAGE_DIR}/spack-configure-args.txt
touch: cannot touch ‘/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169148/spack-build/spack-stage-coreneuron-develop-c4amjtoqlmcvk6kcsa63rseyyuib5y2u/spack-configure-args.txt’: No such file or directory
srun: error: r1i7n21: task 0: Exited with exit code 1
section_end:1645984045:step_script section_start:1645984045:upload_artifacts_on_failure Uploading artifacts for failed job
Uploading artifacts...
Runtime platform  arch=amd64 os=linux pid=146101 revision=58ba2b95 version=14.2.0
install.xml: found 1 matching files and directories
spack_build_info.env: found 1 matching files and directories
spack_clone_variables.env: found 1 matching files and directories
initial_environment.env: found 1 matching files and directories
spack-build-env.txt: found 1 matching files and directories
spack-build-out.txt: found 1 matching files and directories
spack-configure-args.txt: found 1 matching files and directories
Uploading artifacts as "archive" to coordinator... ok id=169148 responseStatus=201 Created token=VPGKnVtU
Uploading artifacts...
Runtime platform  arch=amd64 os=linux pid=146137 revision=58ba2b95 version=14.2.0
install.xml: found 1 matching files and directories
Uploading artifacts as "junit" to coordinator... ok id=169148 responseStatus=201 Created token=VPGKnVtU
Uploading artifacts...
Runtime platform  arch=amd64 os=linux pid=146180 revision=58ba2b95 version=14.2.0
spack_build_info.env: found 1 matching files and directories
spack_clone_variables.env: found 1 matching files and directories
Uploading artifacts as "dotenv" to coordinator... ok id=169148 responseStatus=201 Created token=VPGKnVtU
section_end:1645984047:upload_artifacts_on_failure section_start:1645984047:cleanup_file_variables Cleaning up project directory and file based variables
section_end:1645984047:cleanup_file_variables ERROR: Job failed: exit status 1

Running with gitlab-runner 14.3.2 (e0218c92)
 on BB5 map runner pnPo3yJy
section_start:1645983658:resolve_secrets Resolving secrets
section_end:1645983658:resolve_secrets section_start:1645983658:prepare_executor Preparing the "custom" executor
Using Custom executor with driver BB5 PROD runner v0.0.3...
BB5 PROD runner running on bbpv1.epfl.ch, version 14.3.2, user
TMPDIR is /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/tmp/custom-executor224416815, slurm job id
Runner ID 29, project root hpc, project name coreneuron
Pipeline ID 40305, build ref b78986445f9d60261c743bdbd9ff77689666ae2f, job ID 169147
Build dir /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P40305, optional exclusive flag , optional cpus per task flag --cpus-per-task=8
A slurm job will be created with name GL_J169147_PROD_P112_CP0_C0
Job parameters: memory=76G, cpus_per_task=8, duration=1:00:00, constraint=cpu ntasks=2 account=proj9998 user=bbpcihpcproj12 partition=prod
sbatch: INFO: Activating auto partition selection plugin, please report errors to HPC/CS
sbatch: INFO: Job specifies cpu constraint, setting --constraint=[skl|clx]
Submitted batch job 302166
job state: R
sbatch: sbatch -p prod -A proj9998 --ntasks=2 --cpus-per-task=8 --mem=76G --job-name=GL_J169147_PROD_P112_CP0_C0 -C cpu --no-requeue -D /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P40305 --time=1:00:00 --wrap="sleep infinity"
srun: srun --mpi=none --chdir=/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P40305 --ntasks=2 --jobid=302166 --cpus-per-task=8 --mem=76G
section_end:1645983675:prepare_executor section_start:1645983675:prepare_script Preparing environment
Running on r1i7n20 via bbpv1.epfl.ch...
section_end:1645983679:prepare_script section_start:1645983679:get_sources Getting source from Git repository
Skipping Git repository setup
Skipping Git checkout
Skipping Git submodules setup
section_end:1645983680:get_sources section_start:1645983680:restore_cache Restoring cache
Checking cache for build:coreneuron:nmodl:nvhpc:omp-8...
Runtime platform  arch=amd64 os=linux pid=203240 revision=58ba2b95 version=14.2.0
cache.zip is up to date 
Successfully extracted cache
section_end:1645983682:restore_cache section_start:1645983682:download_artifacts Downloading artifacts
Downloading artifacts for build:nmodl (169144)...
Runtime platform  arch=amd64 os=linux pid=203343 revision=58ba2b95 version=14.2.0
Downloading artifacts from coordinator... ok  id=169144 responseStatus=200 OK token=TRD26edM
section_end:1645983683:download_artifacts section_start:1645983683:step_script Executing "step_script" stage of the job script
WARNING: Starting with version 14.0 the 'build_script' stage will be replaced with 'step_script': https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26426
$ SPACK_PACKAGE_DEPENDENCIES="${SPACK_PACKAGE_DEPENDENCIES}^hpe-mpi%gcc"
$ SPACK_BUILD="${PWD}/spack-build"
$ env -0 | sort -z | xargs -0 -L 1 echo > initial_environment.env
$ . ${SPACK_ROOT}/share/spack/setup-env.sh
$ export SPACK_USER_CONFIG_PATH=${PWD}/spack-config
$ mkdir ${SPACK_USER_CONFIG_PATH}
$ cat > ${SPACK_USER_CONFIG_PATH}/config.yaml << END_SCRIPT # collapsed multi-line command
$ spack config blame config
--- config:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169147/spack-config/config.yaml:2 build_stage:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169147/spack-config/config.yaml:3 - /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169147/spack-build
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169147/spack-config/config.yaml:4 source_cache: /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169147/spack-source-cache
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/config.yaml:2 ccache: True
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:2 install_tree:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:3 root: $user_cache_path/software
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:4 projections:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:5 all: install_{compiler.name}-{compiler.version}-{target}/{name}-{version}-{hash:6}
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:6 module_roots:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:7 tcl: $user_cache_path/modules
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:9 environments:
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:10 root: $user_cache_path/environments
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:88 concretization: separately
/gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:12 build_jobs: 8
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:18 extensions:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:18 - $spack/bluebrain/spack-scripting
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:19 # This is the path to the root of the Spack install tree.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:19 # You can use $spack here to refer to the root of the spack instance.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:37 template_dirs:
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:37 - $spack/share/spack/templates
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Temporary locations Spack can try to use for builds.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Recommended options are given below.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Builds can be faster in temporary directories on some (e.g., HPC) systems.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Specifying `$tempdir` will ensure use of the default temporary directory
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # (i.e., ``$TMP` or ``$TMPDIR``).
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # Another option that prevents conflicts and potential permission issues is
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # to specify `$user_cache_path/stage`, which ensures each user builds in their
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # home directory.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # A more traditional path uses the value of `$spack/var/spack/stage`, which
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # builds directly inside Spack's instance without staging them in a
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # temporary space. Problems with specifying a path inside a Spack instance
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # are that it precludes its use as a system package and its ability to be
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # pip installable.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # In any case, if the username is not already in the path, Spack will append
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # the value of `$user` in an attempt to avoid potential conflicts between
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # users in shared temporary spaces.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 #
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # The build stage can be purged with `spack clean --stage` and
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # `spack clean -a`, so it is important that the specified directory uniquely
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:38 # identifies Spack staging to avoid accidentally wiping out non-Spack work.
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:73 test_stage: $user_cache_path/test
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:82 misc_cache: $user_cache_path/cache
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:94 connect_timeout: 10
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:99 verify_ssl: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:107 suppress_gpg_warnings: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:113 install_missing_compilers: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:118 checksum: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:123 deprecated: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:128 dirty: False
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:136 build_language: C
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:145 locks: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:150 url_fetch_method: urllib
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:176 concretizer: clingo
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:183 db_lock_timeout: 300
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:191 package_lock_timeout: null
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:197 shared_linking: rpath
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:202 allow_sgid: True
/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169142/spack/etc/spack/defaults/config.yaml:207 terminal_title: False
_builtin debug: False
$ export XDG_CONFIG_HOME=${PWD}/local_config
$ mkdir -p "${XDG_CONFIG_HOME}/git"
$ echo -e "[url \"https://gitlab-ci-token:${CI_JOB_TOKEN}@bbpgitlab.epfl.ch/\"]\n insteadOf = git@bbpgitlab.epfl.ch:" > "${XDG_CONFIG_HOME}/git/config"
$ cat "${XDG_CONFIG_HOME}/git/config"
[url "https://gitlab-ci-token:[MASKED]@bbpgitlab.epfl.ch/"]
insteadOf = git@bbpgitlab.epfl.ch:
$ SPACK_FULL_SPEC="${SPACK_PACKAGE}${SPACK_PACKAGE_COMPILER:+%}${SPACK_PACKAGE_COMPILER} ${SPACK_PACKAGE_SPEC} ${SPACK_PACKAGE_DEPENDENCIES} ${SPACK_PACKAGE_DEPENDENCY_ON_PREVIOUS_JOB}"
$ echo "Preparing to install ${SPACK_FULL_SPEC}"
Preparing to install coreneuron%nvhpc +nmodl+openmp+gpu+tests~legacy-unit~report~sympy build_type=RelWithDebInfo ^hpe-mpi%gcc ^/cnhmkfzsucqnmy6dfxyfikno2yj57grf
$ JSON_SPEC=$(spack spec --json ${SPACK_FULL_SPEC})
$ SPACK_INSTALLED_HASH=$(module load unstable python; echo "${JSON_SPEC}" | python -c "import json, sys; print(json.loads(sys.stdin.read())[\"spec\"][\"nodes\"][0][\"hash\"])")
$ echo "Determined its hash will be ${SPACK_INSTALLED_HASH}"
Determined its hash will be wdmfi5ouywy7yrclr3naov4e7v746rbq
$ SPACK_STAGE_DIR=${SPACK_BUILD}/spack-stage-${SPACK_PACKAGE}-develop-${SPACK_INSTALLED_HASH}
$ SPACK_BUILD_DIR=${SPACK_STAGE_DIR}/spack-build-${SPACK_INSTALLED_HASH:0:7}
$ SPACK_SOURCE_DIR=${SPACK_STAGE_DIR}/spack-src
$ if [ ${SPACK_USE_CCACHE+x} ]; then
$ module load unstable ccache
$ export CCACHE_BASEDIR=$(realpath -P ${CI_BUILDS_DIR})
$ echo CCACHE_BASEDIR=${CCACHE_BASEDIR}
CCACHE_BASEDIR=/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305
$ export CCACHE_MAXSIZE=512M
$ export CCACHE_DIR="${TMPDIR}/ccache"
$ mkdir -p ${CCACHE_DIR}
$ if [ -f ${CI_PROJECT_DIR}/ccache.tar ]; then
$ tar -C "${CCACHE_DIR}" -xf "${CI_PROJECT_DIR}/ccache.tar"
$ fi
$ ccache --zero-stats
Statistics zeroed
$ ccache --show-stats --verbose
Summary:
Cache directory: /nvme/bbpcihpcproj12/302166/ccache
Primary config: /nvme/bbpcihpcproj12/302166/ccache/ccache.conf
Secondary config: /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/ccache-4.4.2-6qbndl/etc/ccache.conf
Stats updated: Sun Feb 27 18:41:47 2022
Hits: 0 / 0
Direct: 0 / 0
Preprocessed: 0 / 0
Misses: 0
Direct: 0
Preprocessed: 0
Primary storage:
Hits: 0 / 0
Misses: 0
Cache size (GB): 0.20 / 0.51 (39.33 %)
Files: 7245
$ fi
$ module load unstable git ${SPACK_EXTRA_MODULES}
$ spack uninstall -y --dependents /${SPACK_INSTALLED_HASH} || true
==> Error: No installed spec matches the hash: 'wdmfi5ouywy7yrclr3naov4e7v746rbq'
$ spack spec -Il ${SPACK_FULL_SPEC}
Input spec
--------------------------------
- coreneuron%nvhpc+gpu~legacy-unit+nmodl+openmp~report~sympy+tests build_type=RelWithDebInfo
- ^hpe-mpi%gcc
[+] ^nmodl@develop%gcc@11.2.0~ipo~legacy-unit~python build_type=RelWithDebInfo arch=linux-rhel7-skylake
[^] ^py-jinja2@3.0.1%gcc@11.2.0~i18n arch=linux-rhel7-skylake
[^] ^py-markupsafe@2.0.1%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^python@3.9.7%gcc@11.2.0+bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93189bc278fbc37a50ed7f183bd8aaf249a8e1670a465f0db6bb4f8cf87,4c2457325f2b608b1b6a2c63087df8c26e07db3e3d493caf36a56f0ecf6fb768,f2fd060afc4b4618fe8104c4c5d771f36dc55b1db5a4623785a4ea707ec72fb4 arch=linux-rhel7-skylake
[^] ^py-pytest@6.2.4%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-attrs@21.2.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-iniconfig@1.1.1%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-packaging@21.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-pyparsing@2.4.7%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-pluggy@0.13.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-py@1.9.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-setuptools@57.4.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-toml@0.10.2%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-pyyaml@5.3.1%gcc@11.2.0+libyaml arch=linux-rhel7-skylake
[^] ^libyaml@0.2.5%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-sympy@1.8%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ^py-mpmath@1.1.0%gcc@11.2.0 arch=linux-rhel7-skylake
Concretized
--------------------------------
- wdmfi5o coreneuron@1.0.0.20220218%nvhpc@22.2~caliper~codegenopt+gpu~ipo~ispc~knl~legacy-unit+mpi+nmodl+openmp~profile~report+shared~sympy~sympyopt+tests~unified build_type=RelWithDebInfo arch=linux-rhel7-skylake
- ocv5kkj ^bison@3.8.2%nvhpc@22.2 arch=linux-rhel7-skylake
[^] bfwiaoz ^boost@1.78.0%nvhpc@22.2+atomic+chrono~clanglibcpp~container~context~coroutine+date_time~debug+exception~fiber+filesystem+graph~icu+iostreams+locale+log+math~mpi+multithreaded~numpy+pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave cxxstd=98 visibility=hidden arch=linux-rhel7-skylake
- 55wtnz6 ^cmake@3.21.4%nvhpc@22.2~doc+ncurses+openssl+ownlibs~qt build_type=Release arch=linux-rhel7-skylake
[^] wd5k5wi ^cuda@11.6.0%nvhpc@22.2~dev arch=linux-rhel7-skylake
- xzdnug7 ^flex@2.6.3%nvhpc@22.2+lex~nls arch=linux-rhel7-skylake
[^] hyunzkn ^hpe-mpi@2.25.hmpt%gcc@11.2.0 arch=linux-rhel7-skylake
- ij6isvr ^ninja@1.10.2%nvhpc@22.2 arch=linux-rhel7-skylake
[+] cnhmkfz ^nmodl@develop%gcc@11.2.0~ipo~legacy-unit~python build_type=RelWithDebInfo arch=linux-rhel7-skylake
[^] onq3mhg ^py-jinja2@3.0.1%gcc@11.2.0~i18n arch=linux-rhel7-skylake
[^] 4ectow5 ^py-markupsafe@2.0.1%gcc@11.2.0 arch=linux-rhel7-skylake
[^] 72xzp3v ^python@3.9.7%gcc@11.2.0+bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93189bc278fbc37a50ed7f183bd8aaf249a8e1670a465f0db6bb4f8cf87,4c2457325f2b608b1b6a2c63087df8c26e07db3e3d493caf36a56f0ecf6fb768,f2fd060afc4b4618fe8104c4c5d771f36dc55b1db5a4623785a4ea707ec72fb4 arch=linux-rhel7-skylake
[^] 22arfs4 ^py-pytest@6.2.4%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ascbeii ^py-attrs@21.2.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] kvw3vhm ^py-iniconfig@1.1.1%gcc@11.2.0 arch=linux-rhel7-skylake
[^] 7iyiygo ^py-packaging@21.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] c7qvw2q ^py-pyparsing@2.4.7%gcc@11.2.0 arch=linux-rhel7-skylake
[^] mazoiox ^py-pluggy@0.13.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] y7rfzdj ^py-py@1.9.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] v4z3s5e ^py-setuptools@57.4.0%gcc@11.2.0 arch=linux-rhel7-skylake
[^] w4gddqx ^py-toml@0.10.2%gcc@11.2.0 arch=linux-rhel7-skylake
[^] ci5oe5b ^py-pyyaml@5.3.1%gcc@11.2.0+libyaml arch=linux-rhel7-skylake
[^] xj2wlac ^libyaml@0.2.5%gcc@11.2.0 arch=linux-rhel7-skylake
[^] xtett6q ^py-sympy@1.8%gcc@11.2.0 arch=linux-rhel7-skylake
[^] dzb2mfs ^py-mpmath@1.1.0%gcc@11.2.0 arch=linux-rhel7-skylake
$ spack ${SPACK_INSTALL_EXTRA_FLAGS} install -j${SLURM_CPUS_PER_TASK} --log-format=junit --log-file=${CI_PROJECT_DIR}/install.xml --keep-stage ${SPACK_FULL_SPEC} || install_failed=1
==> bison@3.8.2 : has external module in ['bison/3.8.2']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/bison-3.8.2-ej2cew (external bison-3.8.2-ocv5kkjjvqshvbpp4gh3xrg4aj3xxh2k)
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/boost-1.78.0-hiorc7 (external boost-1.78.0-bfwiaozbrpjr7blzjkfcz67vabdnspxj)
==> cmake@3.21.4 : has external module in ['cmake/3.21.4']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/cmake-3.21.4-cdyb7k (external cmake-3.21.4-55wtnz6tfxjhgdgfbzzwzuhyl746wv7j)
==> cuda@11.6.0 : has external module in ['cuda/11.6.0']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/cuda-11.6.0-34qjta (external cuda-11.6.0-wd5k5wioelahwux72y3g3bytolebtkoh)
==> flex@2.6.3 : has external module in ['flex/2.6.3']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/flex-2.6.3-3mtl7j (external flex-2.6.3-xzdnug7pkmqiem4ll6p4chgbnwyt7epd)
==> hpe-mpi@2.25.hmpt : has external module in ['hpe-mpi/2.25.hmpt']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/hpe-mpi-2.25.hmpt-4ukyxt (external hpe-mpi-2.25.hmpt-hyunzknktdzri34bx26bupkvapehonql)
==> ninja@1.10.2 : has external module in ['ninja/1.10.2']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/ninja-1.10.2-eh33rh (external ninja-1.10.2-ij6isvr7fpc6nbooavdi3abtviuzpngb)
==> python@3.9.7 : has external module in ['python/3.9.7']
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/python-3.9.7-yj5alh (external python-3.9.7-72xzp3vgzh5b424qevkfruhs3ajzqbiy)
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/libyaml-0.2.5-xj2wla
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-iniconfig-1.1.1-kvw3vh
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-mpmath-1.1.0-dzb2mf
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-toml-0.10.2-w4gddq
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-py-1.9.0-y7rfzd
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-setuptools-57.4.0-v4z3s5
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-markupsafe-2.0.1-4ectow
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-pyparsing-2.4.7-c7qvw2
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-attrs-21.2.0-ascbei
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-pluggy-0.13.0-mazoio
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-pyyaml-5.3.1-ci5oe5
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-sympy-1.8-xtett6
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-jinja2-3.0.1-onq3mh
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-packaging-21.0-7iyiyg
[+] /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_applications/install_gcc-11.2.0-skylake/py-pytest-6.2.4-22arfs
[+] /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/software/install_gcc-11.2.0-skylake/nmodl-develop-cnhmkf
==> Installing coreneuron-1.0.0.20220218-wdmfi5ouywy7yrclr3naov4e7v746rbq
==> No binary for coreneuron-1.0.0.20220218-wdmfi5ouywy7yrclr3naov4e7v746rbq found: installing from source
==> No patches needed for coreneuron
==> coreneuron: Executing phase: 'cmake'
==> coreneuron: Executing phase: 'build'
==> coreneuron: Executing phase: 'install'
==> coreneuron: Successfully installed coreneuron-1.0.0.20220218-wdmfi5ouywy7yrclr3naov4e7v746rbq
Fetch: 4.30s. Build: 5m 0.38s. Total: 5m 4.68s.
[+] /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/software/install_nvhpc-22.2-skylake/coreneuron-1.0.0.20220218-wdmfi5
$ chmod -R g+rX "${SPACK_BUILD}"
$ if [[ ${install_failed} == 1 ]]; then exit 1; fi
$ if [ ${SPACK_USE_CCACHE+x} ]; then
$ ccache --cleanup
$ ccache --show-stats --verbose
Summary:
Cache directory: /nvme/bbpcihpcproj12/302166/ccache
Primary config: /nvme/bbpcihpcproj12/302166/ccache/ccache.conf
Secondary config: /gpfs/bbp.cscs.ch/ssd/apps/bsd/2022-01-10/stage_externals/install_gcc-11.2.0-skylake/ccache-4.4.2-6qbndl/etc/ccache.conf
Stats updated: Sun Feb 27 18:47:39 2022
Hits: 5 / 97 (5.15 %)
Direct: 0 / 97 (0.00 %)
Preprocessed: 5 / 97 (5.15 %)
Misses: 92
Direct: 97
Preprocessed: 92
Uncacheable: 17
Primary storage:
Hits: 5 / 194 (2.58 %)
Misses: 189
Cache size (GB): 0.21 / 0.51 (41.05 %)
Files: 7434
Uncacheable:
Called for linking: 14
No input file: 3
$ tar -C "${CCACHE_DIR}" -cf "${CI_PROJECT_DIR}/ccache.tar" .
$ fi
$ touch ${SPACK_STAGE_DIR}/spack-configure-args.txt
touch: cannot touch ‘/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P40305/J169147/spack-build/spack-stage-coreneuron-develop-wdmfi5ouywy7yrclr3naov4e7v746rbq/spack-configure-args.txt’: No such file or directory
srun: error: r1i7n20: task 0: Exited with exit code 1
section_end:1645984059:step_script section_start:1645984059:upload_artifacts_on_failure Uploading artifacts for failed job
Uploading artifacts...
Runtime platform  arch=amd64 os=linux pid=210989 revision=58ba2b95 version=14.2.0
install.xml: found 1 matching files and directories
spack_build_info.env: found 1 matching files and directories
spack_clone_variables.env: found 1 matching files and directories
initial_environment.env: found 1 matching files and directories
spack-build-env.txt: found 1 matching files and directories
spack-build-out.txt: found 1 matching files and directories
spack-configure-args.txt: found 1 matching files and directories
Uploading artifacts as "archive" to coordinator... ok id=169147 responseStatus=201 Created token=RJzBwx5z
Uploading artifacts...
Runtime platform  arch=amd64 os=linux pid=211028 revision=58ba2b95 version=14.2.0
install.xml: found 1 matching files and directories
Uploading artifacts as "junit" to coordinator... ok id=169147 responseStatus=201 Created token=RJzBwx5z
Uploading artifacts...
Runtime platform  arch=amd64 os=linux pid=211069 revision=58ba2b95 version=14.2.0
spack_build_info.env: found 1 matching files and directories
spack_clone_variables.env: found 1 matching files and directories
Uploading artifacts as "dotenv" to coordinator... ok id=169147 responseStatus=201 Created token=RJzBwx5z
section_end:1645984061:upload_artifacts_on_failure section_start:1645984061:cleanup_file_variables Cleaning up project directory and file based variables
section_end:1645984061:cleanup_file_variables ERROR: Job failed: exit status 1

Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Log was not fetched because job had status: skipped
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment