Skip to content

Instantly share code, notes, and snippets.

@douglatornell
Created June 23, 2021 16:28
Show Gist options
  • Save douglatornell/f5821b42c0153494592133010c6f8a7c to your computer and use it in GitHub Desktop.
Save douglatornell/f5821b42c0153494592133010c6f8a7c to your computer and use it in GitHub Desktop.
Script to do HDF5 to netCDF4 conversion of MOHID results in MIDOSS project Monte-Carlo results tree. Created for post-processing of the `north_strait_5th362` batch of runs for which the in-job conversion failed because the `SLURM_TMPDIR` local scratch storage on the node filled.
#!/bin/bash
RESULTS_DIR="/scratch/dlatorne/MIDOSS/runs/monte-carlo/north_strait_5th362_2021-06-14T150753/results/north_strait_5th362"
HDF5_TO_NETCDF4="${HOME}/.local/bin/hdf5-to-netcdf4"
module load python/3.8.2
module load nco/4.9.5
TMPDIR="${SLURM_TMPDIR}"
for n in {0..361}
do
if test -f ${RESULTS_DIR}-${n}/Lagrangian*.hdf5
then
LAGRANGIAN=$(find ${RESULTS_DIR}-${n} -type f -name Lagrangian*.hdf5 | xargs basename -s .hdf5)
echo ${LAGRANGIAN}
echo "Results hdf5 to netCDF4 conversion started at $(date)"
cp -v ${RESULTS_DIR}-${n}/${LAGRANGIAN}.hdf5 ${SLURM_TMPDIR}/ && \
${HDF5_TO_NETCDF4} -v info \
${SLURM_TMPDIR}/${LAGRANGIAN}.hdf5 \
${SLURM_TMPDIR}/${LAGRANGIAN}.nc && \
mv -v ${SLURM_TMPDIR}/${LAGRANGIAN}.nc ${RESULTS_DIR}-${n}/
rm -v ${RESULTS_DIR}-${n}/${LAGRANGIAN}.hdf5
echo "Results hdf5 to netCDF4 conversion ended at $(date)"
fi
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment