MPI.jl working

45886644_1_rank.0_stderr

45886644_1_rank.0_stdout

Hello, world, I am rank 0 of 6.

45886644_1_rank.1_stderr

45886644_1_rank.1_stdout

Hello, world, I am rank 1 of 6.

45886644_1_rank.2_stderr

45886644_1_rank.2_stdout

Hello, world, I am rank 2 of 6.

45886644_1_rank.3_stderr

45886644_1_rank.3_stdout

Hello, world, I am rank 3 of 6.

45886644_1_rank.4_stderr

45886644_1_rank.4_stdout

Hello, world, I am rank 4 of 6.

45886644_1_rank.5_stderr

45886644_1_rank.5_stdout

Hello, world, I am rank 5 of 6.

hello.jl

using MPI
MPI.Init()

comm = MPI.COMM_WORLD
println("Hello, world, I am rank $(MPI.Comm_rank(comm)) of $(MPI.Comm_size(comm)).")

mpi-job.sh

#!/bin/bash
#SBATCH --time=0:10:00
#SBATCH --nodes=2            # node count
#SBATCH --ntasks-per-node=3  # number of tasks per node
#SBATCH --cpus-per-task=1    # cpu-cores per task
#SBATCH --mem=2G
#SBATCH --output=output-%j.txt

source /etc/profile
module load sdk/nvidia/22.7

# have the shell echo the commands that are run
set -x

export JULIA_NUM_THREADS=1
julia --project -e 'using Pkg; pkg"instantiate"'
julia --project -e 'using Pkg; pkg"precompile"'
mpiexec --mca mpi_cuda_support 0 -output-filename $SLURM_JOB_ID julia hello.jl

#mpiexec --mca mpi_cuda_support 0 -output-filename $SLURM_JOB_ID ./hello

exit 0

output-45886644.txt

+ export JULIA_NUM_THREADS=1
+ JULIA_NUM_THREADS=1
+ julia --project -e 'using Pkg; pkg"instantiate"'
┌ Warning: The Pkg REPL mode is intended for interactive use only, and should not be used from scripts. It is recommended to use the functional API instead.
└ @ Pkg.REPLMode /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/Pkg/src/REPLMode/REPLMode.jl:379
+ julia --project -e 'using Pkg; pkg"precompile"'
┌ Warning: The Pkg REPL mode is intended for interactive use only, and should not be used from scripts. It is recommended to use the functional API instead.
└ @ Pkg.REPLMode /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/Pkg/src/REPLMode/REPLMode.jl:379
+ mpiexec --mca mpi_cuda_support 0 -output-filename 45886644 julia hello.jl
--------------------------------------------------------------------------
WARNING: No preset parameters were found for the device that Open MPI
detected:

  Local host:            compute-2-30
  Device name:           mlx5_0
  Device vendor ID:      0x02c9
  Device vendor part ID: 4123

Default device parameters will be used, which may result in lower
performance.  You can edit any of the files specified by the
btl_openib_device_param_files MCA parameter to set values for your
device.

NOTE: You can turn off this warning by setting the MCA parameter
      btl_openib_warn_no_device_params_found to 0.
--------------------------------------------------------------------------
Hello, world, I am rank 2 of 6.
Hello, world, I am rank 3 of 6.
Hello, world, I am rank 4 of 6.Hello, world, I am rank 5 of 6.

Hello, world, I am rank 0 of 6.
Hello, world, I am rank 1 of 6.
[compute-2-30.hamming.cluster:55987] 5 more processes have sent help message help-mpi-btl-openib.txt / no device params found
[compute-2-30.hamming.cluster:55987] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
+ exit 0