Skip to content

Instantly share code, notes, and snippets.

@vivek-bala

vivek-bala/jsrun_10_2_runs_summitdev.md

Created December 20, 2018 16:20
Show Gist options
  • Save vivek-bala/e12a13057ad8ee37ffb3a718affeaf9d to your computer and use it in GitHub Desktop.
Save vivek-bala/e12a13057ad8ee37ffb3a718affeaf9d to your computer and use it in GitHub Desktop.
Download ZIP
Raw
jsrun_10_2_runs_summitdev.md

OpenMP (env variables)=4, smt level=4:

#BSUB -alloc_flags 'gpumps smt4' 
$ export  OMP_NUM_THREADS=4
$ jsrun -n 1 -a 1 -c 4 -g 1 /ccs/proj/csc190/Hello_jsrun/jsrun_layout
*** MPI Ranks: 1, OpenMP Threads: 4, GPUs per Resource Set: 1 ***
========================================================================
MPI Rank 000, OMP_thread 00 on HWThread 001 of Node summitdev-r0c1n03 - RT_GPU_id 0 : GPU_id 0
MPI Rank 000, OMP_thread 01 on HWThread 002 of Node summitdev-r0c1n03 - RT_GPU_id 0 : GPU_id 0
MPI Rank 000, OMP_thread 02 on HWThread 003 of Node summitdev-r0c1n03 - RT_GPU_id 0 : GPU_id 0
MPI Rank 000, OMP_thread 03 on HWThread 000 of Node summitdev-r0c1n03 - RT_GPU_id 0 : GPU_id 0

#BSUB -alloc_flags 'gpumps smt4' 
$ export  OMP_NUM_THREADS=4
$ jsrun -n 1 -a 1 /ccs/proj/csc190/Hello_jsrun/jsrun_layout
*** MPI Ranks: 1, OpenMP Threads: 4, GPUs per Resource Set: 0 ***
========================================================================
MPI Rank 000 of 001 on HWThread 003 of Node summitdev-r0c1n03, OMP_threadID 2 of 4
MPI Rank 000 of 001 on HWThread 001 of Node summitdev-r0c1n03, OMP_threadID 0 of 4
MPI Rank 000 of 001 on HWThread 002 of Node summitdev-r0c1n03, OMP_threadID 1 of 4
MPI Rank 000 of 001 on HWThread 000 of Node summitdev-r0c1n03, OMP_threadID 3 of 4

#BSUB -alloc_flags 'gpumps smt4' 
$ export  OMP_NUM_THREADS=4
$ jsrun -n 1 -a 2 /ccs/proj/csc190/Hello_jsrun/jsrun_layout
*** MPI Ranks: 2, OpenMP Threads: 4, GPUs per Resource Set: 0 ***
========================================================================
MPI Rank 000 of 002 on HWThread 001 of Node summitdev-r0c2n15, OMP_threadID 0 of 4
MPI Rank 000 of 002 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 2 of 4
MPI Rank 000 of 002 on HWThread 003 of Node summitdev-r0c2n15, OMP_threadID 1 of 4
MPI Rank 000 of 002 on HWThread 001 of Node summitdev-r0c2n15, OMP_threadID 3 of 4
MPI Rank 001 of 002 on HWThread 002 of Node summitdev-r0c2n15, OMP_threadID 1 of 4
MPI Rank 001 of 002 on HWThread 001 of Node summitdev-r0c2n15, OMP_threadID 0 of 4
MPI Rank 001 of 002 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 2 of 4
MPI Rank 001 of 002 on HWThread 003 of Node summitdev-r0c2n15, OMP_threadID 3 of 4

OpenMP=4, smt level=1:

#BSUB -alloc_flags 'gpumps smt1' 
$ export  OMP_NUM_THREADS=4
$ jsrun -n 1 -a 1 -c 4 -g 1 /ccs/proj/csc190/Hello_jsrun/jsrun_layout
*** MPI Ranks: 1, OpenMP Threads: 4, GPUs per Resource Set: 1 ***
========================================================================
MPI Rank 000, OMP_thread 03 on HWThread 000 of Node summitdev-r0c2n15 - RT_GPU_id 0 : GPU_id 0 
MPI Rank 000, OMP_thread 02 on HWThread 000 of Node summitdev-r0c2n15 - RT_GPU_id 0 : GPU_id 0 
MPI Rank 000, OMP_thread 01 on HWThread 000 of Node summitdev-r0c2n15 - RT_GPU_id 0 : GPU_id 0 
MPI Rank 000, OMP_thread 00 on HWThread 000 of Node summitdev-r0c2n15 - RT_GPU_id 0 : GPU_id 0 

#BSUB -alloc_flags 'gpumps smt1' 
$ export  OMP_NUM_THREADS=4
$ jsrun -n 1 -a 1 /ccs/proj/csc190/Hello_jsrun/jsrun_layout
*** MPI Ranks: 1, OpenMP Threads: 4, GPUs per Resource Set: 0 ***
========================================================================
MPI Rank 000 of 001 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 0 of 4
MPI Rank 000 of 001 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 3 of 4
MPI Rank 000 of 001 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 2 of 4
MPI Rank 000 of 001 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 1 of 4

#BSUB -alloc_flags 'gpumps smt1' 
$ export  OMP_NUM_THREADS=4
$ jsrun -n 1 -a 2 /ccs/proj/csc190/Hello_jsrun/jsrun_layout
*** MPI Ranks: 2, OpenMP Threads: 4, GPUs per Resource Set: 0 ***
========================================================================
MPI Rank 000 of 002 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 0 of 4
MPI Rank 000 of 002 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 3 of 4
MPI Rank 000 of 002 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 2 of 4
MPI Rank 000 of 002 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 1 of 4
MPI Rank 001 of 002 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 0 of 4
MPI Rank 001 of 002 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 3 of 4
MPI Rank 001 of 002 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 2 of 4
MPI Rank 001 of 002 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 1 of 4

OpenMP=1, smt level=1:

#BSUB -alloc_flags 'gpumps smt1' 
$ export  OMP_NUM_THREADS=1
$ jsrun -n 1 -a 1 -c 1 -g 1 /ccs/proj/csc190/Hello_jsrun/jsrun_layout
########################################################################

*** MPI Ranks: 1, OpenMP Threads: 1, GPUs per Resource Set: 1 ***
========================================================================
MPI Rank 000, OMP_thread 00 on HWThread 000 of Node summitdev-r0c2n15 - RT_GPU_id 0 : GPU_id 0 

#BSUB -alloc_flags 'gpumps smt1' 
$ export  OMP_NUM_THREADS=1
$ jsrun -n 1 -a 1 /ccs/proj/csc190/Hello_jsrun/jsrun_layout
########################################################################

*** MPI Ranks: 1, OpenMP Threads: 1, GPUs per Resource Set: 0 ***
========================================================================
MPI Rank 000 of 001 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 0 of 1

#BSUB -alloc_flags 'gpumps smt1' 
$ export  OMP_NUM_THREADS=1
$ jsrun -n 1 -a 2 /ccs/proj/csc190/Hello_jsrun/jsrun_layout
########################################################################

########################################################################

*** MPI Ranks: 2, OpenMP Threads: 1, GPUs per Resource Set: 0 ***
========================================================================
MPI Rank 000 of 002 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 0 of 1
MPI Rank 001 of 002 on HWThread 000 of Node summitdev-r0c2n15, OMP_threadID 0 of 1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment