lcw/45886605_1_rank.00_stderr

## 45886605_1_rank.00_stderr
[compute-2-30.hamming.cluster:40843] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40843] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40843] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40843] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40843] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40843] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40843] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.00_stdout

      
    Raw
  

              45886605_1_rank.00_stdout
            
          
## 45886605_1_rank.01_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_setmark_pool_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:803 [inlined]
gc_setmark_pool at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:827 [inlined]
gc_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:834 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2771
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7f6b56df5c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7f6b56df4fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965
eval at ./boot.jl:368 [inlined]
include_string at ./loading.jl:1428
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559

## 45886605_1_rank.01_stdout

      
    Raw
  

              45886605_1_rank.01_stdout
            
          
## 45886605_1_rank.02_stderr
[compute-2-30.hamming.cluster:40846] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40846] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40846] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40846] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40846] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40846] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40846] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.02_stdout

      
    Raw
  

              45886605_1_rank.02_stdout
            
          
## 45886605_1_rank.03_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
jl_gc_premark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3025 [inlined]
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3097
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7ff2e4611c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7ff2e4610fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200

## 45886605_1_rank.03_stdout

      
    Raw
  

              45886605_1_rank.03_stdout
            
          
## 45886605_1_rank.04_stderr
[compute-2-30.hamming.cluster:40850] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40850] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40850] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40850] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40850] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40850] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40850] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.04_stdout

      
    Raw
  

              45886605_1_rank.04_stdout
            
          
## 45886605_1_rank.05_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
jl_gc_premark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3025 [inlined]
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3097
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7f8026b07c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7f8026b06fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965

## 45886605_1_rank.05_stdout

      
    Raw
  

              45886605_1_rank.05_stdout
            
          
## 45886605_1_rank.06_stderr
[compute-2-30.hamming.cluster:40854] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40854] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40854] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40854] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40854] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40854] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40854] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.06_stdout

      
    Raw
  

              45886605_1_rank.06_stdout
            
          
## 45886605_1_rank.07_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
jl_gc_premark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3025 [inlined]
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3097
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7f8663f20c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7f8663f1ffcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144

## 45886605_1_rank.07_stdout

      
    Raw
  

              45886605_1_rank.07_stdout
            
          
## 45886605_1_rank.08_stderr
[compute-2-30.hamming.cluster:40858] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40858] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40858] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40858] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40858] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40858] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40858] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.08_stdout

      
    Raw
  

              45886605_1_rank.08_stdout
            
          
## 45886605_1_rank.09_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_try_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1764 [inlined]
gc_mark_scan_obj8 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2006 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2302
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:478 [inlined]
vect at ./array.jl:125 [inlined]
finish at ./compiler/typeinfer.jl:446
_typeinf at ./compiler/typeinfer.jl:243
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fc2195d7c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fc2195d6fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144

## 45886605_1_rank.09_stdout

      
    Raw
  

              45886605_1_rank.09_stdout
            
          
## 45886605_1_rank.10_stderr
[compute-2-30.hamming.cluster:40862] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40862] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40862] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40862] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40862] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40862] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40862] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.10_stdout

      
    Raw
  

              45886605_1_rank.10_stdout
            
          
## 45886605_1_rank.11_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_try_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1764 [inlined]
gc_mark_scan_obj8 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2006 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2302
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fe4e7cf1c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fe4e7cf0fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144

## 45886605_1_rank.11_stdout

      
    Raw
  

              45886605_1_rank.11_stdout
            
          
## 45886605_1_rank.12_stderr
[compute-2-30.hamming.cluster:40866] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40866] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40866] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40866] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40866] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40866] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40866] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.12_stdout

      
    Raw
  

              45886605_1_rank.12_stdout
            
          
## 45886605_1_rank.13_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
jl_gc_premark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3025 [inlined]
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3097
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fa2f5fb1c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fa2f5fb0fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965
eval at ./boot.jl:368 [inlined]
include_string at ./loading.jl:1428
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559

## 45886605_1_rank.13_stdout

      
    Raw
  

              45886605_1_rank.13_stdout
            
          
## 45886605_1_rank.14_stderr
[compute-2-30.hamming.cluster:40870] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40870] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40870] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40870] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40870] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40870] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40870] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.14_stdout

      
    Raw
  

              45886605_1_rank.14_stdout
            
          
## 45886605_1_rank.15_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_try_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1764 [inlined]
gc_mark_scan_objarray at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1876 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2267
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:473 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:192
IncrementalCompact at ./compiler/ssair/ir.jl:574
batch_inline! at ./compiler/ssair/inlining.jl:585
ssa_inlining_pass! at ./compiler/ssair/inlining.jl:85
jfptr_ssa_inlining_passNOT._13336.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
run_passes at ./compiler/optimize.jl:539
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:868
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:835
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:148
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fad21230c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fad2122ffcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144

## 45886605_1_rank.15_stdout

      
    Raw
  

              45886605_1_rank.15_stdout
            
          
## 45886605_1_rank.16_stderr
[compute-2-30.hamming.cluster:40874] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40874] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40874] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40874] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40874] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40874] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40874] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.16_stdout

      
    Raw
  

              45886605_1_rank.16_stdout
            
          
## 45886605_1_rank.17_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_try_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1764 [inlined]
gc_mark_scan_obj8 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2006 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2302
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fafb11a0c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fafb119ffcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200

## 45886605_1_rank.17_stdout

      
    Raw
  

              45886605_1_rank.17_stdout
            
          
## 45886605_1_rank.18_stderr
[compute-2-30.hamming.cluster:40878] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40878] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40878] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40878] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40878] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40878] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40878] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.18_stdout

      
    Raw
  

              45886605_1_rank.18_stdout
            
          
## 45886605_1_rank.19_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_try_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1764 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2475
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7f0af9213c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7f0af9212fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965
eval at ./boot.jl:368 [inlined]
include_string at ./loading.jl:1428
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559

## 45886605_1_rank.19_stdout

      
    Raw
  

              45886605_1_rank.19_stdout
            
          
## 45886605_1_rank.20_stderr
[compute-2-30.hamming.cluster:40883] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40883] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40883] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40883] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40883] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40883] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40883] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.20_stdout

      
    Raw
  

              45886605_1_rank.20_stdout
            
          
## 45886605_1_rank.21_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
jl_gc_premark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3025 [inlined]
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3097
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
jl_gc_alloc_buf at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:403 [inlined]
jl_gc_alloc_buf at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:401 [inlined]
array_resize_buffer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:698
jl_array_grow_at_end at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:893
_growend! at ./array.jl:1011 [inlined]
resize! at ./array.jl:1236 [inlined]
resize! at ./compiler/ssair/ir.jl:1142
batch_inline! at ./compiler/ssair/inlining.jl:596
ssa_inlining_pass! at ./compiler/ssair/inlining.jl:85
jfptr_ssa_inlining_passNOT._13336.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
run_passes at ./compiler/optimize.jl:539
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:868
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:835
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:148
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fa26f760c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fa26f75ffcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200

## 45886605_1_rank.21_stdout

      
    Raw
  

              45886605_1_rank.21_stdout
            
          
## 45886605_1_rank.22_stderr
[compute-2-30.hamming.cluster:40887] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40887] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40887] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40887] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40887] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40887] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40887] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.22_stdout

      
    Raw
  

              45886605_1_rank.22_stdout
            
          
## 45886605_1_rank.23_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
reset_page at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1209 [inlined]
sweep_page at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1342 [inlined]
sweep_pool_page at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1447 [inlined]
sweep_pool_pagetable0 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1467 [inlined]
sweep_pool_pagetable1 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1487 [inlined]
sweep_pool_pagetable at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1517 [inlined]
gc_sweep_pool at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1592
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3243
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7f9261f72c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7f9261f71fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965

## 45886605_1_rank.23_stdout

      
    Raw
  

              45886605_1_rank.23_stdout
            
          
## 45886605_1_rank.24_stderr
[compute-2-30.hamming.cluster:40891] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40891] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40891] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40891] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40891] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40891] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40891] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.24_stdout

      
    Raw
  

              45886605_1_rank.24_stdout
            
          
## 45886605_1_rank.25_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2592
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fadd8432c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fadd8431fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144

## 45886605_1_rank.25_stdout

      
    Raw
  

              45886605_1_rank.25_stdout
            
          
## 45886605_1_rank.26_stderr
[compute-2-30.hamming.cluster:40895] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40895] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40895] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40895] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40895] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40895] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40895] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.26_stdout

      
    Raw
  

              45886605_1_rank.26_stdout
            
          
## 45886605_1_rank.27_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_try_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1764 [inlined]
gc_mark_scan_obj8 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2006 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2302
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7f520045fc43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7f520045efcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200

## 45886605_1_rank.27_stdout

      
    Raw
  

              45886605_1_rank.27_stdout
            
          
## 45886605_1_rank.28_stderr
[compute-2-30.hamming.cluster:40899] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40899] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40899] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40899] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40899] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40899] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40899] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.28_stdout

      
    Raw
  

              45886605_1_rank.28_stdout
            
          
## 45886605_1_rank.29_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
jl_gc_premark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3025 [inlined]
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3097
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fb514a1fc43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fb514a1efcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965

## 45886605_1_rank.29_stdout

      
    Raw
  

              45886605_1_rank.29_stdout
            
          
## 45886605_1_rank.30_stderr
[compute-2-30.hamming.cluster:40903] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40903] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40903] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40903] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40903] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40903] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40903] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

## 45886605_1_rank.30_stdout

      
    Raw
  

              45886605_1_rank.30_stdout
            
          
## 45886605_1_rank.31_stderr

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_setmark_pool_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:803 [inlined]
gc_setmark_pool at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:827 [inlined]
gc_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:834 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2781
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:478 [inlined]
vect at ./array.jl:125 [inlined]
finish at ./compiler/typeinfer.jl:446
_typeinf at ./compiler/typeinfer.jl:243
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fd6f796ec43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fd6f796dfcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965
eval at ./boot.jl:368 [inlined]
include_string at ./loading.jl:1428
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559

## 45886605_1_rank.31_stdout

      
    Raw
  

              45886605_1_rank.31_stdout
            
          
## 45886605_1_rank.32_stderr

      
    Raw
  

              45886605_1_rank.32_stderr
            
          
## 45886605_1_rank.32_stdout

      
    Raw
  

              45886605_1_rank.32_stdout
            
          
## 45886605_1_rank.33_stderr

      
    Raw
  

              45886605_1_rank.33_stderr
            
          
## 45886605_1_rank.33_stdout

      
    Raw
  

              45886605_1_rank.33_stdout
            
          
## 45886605_1_rank.34_stderr

      
    Raw
  

              45886605_1_rank.34_stderr
            
          
## 45886605_1_rank.34_stdout

      
    Raw
  

              45886605_1_rank.34_stdout
            
          
## 45886605_1_rank.35_stderr

      
    Raw
  

              45886605_1_rank.35_stderr
            
          
## 45886605_1_rank.35_stdout

      
    Raw
  

              45886605_1_rank.35_stdout
            
          
## 45886605_1_rank.36_stderr

      
    Raw
  

              45886605_1_rank.36_stderr
            
          
## 45886605_1_rank.36_stdout

      
    Raw
  

              45886605_1_rank.36_stdout
            
          
## 45886605_1_rank.37_stderr

      
    Raw
  

              45886605_1_rank.37_stderr
            
          
## 45886605_1_rank.37_stdout

      
    Raw
  

              45886605_1_rank.37_stdout
            
          
## 45886605_1_rank.38_stderr

      
    Raw
  

              45886605_1_rank.38_stderr
            
          
## 45886605_1_rank.38_stdout

      
    Raw
  

              45886605_1_rank.38_stdout
            
          
## 45886605_1_rank.39_stderr

      
    Raw
  

              45886605_1_rank.39_stderr
            
          
## 45886605_1_rank.39_stdout

      
    Raw
  

              45886605_1_rank.39_stdout
            
          
## 45886605_1_rank.40_stderr

      
    Raw
  

              45886605_1_rank.40_stderr
            
          
## 45886605_1_rank.40_stdout

      
    Raw
  

              45886605_1_rank.40_stdout
            
          
## 45886605_1_rank.41_stderr

      
    Raw
  

              45886605_1_rank.41_stderr
            
          
## 45886605_1_rank.41_stdout

      
    Raw
  

              45886605_1_rank.41_stdout
            
          
## 45886605_1_rank.42_stderr

      
    Raw
  

              45886605_1_rank.42_stderr
            
          
## 45886605_1_rank.42_stdout

      
    Raw
  

              45886605_1_rank.42_stdout
            
          
## 45886605_1_rank.43_stderr

      
    Raw
  

              45886605_1_rank.43_stderr
            
          
## 45886605_1_rank.43_stdout

      
    Raw
  

              45886605_1_rank.43_stdout
            
          
## 45886605_1_rank.44_stderr

      
    Raw
  

              45886605_1_rank.44_stderr
            
          
## 45886605_1_rank.44_stdout

      
    Raw
  

              45886605_1_rank.44_stdout
            
          
## 45886605_1_rank.45_stderr

      
    Raw
  

              45886605_1_rank.45_stderr
            
          
## 45886605_1_rank.45_stdout

      
    Raw
  

              45886605_1_rank.45_stdout
            
          
## 45886605_1_rank.46_stderr

      
    Raw
  

              45886605_1_rank.46_stderr
            
          
## 45886605_1_rank.46_stdout

      
    Raw
  

              45886605_1_rank.46_stdout
            
          
## 45886605_1_rank.47_stderr

      
    Raw
  

              45886605_1_rank.47_stderr
            
          
## 45886605_1_rank.47_stdout

      
    Raw
  

              45886605_1_rank.47_stdout
            
          
## 45886605_1_rank.48_stderr

      
    Raw
  

              45886605_1_rank.48_stderr
            
          
## 45886605_1_rank.48_stdout

      
    Raw
  

              45886605_1_rank.48_stdout
            
          
## 45886605_1_rank.49_stderr

      
    Raw
  

              45886605_1_rank.49_stderr
            
          
## 45886605_1_rank.49_stdout

      
    Raw
  

              45886605_1_rank.49_stdout
            
          
## 45886605_1_rank.50_stderr

      
    Raw
  

              45886605_1_rank.50_stderr
            
          
## 45886605_1_rank.50_stdout

      
    Raw
  

              45886605_1_rank.50_stdout
            
          
## 45886605_1_rank.51_stderr

      
    Raw
  

              45886605_1_rank.51_stderr
            
          
## 45886605_1_rank.51_stdout

      
    Raw
  

              45886605_1_rank.51_stdout
            
          
## 45886605_1_rank.52_stderr

      
    Raw
  

              45886605_1_rank.52_stderr
            
          
## 45886605_1_rank.52_stdout

      
    Raw
  

              45886605_1_rank.52_stdout
            
          
## 45886605_1_rank.53_stderr

      
    Raw
  

              45886605_1_rank.53_stderr
            
          
## 45886605_1_rank.53_stdout

      
    Raw
  

              45886605_1_rank.53_stdout
            
          
## 45886605_1_rank.54_stderr

      
    Raw
  

              45886605_1_rank.54_stderr
            
          
## 45886605_1_rank.54_stdout

      
    Raw
  

              45886605_1_rank.54_stdout
            
          
## 45886605_1_rank.55_stderr

      
    Raw
  

              45886605_1_rank.55_stderr
            
          
## 45886605_1_rank.55_stdout

      
    Raw
  

              45886605_1_rank.55_stdout
            
          
## 45886605_1_rank.56_stderr

      
    Raw
  

              45886605_1_rank.56_stderr
            
          
## 45886605_1_rank.56_stdout

      
    Raw
  

              45886605_1_rank.56_stdout
            
          
## 45886605_1_rank.57_stderr

      
    Raw
  

              45886605_1_rank.57_stderr
            
          
## 45886605_1_rank.57_stdout

      
    Raw
  

              45886605_1_rank.57_stdout
            
          
## 45886605_1_rank.58_stderr

      
    Raw
  

              45886605_1_rank.58_stderr
            
          
## 45886605_1_rank.58_stdout

      
    Raw
  

              45886605_1_rank.58_stdout
            
          
## 45886605_1_rank.59_stderr

      
    Raw
  

              45886605_1_rank.59_stderr
            
          
## 45886605_1_rank.59_stdout

      
    Raw
  

              45886605_1_rank.59_stdout
            
          
## 45886605_1_rank.60_stderr

      
    Raw
  

              45886605_1_rank.60_stderr
            
          
## 45886605_1_rank.60_stdout

      
    Raw
  

              45886605_1_rank.60_stdout
            
          
## 45886605_1_rank.61_stderr

      
    Raw
  

              45886605_1_rank.61_stderr
            
          
## 45886605_1_rank.61_stdout

      
    Raw
  

              45886605_1_rank.61_stdout
            
          
## 45886605_1_rank.62_stderr

      
    Raw
  

              45886605_1_rank.62_stderr
            
          
## 45886605_1_rank.62_stdout

      
    Raw
  

              45886605_1_rank.62_stdout
            
          
## 45886605_1_rank.63_stderr

      
    Raw
  

              45886605_1_rank.63_stderr
            
          
## 45886605_1_rank.63_stdout

      
    Raw
  

              45886605_1_rank.63_stdout
            
          
## hello.jl
using MPI
MPI.Init()

comm = MPI.COMM_WORLD
println("Hello, world, I am rank $(MPI.Comm_rank(comm)) of $(MPI.Comm_size(comm)).")


## mpi-job.sh
#!/bin/bash
#SBATCH --time=0:10:00
##SBATCH --ntasks=3
#SBATCH --nodes=2            # node count
#SBATCH --ntasks-per-node=32 # number of tasks per node
#SBATCH --cpus-per-task=4    # cpu-cores per task
#SBATCH --mem=2G
#SBATCH --output=output-%j.txt

source /etc/profile
module load sdk/nvidia/22.7

# have the shell echo the commands that are run
set -x

mpiexec --mca mpi_cuda_support 0 -output-filename $SLURM_JOB_ID julia hello.jl

exit 0

## output-45886605.txt
+ mpiexec --mca mpi_cuda_support 0 -output-filename 45886605 julia hello.jl
[compute-2-30.hamming.cluster:40895] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40895] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40895] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40895] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40843] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40843] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40843] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40843] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40846] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40846] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40846] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40846] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40850] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40850] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40850] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40850] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40854] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40854] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40854] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40854] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40858] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40858] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40858] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40858] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40862] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40862] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40862] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40862] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40866] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40866] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40866] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40866] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40870] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40870] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40870] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40870] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40874] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40874] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40874] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40874] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40878] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40878] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40878] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40878] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40883] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40883] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40883] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40883] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40887] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40887] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40887] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40887] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40891] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40891] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40891] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40891] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40899] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40899] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40899] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40899] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
[compute-2-30.hamming.cluster:40903] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
[compute-2-30.hamming.cluster:40903] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
[compute-2-30.hamming.cluster:40903] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
[compute-2-30.hamming.cluster:40903] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40883] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40883] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40843] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40843] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40854] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40899] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40899] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40895] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40895] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40846] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40846] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40850] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40850] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40858] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40858] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40862] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40862] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40866] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40866] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40870] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40870] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40874] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40874] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40878] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40878] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40903] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40903] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40887] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40887] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
--------------------------------------------------------------------------
It looks like pmix_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
PMIX developer):

  progress thread failed
  --> Returned value -1 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
[compute-2-30.hamming.cluster:40891] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
[compute-2-30.hamming.cluster:40891] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
[compute-2-30.hamming.cluster:40854] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40899] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40883] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40895] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40846] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40843] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40850] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40858] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40862] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40866] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40854] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40874] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40878] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40903] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40887] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[compute-2-30.hamming.cluster:40891] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
[compute-2-30.hamming.cluster:40870] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------

signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
reset_page at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1209 [inlined]
sweep_page at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1342 [inlined]
sweep_pool_page at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1447 [inlined]
sweep_pool_pagetable0 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1467 [inlined]
sweep_pool_pagetable1 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1487 [inlined]
sweep_pool_pagetable at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1517 [inlined]
gc_sweep_pool at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1592
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3243
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-rel
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_setmark_pool_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:803 [inlined]
gc_setmark_pool at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:827 [inlined]
gc_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:834 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2771
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7f6b56df5c43)
run_load_time_hooks at /home/lwilcox/.
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
jl_gc_premark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3025 [inlined]
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3097
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7ff2e4611c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7ff2e4610fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
jl_gc_premark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3025 [inlined]
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3097
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fb514a1fc43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fb514a1efcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
jl_gc_premark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3025 [inlined]
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3097
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7f8026b07c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7f8026b06fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_try_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1764 [inlined]
gc_mark_scan_obj8 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2006 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2302
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fafb11a0c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/s
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
jl_gc_premark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3025 [inlined]
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3097
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
jl_gc_alloc_buf at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:403 [inlined]
jl_gc_alloc_buf at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:401 [inlined]
array_resize_buffer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:698
jl_array_grow_at_end at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:893
_growend! at ./array.jl:1011 [inlined]
resize! at ./array.jl:1236 [inlined]
resize! at ./compiler/ssair/ir.jl:1142
batch_inline! at ./compiler/ssair/inlining.jl:596
ssa_inlining_pass! at ./compiler/ssair/inlining.jl:85
jfptr_ssa_inlining_passNOT._13336.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
run_passes at ./compiler/optimize.jl:539
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:868
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:835
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:148
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_gene
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
jl_gc_premark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3025 [inlined]
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3097
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7f8663f20c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7f8663f1ffcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_try_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1764 [inlined]
gc_mark_scan_obj8 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2006 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2302
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7f520045fc43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/s
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_try_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1764 [inlined]
gc_mark_scan_obj8 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2006 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2302
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:478 [inlined]
vect at ./array.jl:125 [inlined]
finish at ./compiler/typeinfer.jl:446
_typeinf at ./compiler/typeinfer.jl:243
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fc2195d7c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fc2195d6fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_setmark_pool_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:803 [inlined]
gc_setmark_pool at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:827 [inlined]
gc_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:834 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2781
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:478 [inlined]
vect at ./array.jl:125 [inlined]
finish at ./compiler/typeinfer.jl:446
_typeinf at ./compiler/typeinfer.jl:243
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fd6f796ec43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fd6f796dfcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-relea
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_try_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1764 [inlined]
gc_mark_scan_obj8 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2006 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2302
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fe4e7cf1c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/s
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2592
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fadd8432c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fadd8431fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialan
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
jl_gc_premark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3025 [inlined]
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3097
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fa2f5fb1c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fa2f5fb0fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_try_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1764 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2475
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:468 [inlined]
fill at ./array.jl:536 [inlined]
fill at ./array.jl:534 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:195
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
IRCode at ./compiler/ssair/ir.jl:288 [inlined]
convert_to_ircode at ./compiler/optimize.jl:621
run_passes at ./compiler/optimize.jl:535
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7f0af9213c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7f0af9212fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/j
signal (15): Terminated
in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
gc_try_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1764 [inlined]
gc_mark_scan_objarray at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1876 [inlined]
gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2267
_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
Array at ./boot.jl:459 [inlined]
Array at ./boot.jl:473 [inlined]
InstructionStream at ./compiler/ssair/ir.jl:192
IncrementalCompact at ./compiler/ssair/ir.jl:574
batch_inline! at ./compiler/ssair/inlining.jl:585
ssa_inlining_pass! at ./compiler/ssair/inlining.jl:85
jfptr_ssa_inlining_passNOT._13336.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
run_passes at ./compiler/optimize.jl:539
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:868
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:835
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:148
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/ulia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965
eval at ./boot.jl:368 [inlined]
include_string at ./loading.jl:1428
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
ease-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7f9261f72c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7f9261f71fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965
julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7f6b56df4fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965
eval at ./boot.jl:368 [inlined]
include_string at ./loading.jl:1428
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965
-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965
rc/MPI.jl:137
unknown function (ip: 0x7fafb119ffcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
rate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fa26f760c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fa26f75ffcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
rc/MPI.jl:137
unknown function (ip: 0x7f520045efcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
se-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965
eval at ./boot.jl:368 [inlined]
include_string at ./loading.jl:1428
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
rc/MPI.jl:137
unknown function (ip: 0x7fe4e7cf0fcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
g/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965
eval at ./boot.jl:368 [inlined]
include_string at ./loading.jl:1428
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
julia.h:1843 [inlined]
jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
foreach at ./abstractarray.jl:2774
unknown function (ip: 0x7fad21230c43)
run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
unknown function (ip: 0x7fad2122ffcf)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
_include_from_serialized at ./loading.jl:831
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
--------------------------------------------------------------------------
mpiexec detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[20244,1],0]
  Exit code:    1
--------------------------------------------------------------------------
+ exit 0
	[compute-2-30.hamming.cluster:40843] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
	[compute-2-30.hamming.cluster:40843] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
	[compute-2-30.hamming.cluster:40843] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
	[compute-2-30.hamming.cluster:40843] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
	--------------------------------------------------------------------------
	It looks like pmix_init failed for some reason; your parallel process is
	likely to abort. There are many reasons that a parallel process can
	fail during pmix_init; some of which are due to configuration or
	environment problems. This failure appears to be an internal failure;
	here's some additional information (which may only be relevant to an
	PMIX developer):

	progress thread failed
	--> Returned value -1 instead of PMIX_SUCCESS
	--------------------------------------------------------------------------
	[compute-2-30.hamming.cluster:40843] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
	[compute-2-30.hamming.cluster:40843] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
	*** An error occurred in MPI_Init_thread
	*** on a NULL communicator
	*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
	*** and potentially your MPI job)
	[compute-2-30.hamming.cluster:40843] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

	signal (15): Terminated
	in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
	gc_setmark_pool_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:803 [inlined]
	gc_setmark_pool at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:827 [inlined]
	gc_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:834 [inlined]
	gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2771
	_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
	ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
	maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
	jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
	jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
	jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
	jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
	_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
	_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
	ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
	Array at ./boot.jl:459 [inlined]
	Array at ./boot.jl:468 [inlined]
	fill at ./array.jl:536 [inlined]
	fill at ./array.jl:534 [inlined]
	InstructionStream at ./compiler/ssair/ir.jl:195
	NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
	NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
	IRCode at ./compiler/ssair/ir.jl:288 [inlined]
	convert_to_ircode at ./compiler/optimize.jl:621
	run_passes at ./compiler/optimize.jl:535
	optimize at ./compiler/optimize.jl:504 [inlined]
	_typeinf at ./compiler/typeinfer.jl:257
	typeinf at ./compiler/typeinfer.jl:213
	typeinf_edge at ./compiler/typeinfer.jl:877
	abstract_call_method at ./compiler/abstractinterpretation.jl:647
	abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
	abstract_call_known at ./compiler/abstractinterpretation.jl:1716
	abstract_call at ./compiler/abstractinterpretation.jl:1786
	abstract_call at ./compiler/abstractinterpretation.jl:1753
	abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
	typeinf_local at ./compiler/abstractinterpretation.jl:2386
	typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
	_typeinf at ./compiler/typeinfer.jl:230
	typeinf at ./compiler/typeinfer.jl:213
	typeinf_ext at ./compiler/typeinfer.jl:967
	typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
	typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
	jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
	_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
	ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
	jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
	jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
	jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
	jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
	jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
	_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
	ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
	foreach at ./abstractarray.jl:2774
	unknown function (ip: 0x7f6b56df5c43)
	run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
	__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
	unknown function (ip: 0x7f6b56df4fcf)
	_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
	ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
	jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
	jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
	ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
	_include_from_serialized at ./loading.jl:831
	_require_search_from_serialized at ./loading.jl:1039
	_require at ./loading.jl:1315
	_require_prelocked at ./loading.jl:1200
	macro expansion at ./loading.jl:1180 [inlined]
	macro expansion at ./lock.jl:223 [inlined]
	require at ./loading.jl:1144
	jfptr_require_29799.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
	_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
	ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
	jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
	call_require at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
	eval_import_path at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:499
	jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:725
	jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
	ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965
	eval at ./boot.jl:368 [inlined]
	include_string at ./loading.jl:1428
	_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
	ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
	[compute-2-30.hamming.cluster:40846] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
	[compute-2-30.hamming.cluster:40846] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
	[compute-2-30.hamming.cluster:40846] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
	[compute-2-30.hamming.cluster:40846] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
	--------------------------------------------------------------------------
	It looks like pmix_init failed for some reason; your parallel process is
	likely to abort. There are many reasons that a parallel process can
	fail during pmix_init; some of which are due to configuration or
	environment problems. This failure appears to be an internal failure;
	here's some additional information (which may only be relevant to an
	PMIX developer):

	progress thread failed
	--> Returned value -1 instead of PMIX_SUCCESS
	--------------------------------------------------------------------------
	[compute-2-30.hamming.cluster:40846] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
	[compute-2-30.hamming.cluster:40846] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
	*** An error occurred in MPI_Init_thread
	*** on a NULL communicator
	*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
	*** and potentially your MPI job)
	[compute-2-30.hamming.cluster:40846] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

	signal (15): Terminated
	in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
	jl_gc_premark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3025 [inlined]
	_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3097
	ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
	maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
	jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
	jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
	jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
	jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
	_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
	_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
	ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
	Array at ./boot.jl:459 [inlined]
	Array at ./boot.jl:468 [inlined]
	fill at ./array.jl:536 [inlined]
	fill at ./array.jl:534 [inlined]
	InstructionStream at ./compiler/ssair/ir.jl:195
	NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
	NewNodeStream at ./compiler/ssair/ir.jl:269 [inlined]
	IRCode at ./compiler/ssair/ir.jl:288 [inlined]
	convert_to_ircode at ./compiler/optimize.jl:621
	run_passes at ./compiler/optimize.jl:535
	optimize at ./compiler/optimize.jl:504 [inlined]
	_typeinf at ./compiler/typeinfer.jl:257
	typeinf at ./compiler/typeinfer.jl:213
	typeinf_edge at ./compiler/typeinfer.jl:877
	abstract_call_method at ./compiler/abstractinterpretation.jl:647
	abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
	abstract_call_known at ./compiler/abstractinterpretation.jl:1716
	abstract_call at ./compiler/abstractinterpretation.jl:1786
	abstract_call at ./compiler/abstractinterpretation.jl:1753
	abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
	typeinf_local at ./compiler/abstractinterpretation.jl:2386
	typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
	_typeinf at ./compiler/typeinfer.jl:230
	typeinf at ./compiler/typeinfer.jl:213
	typeinf_ext at ./compiler/typeinfer.jl:967
	typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
	typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
	jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
	_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
	ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
	jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
	jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
	jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
	jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
	jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
	_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
	ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
	foreach at ./abstractarray.jl:2774
	unknown function (ip: 0x7ff2e4611c43)
	run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
	__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
	unknown function (ip: 0x7ff2e4610fcf)
	_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
	ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
	jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
	jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
	ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
	_include_from_serialized at ./loading.jl:831
	_require_search_from_serialized at ./loading.jl:1039
	_require at ./loading.jl:1315
	_require_prelocked at ./loading.jl:1200
	[compute-2-30.hamming.cluster:40850] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
	[compute-2-30.hamming.cluster:40850] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
	[compute-2-30.hamming.cluster:40850] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
	[compute-2-30.hamming.cluster:40850] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
	--------------------------------------------------------------------------
	It looks like pmix_init failed for some reason; your parallel process is
	likely to abort. There are many reasons that a parallel process can
	fail during pmix_init; some of which are due to configuration or
	environment problems. This failure appears to be an internal failure;
	here's some additional information (which may only be relevant to an
	PMIX developer):

	progress thread failed
	--> Returned value -1 instead of PMIX_SUCCESS
	--------------------------------------------------------------------------
	[compute-2-30.hamming.cluster:40850] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
	[compute-2-30.hamming.cluster:40850] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
	*** An error occurred in MPI_Init_thread
	*** on a NULL communicator
	*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
	*** and potentially your MPI job)
	[compute-2-30.hamming.cluster:40850] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
	[compute-2-30.hamming.cluster:40854] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
	[compute-2-30.hamming.cluster:40854] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
	[compute-2-30.hamming.cluster:40854] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
	[compute-2-30.hamming.cluster:40854] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
	--------------------------------------------------------------------------
	It looks like pmix_init failed for some reason; your parallel process is
	likely to abort. There are many reasons that a parallel process can
	fail during pmix_init; some of which are due to configuration or
	environment problems. This failure appears to be an internal failure;
	here's some additional information (which may only be relevant to an
	PMIX developer):

	progress thread failed
	--> Returned value -1 instead of PMIX_SUCCESS
	--------------------------------------------------------------------------
	[compute-2-30.hamming.cluster:40854] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
	[compute-2-30.hamming.cluster:40854] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
	*** An error occurred in MPI_Init_thread
	*** on a NULL communicator
	*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
	*** and potentially your MPI job)
	[compute-2-30.hamming.cluster:40854] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
	[compute-2-30.hamming.cluster:40858] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
	[compute-2-30.hamming.cluster:40858] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
	[compute-2-30.hamming.cluster:40858] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
	[compute-2-30.hamming.cluster:40858] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
	--------------------------------------------------------------------------
	It looks like pmix_init failed for some reason; your parallel process is
	likely to abort. There are many reasons that a parallel process can
	fail during pmix_init; some of which are due to configuration or
	environment problems. This failure appears to be an internal failure;
	here's some additional information (which may only be relevant to an
	PMIX developer):

	progress thread failed
	--> Returned value -1 instead of PMIX_SUCCESS
	--------------------------------------------------------------------------
	[compute-2-30.hamming.cluster:40858] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
	[compute-2-30.hamming.cluster:40858] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
	*** An error occurred in MPI_Init_thread
	*** on a NULL communicator
	*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
	*** and potentially your MPI job)
	[compute-2-30.hamming.cluster:40858] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

	signal (15): Terminated
	in expression starting at /home/lwilcox/scalable/MPI/hello.jl:1
	gc_try_setmark at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1764 [inlined]
	gc_mark_scan_obj8 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2006 [inlined]
	gc_mark_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:2302
	_jl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3117
	ijl_gc_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3346
	maybe_collect at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:903 [inlined]
	jl_gc_pool_alloc_inner at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
	jl_gc_pool_alloc_noinline at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
	jl_gc_alloc_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
	jl_gc_alloc at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gc.c:3391
	_new_array_ at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:134 [inlined]
	_new_array at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:198 [inlined]
	ijl_alloc_array_1d at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/array.c:436
	Array at ./boot.jl:459 [inlined]
	Array at ./boot.jl:478 [inlined]
	vect at ./array.jl:125 [inlined]
	finish at ./compiler/typeinfer.jl:446
	_typeinf at ./compiler/typeinfer.jl:243
	typeinf at ./compiler/typeinfer.jl:213
	typeinf_edge at ./compiler/typeinfer.jl:877
	abstract_call_method at ./compiler/abstractinterpretation.jl:647
	abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
	abstract_call_known at ./compiler/abstractinterpretation.jl:1716
	abstract_call at ./compiler/abstractinterpretation.jl:1786
	abstract_call at ./compiler/abstractinterpretation.jl:1753
	abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
	typeinf_local at ./compiler/abstractinterpretation.jl:2386
	typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
	_typeinf at ./compiler/typeinfer.jl:230
	typeinf at ./compiler/typeinfer.jl:213
	typeinf_ext at ./compiler/typeinfer.jl:967
	typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
	typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
	jfptr_typeinf_ext_toplevel_17873.clone_1 at /home/lwilcox/local/julia/1.8.4/lib/julia/sys.so (unknown line)
	_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
	ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
	jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
	jl_type_infer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:315
	jl_generate_fptr_impl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
	jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2091 [inlined]
	jl_compile_method_internal at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2035
	_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2369 [inlined]
	ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
	foreach at ./abstractarray.jl:2774
	unknown function (ip: 0x7fc2195d7c43)
	run_load_time_hooks at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:57 [inlined]
	__init__ at /home/lwilcox/.julia/packages/MPI/APiiL/src/MPI.jl:137
	unknown function (ip: 0x7fc2195d6fcf)
	_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
	ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
	jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
	jl_module_run_initializer at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:75
	ijl_init_restored_modules at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/dump.c:2866
	_include_from_serialized at ./loading.jl:831
	_require_search_from_serialized at ./loading.jl:1039
	_require at ./loading.jl:1315
	_require_prelocked at ./loading.jl:1200
	macro expansion at ./loading.jl:1180 [inlined]
	macro expansion at ./lock.jl:223 [inlined]
	require at ./loading.jl:1144
	[compute-2-30.hamming.cluster:40862] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
	[compute-2-30.hamming.cluster:40862] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
	[compute-2-30.hamming.cluster:40862] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
	[compute-2-30.hamming.cluster:40862] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
	--------------------------------------------------------------------------
	It looks like pmix_init failed for some reason; your parallel process is
	likely to abort. There are many reasons that a parallel process can
	fail during pmix_init; some of which are due to configuration or
	environment problems. This failure appears to be an internal failure;
	here's some additional information (which may only be relevant to an
	PMIX developer):

	progress thread failed
	--> Returned value -1 instead of PMIX_SUCCESS
	--------------------------------------------------------------------------
	[compute-2-30.hamming.cluster:40862] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
	[compute-2-30.hamming.cluster:40862] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
	*** An error occurred in MPI_Init_thread
	*** on a NULL communicator
	*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
	*** and potentially your MPI job)
	[compute-2-30.hamming.cluster:40862] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
	[compute-2-30.hamming.cluster:40866] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 134
	[compute-2-30.hamming.cluster:40866] OPAL ERROR: Error in file ../../opal/runtime/opal_progress_threads.c at line 193
	[compute-2-30.hamming.cluster:40866] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 140
	[compute-2-30.hamming.cluster:40866] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/runtime/pmix_progress_threads.c at line 199
	--------------------------------------------------------------------------
	It looks like pmix_init failed for some reason; your parallel process is
	likely to abort. There are many reasons that a parallel process can
	fail during pmix_init; some of which are due to configuration or
	environment problems. This failure appears to be an internal failure;
	here's some additional information (which may only be relevant to an
	PMIX developer):

	progress thread failed
	--> Returned value -1 instead of PMIX_SUCCESS
	--------------------------------------------------------------------------
	[compute-2-30.hamming.cluster:40866] PMIX ERROR: ERROR in file ../../../../../../../opal/mca/pmix/pmix2x/pmix/src/client/pmix_client.c at line 426
	[compute-2-30.hamming.cluster:40866] OPAL ERROR: Error in file ../../../../../opal/mca/pmix/pmix2x/pmix2x_client.c at line 112
	*** An error occurred in MPI_Init_thread
	*** on a NULL communicator
	*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
	*** and potentially your MPI job)
	[compute-2-30.hamming.cluster:40866] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!