different parcelport error

gistfile1.txt

+ cd /work/dmarce1/octotiger/release
+ make -k all -j21
make: Nothing to be done for `all'.
+ mkdir 49309.smic3
mkdir: cannot create directory `49309.smic3': File exists
+ cp ./octotiger ./49309.smic3
+ cd 49309.smic3
+ uniq /var/spool/torque/aux//49309.smic3
+ export HPX_NODEFILE=actual.nodes.49309.smic3
+ HPX_NODEFILE=actual.nodes.49309.smic3
+ unset PBS_NODEFILE
+ mpirun -f actual.nodes.49309.smic3 ./octotiger -t20 -Ihpx.stacks.use_guard_pages=0
WARNING: Error in initializing MVAPICH2 ptmalloc library.Continuing without InfiniBand registration cache support.
Regrid gather
Regrid scatter
Regrid done
---------------Created Level 1---------------
Regrid gather
Regrid scatter
Regrid done
---------------Created Level 2---------------
Regrid gather
Regrid scatter
Regrid done
---------------Created Level 3---------------
Regrid gather

{stack-trace}: 4 frames:
0x2b650d24ef5f  : hpx::detail::backtrace(unsigned long) + 0x7f in /home/dmarce1/lib/libhpx.so.0
0x2b650d2504dd  : boost::exception_ptr hpx::detail::get_exception<hpx::exception>(hpx::exception const&, std::string const&, std::string const&, long, std::string const&) + 0x23d in /home/dmarce1/lib/libhpx.so.0
0x2b650d2500ac  : void hpx::detail::throw_exception<hpx::exception>(hpx::exception const&, std::string const&, std::string const&, long) + 0x10c in /home/dmarce1/lib/libhpx.so.0
0x2b650da2c815  : hpx::agas::server::primary_namespace::decrement_sweep(std::list<hpx::agas::server::primary_namespace::free_entry, std::allocator<hpx::agas::server::primary_namespace::free_entry> >&, hpx::naming::gid_type const&, hpx::naming::gid_type const&, long, hpx::error_code&) + 0xc15 in /home/dmarce1/lib/libhpx.so.0
{env}: 103 entries:
  ACLOCAL_PATH=/usr/local/packages/cmake/2.8.12/INTEL-14.0.2/share/aclocal
  BASH_FUNC_module()=() {  eval `/usr/local/packages/Modules/$MODULE_VERSION/bin/modulecmd bash $*`
}
  CPATH=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/ipp/include:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/mkl/include:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/tbb/include:/usr/local/packages/mvapich2/2.0/INTEL-14.0.2/include:/usr/local/compilers/gcc/4.9.0/include:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/mkl/include:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/tbb/include:/home/dmarce1/include:/usr/local/compilers/Intel/parallel_studio-2015u3//impi/5.0.3.048/intel64/include
  CUDA_NIC_INTEROP=1
  CVS_RSH=ssh
  GDBSERVER_MIC=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/gdb/target/mic/bin/gdbserver
  GDB_CROSS=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/gdb/intel64_mic/bin/gdb-mic
  GDB_HOST=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/gdb/intel64_mic/bin/gdb-ia-mic
  GFORTRAN_UNBUFFERED_PRECONNECTED=y
  G_BROKEN_FILENAMES=1
  HISTCONTROL=ignoredups
  HISTSIZE=1000
  HOME=/home/dmarce1
  HOSTNAME=smic027
  HPX_NODEFILE=actual.nodes.49309.smic3
  INFOPATH=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/gdb/intel64/share/info/:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/gdb/intel64_mic/share/info/
  INTEL_LICENSE_FILE=/usr/local/compilers/Intel:/usr/local/compilers/Intel/intel_fc_10.1/licenses:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/licenses:/opt/intel/licenses:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/licenses:/opt/intel/licenses:/home/dmarce1/intel/licenses
  INTEL_PYTHONHOME=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/python/intel64/
  IPPROOT=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/ipp
  LANG=en_US.UTF-8
  LANGUAGE_TERRITORY=en_US
  LD_INCLUDE_PATH=/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/mkl/include
  LD_LIBRARY_PATH=/home/dmarce1/lib:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/compiler/lib/intel64:/opt/intel/mic/coi/host-linux-release/lib:/opt/intel/mic/myo/lib:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/mpirt/lib/intel64:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/ipp/../compiler/lib/intel64:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/ipp/lib/intel64:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/ipp/tools/intel64/perfsys:/opt/intel/mic/coi/host-linux-release/lib:/opt/intel/mic/myo/lib:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/compiler/lib/intel64:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/mkl/lib/intel64:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/tbb/lib/intel64/gcc4.4:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/libipt/intel64/lib:/usr/local/packages/mvapich2/2.0/INTEL-14.0.2/lib:/usr/local/compilers/gcc/4.9.0/lib/gcc/x86_64-unknown-linux-gnu/4.9.0:/usr/local/compilers/gcc/4.9.0/lib64:/usr/local/compilers/gcc/4.9.0/lib:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/compiler/lib/intel64:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/mkl/lib/intel64
  LESSOPEN=|/usr/bin/lesspipe.sh %s
  LHPC_DEVELCMAKE=/usr/local/packages/cmake/2.8.12/INTEL-14.0.2/easybuild/cmake-2.8.12-INTEL-14.0.2-easybuild-devel
  LHPC_DEVELGCC=/usr/local/compilers/gcc/4.9.0/easybuild/gcc-4.9.0-easybuild-devel
  LHPC_DEVELMVAPICH2=/usr/local/packages/mvapich2/2.0/INTEL-14.0.2/easybuild/mvapich2-2.0-INTEL-14.0.2-easybuild-devel
  LHPC_ROOTCMAKE=/usr/local/packages/cmake/2.8.12/INTEL-14.0.2
  LHPC_ROOTGCC=/usr/local/compilers/gcc/4.9.0
  LHPC_ROOTMVAPICH2=/usr/local/packages/mvapich2/2.0/INTEL-14.0.2
  LHPC_VERSIONCMAKE=2.8.12
  LHPC_VERSIONGCC=4.9.0
  LHPC_VERSIONMVAPICH2=2.0
  LIBRARY_PATH=/home/dmarce1/lib:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/compiler/lib/intel64:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/ipp/../compiler/lib/intel64:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/ipp/lib/intel64:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/compiler/lib/intel64:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/mkl/lib/intel64:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/tbb/lib/intel64/gcc4.4:/usr/local/packages/mvapich2/2.0/INTEL-14.0.2/lib:/usr/local/compilers/gcc/4.9.0/lib64:/usr/local/compilers/gcc/4.9.0/lib:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/compiler/lib/intel64:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/mkl/lib/intel64
  LM_LICENSE_FILE=/usr/local/compilers/pgi/license.dat
  LOADEDMODULES=gcc/4.9.0:intel/14.0.2:cmake/2.8.12/INTEL-14.0.2:mvapich2/2.0/INTEL-14.0.2
  LOGNAME=dmarce1
  LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
  MAIL=/var/spool/mail/dmarce1
  MANPATH=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/man/en_US:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/man/en_US:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/gdb/intel64/share/man/:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/gdb/intel64_mic/share/man/:/usr/local/packages/mvapich2/2.0/INTEL-14.0.2/share/man:/usr/local/packages/cmake/2.8.12/INTEL-14.0.2/man:/usr/local/compilers/gcc/4.9.0/share/man:/usr/local/compilers/pgi/linux86-64/man:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/man/en_US:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/man/en_US:
  MIC_LD_LIBRARY_PATH=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/compiler/lib/mic:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/mpirt/lib/mic:/opt/intel/mic/coi/device-linux-release/lib:/opt/intel/mic/myo/lib:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/ipp/lib/lib/mic:/opt/intel/mic/coi/device-linux-release/lib:/opt/intel/mic/myo/lib:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/compiler/lib/mic:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/mkl/lib/mic:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/tbb/lib/mic:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/compiler/lib/mic:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/mkl/lib/mic
  MIC_LIBRARY_PATH=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/compiler/lib/mic:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/mpirt/lib/mic:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/tbb/lib/mic
  MKLROOT=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/mkl
  MOABHOMEDIR=/usr/moab
  MODULEPATH=/usr/local/packages/Modules/modulefiles/apps:/usr/local/packages/Modules/modulefiles/admin:/usr/local/packages/Modules/modulefiles/xsede
  MODULESHOME=/usr/local/packages/Modules/3.2.10
  MODULE_VERSION=3.2.10
  MODULE_VERSION_STACK=3.2.10
  MPIR_CVAR_CH3_INTERFACE_HOSTNAME=smic148
  MPM_LAUNCHER=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/mpm/bin/start_mpm.sh
  MV2_COMM_WORLD_LOCAL_RANK=0
  MV2_COMM_WORLD_LOCAL_SIZE=1
  MV2_COMM_WORLD_RANK=70
  MV2_COMM_WORLD_SIZE=100
  MV2_IBA_HCA=mlx4_0
  NLSPATH=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/compiler/lib/intel64/locale/%l_%t/%N:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/ipp/lib/intel64/locale/%l_%t/%N:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/mkl/lib/intel64/locale/%l_%t/%N:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/gdb/intel64_mic/share/locale/%l_%t/%N:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/gdb/intel64/share/locale/%l_%t/%N:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/compiler/lib/intel64/locale/en_US:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/mkl/lib/intel64/locale/en_US
  OLDPWD=/work/dmarce1/octotiger/release
  PATH=/home/dmarce1/bin:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/bin/intel64:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/gdb/intel64_mic/bin:/usr/lib64/qt-3.3/bin:/usr/local/packages/mvapich2/2.0/INTEL-14.0.2/bin:/usr/local/packages/cmake/2.8.12/INTEL-14.0.2/bin:/usr/local/compilers/gcc/4.9.0/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/compilers/pgi/linux86-64/6.1/bin:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/bin/intel64:/opt/ibutils/bin:/home/dmarce1/bin
  PBS_ENVIRONMENT=PBS_INTERACTIVE
  PBS_GPUFILE=/var/spool/torque/aux//49309.smic3gpu
  PBS_JOBCOOKIE=2FEA25494D192C1016E0B4D3430C7280
  PBS_JOBID=49309.smic3
  PBS_JOBNAME=STDIN
  PBS_MOMPORT=15003
  PBS_NODENUM=0
  PBS_NP=2000
  PBS_NUM_NODES=100
  PBS_NUM_PPN=20
  PBS_O_HOME=/home/dmarce1
  PBS_O_HOST=smic1.hpc.lsu.edu
  PBS_O_LANG=en_US.UTF-8
  PBS_O_LOGNAME=dmarce1
  PBS_O_MAIL=/var/spool/mail/dmarce1
  PBS_O_PATH=/home/dmarce1/bin:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/bin/intel64:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/gdb/intel64_mic/bin:/home/dmarce1/bin:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/bin/intel64:/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/debugger/gdb/intel64_mic/bin:/usr/lib64/qt-3.3/bin:/usr/local/packages/mvapich2/2.0/INTEL-14.0.2/bin:/usr/local/packages/cmake/2.8.12/INTEL-14.0.2/bin:/usr/local/compilers/gcc/4.9.0/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/compilers/pgi/linux86-64/6.1/bin:/usr/local/compilers/Intel/cluster_studio_xe_2013.1.046/composer_xe_2013_sp1.2.144/bin/intel64:/opt/ibutils/bin:/opt/dell/srvadmin/bin:/home/dmarce1/bin:/usr/local/compilers/pgi/linux86-64/6.1/bin
  PBS_O_QUEUE=workq
  PBS_O_SHELL=/bin/bash
  PBS_O_WORKDIR=/home/dmarce1
  PBS_QUEUE=workq
  PBS_SERVER=smic3
  PBS_TASKNUM=1
  PBS_VERSION=TORQUE-3.0.6
  PBS_VNODENUM=0
  PBS_WALLTIME=3600
  PGI=/usr/local/compilers/pgi
  PKG_CONFIG_PATH=/usr/local/packages/mvapich2/2.0/INTEL-14.0.2/lib/pkgconfig:/home/dmarce1/lib/pkgconfig
  PMI_FD=5
  PMI_RANK=70
  PMI_SIZE=100
  PWD=/work/dmarce1/octotiger/release/49309.smic3
  QTDIR=/usr/lib64/qt-3.3
  QTINC=/usr/lib64/qt-3.3/include
  QTLIB=/usr/lib64/qt-3.3/lib
  SHELL=/bin/bash
  SHLVL=2
  SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
  SSH_CLIENT=204.90.41.27 33084 22
  SSH_CONNECTION=204.90.41.27 33084 204.90.41.148 22
  TBBROOT=/usr/local/compilers/Intel/parallel_studio-2015u3/composer_xe_2015.3.187/tbb
  TERM=screen
  USER=dmarce1
  _=/usr/local/packages/mvapich2/2.0/INTEL-14.0.2/bin/mpirun
  _LMFILES_=/usr/local/packages/Modules/modulefiles/apps/gcc/4.9.0:/usr/local/packages/Modules/modulefiles/apps/intel/14.0.2:/usr/local/packages/Modules/modulefiles/apps/cmake/2.8.12/INTEL-14.0.2:/usr/local/packages/Modules/modulefiles/apps/mvapich2/2.0/INTEL-14.0.2
  _MODULESBEGINENV_=/home/dmarce1/.modulesbeginenv
{locality-id}: 70
{hostname}: [ (mpi:70) (tcp:127.0.0.1:7980) ]
{process-id}: 26688
{function}: primary_namespace::decrement_sweep
{file}: /work/dmarce1/builds/hpx/src/runtime/agas/server/primary_namespace_server.cpp
{line}: 1153
{os-thread}: 0, worker-thread#0
{thread-id}: 00002b651dc6df00
{thread-description}: primary_namespace_bulk_service_action
{state}: state_running
{auxinfo}: 
{config}:
  HPX_HAVE_NATIVE_TLS=ON
  HPX_HAVE_STACKTRACES=ON
  HPX_HAVE_COMPRESSION_BZIP2=OFF
  HPX_HAVE_COMPRESSION_SNAPPY=OFF
  HPX_HAVE_COMPRESSION_ZLIB=OFF
  HPX_HAVE_PARCEL_COALESCING=ON
  HPX_HAVE_PARCELPORT_TCP=ON
  HPX_HAVE_PARCELPORT_MPI=ON (MPICH V3.1, MPI V3.0)
  HPX_HAVE_PARCELPORT_IPC=OFF
  HPX_HAVE_PARCELPORT_IBVERBS=OFF
  HPX_HAVE_VERIFY_LOCKS=ON
  HPX_HAVE_HWLOC=ON
  HPX_HAVE_ITTNOTIFY=OFF
  HPX_HAVE_RUN_MAIN_EVERYWHERE=OFF
  HPX_LIMIT=5
  HPX_PARCEL_MAX_CONNECTIONS=512
  HPX_PARCEL_MAX_CONNECTIONS_PER_LOCALITY=4
  HPX_INITIAL_AGAS_LOCAL_CACHE_SIZE=256
  HPX_AGAS_LOCAL_CACHE_SIZE_PER_THREAD=32
  HPX_HAVE_MALLOC=tbbmalloc
  HPX_PREFIX (configured)=/work/dmarce1/builds/hpx_build
  HPX_PREFIX=/home/dmarce1
{version}: V0.9.11-trunk (AGAS: V3.0), Git: f75d2b0ff4
{boost}: V1.59.0
{build-type}: release
{date}: Sep 16 2015 17:08:10
{platform}: linux
{compiler}: Intel C++ C++0x mode version 1500
{stdlib}: GNU libstdc++ version 20140422
{what}: negative entry in reference count table, raw({0000004700000001, 0000000000b013ec}), refcount(-1073741824): HPX(invalid_data)

{stack-trace}: 2 frames:
0x2b650d2c3c88  : hpx::termination_handler(int) + 0x158 in /home/dmarce1/lib/libhpx.so.0
0x389ac0f710    : ??? + 0x389ac0f710 in /lib64/libpthread.so.0
{what}: Segmentation fault
{config}:
  HPX_HAVE_NATIVE_TLS=ON
  HPX_HAVE_STACKTRACES=ON
  HPX_HAVE_COMPRESSION_BZIP2=OFF
  HPX_HAVE_COMPRESSION_SNAPPY=OFF
  HPX_HAVE_COMPRESSION_ZLIB=OFF
  HPX_HAVE_PARCEL_COALESCING=ON
  HPX_HAVE_PARCELPORT_TCP=ON
  HPX_HAVE_PARCELPORT_MPI=ON (MPICH V3.1, MPI V3.0)
  HPX_HAVE_PARCELPORT_IPC=OFF
  HPX_HAVE_PARCELPORT_IBVERBS=OFF
  HPX_HAVE_VERIFY_LOCKS=ON
  HPX_HAVE_HWLOC=ON
  HPX_HAVE_ITTNOTIFY=OFF
  HPX_HAVE_RUN_MAIN_EVERYWHERE=OFF
  HPX_LIMIT=5
  HPX_PARCEL_MAX_CONNECTIONS=512
  HPX_PARCEL_MAX_CONNECTIONS_PER_LOCALITY=4
  HPX_INITIAL_AGAS_LOCAL_CACHE_SIZE=256
  HPX_AGAS_LOCAL_CACHE_SIZE_PER_THREAD=32
  HPX_HAVE_MALLOC=tbbmalloc
  HPX_PREFIX (configured)=/work/dmarce1/builds/hpx_build
  HPX_PREFIX=/home/dmarce1
{version}: V0.9.11-trunk (AGAS: V3.0), Git: f75d2b0ff4
{boost}: V1.59.0
{build-type}: release
{date}: Sep 16 2015 17:08:10
{platform}: linux
{compiler}: Intel C++ C++0x mode version 1500
{stdlib}: GNU libstdc++ version 20140422
[smic148:mpi_rank_70][error_sighandler] Caught error: Aborted (signal 6)

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 26688 RUNNING AT smic148
=   EXIT CODE: 6
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================