Created
October 4, 2014 17:56
-
-
Save ashkurti/76a761d05ac71a6848dc to your computer and use it in GitHub Desktop.
Bag of Tasks fails locally (linux environment) - with debug
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [ExTASY-toolsOct2] ardita@jekyll 137% python remote_simple_bot.py | |
| 2014:10:04 18:29:42 radical.pilot.MainProcess: [INFO ] radical.pilot version: 0.20 | |
| 2014:10:04 18:29:43 radical.pilot.MainProcess: [INFO ] using database url mongodb://ec2-184-72-89-141.compute-1.amazonaws.com:27017/ | |
| 2014:10:04 18:29:43 radical.pilot.MainProcess: [INFO ] using database name radicalpilot | |
| 2014:10:04 18:29:43 radical.pilot.MainProcess: [INFO ] Loaded resource configurations from /users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/configs/localhost.json | |
| 2014:10:04 18:29:43 radical.pilot.MainProcess: [INFO ] Loaded resource configurations from /users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/configs/futuregrid.json | |
| 2014:10:04 18:29:43 radical.pilot.MainProcess: [INFO ] Loaded resource configurations from /users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/configs/lrz.json | |
| 2014:10:04 18:29:43 radical.pilot.MainProcess: [INFO ] Loaded resource configurations from /users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/configs/epsrc.json | |
| 2014:10:04 18:29:43 radical.pilot.MainProcess: [INFO ] Loaded resource configurations from /users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/configs/radical.json | |
| 2014:10:04 18:29:43 radical.pilot.MainProcess: [INFO ] Loaded resource configurations from /users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/configs/das4.json | |
| 2014:10:04 18:29:43 radical.pilot.MainProcess: [INFO ] Loaded resource configurations from /users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/configs/ncar.json | |
| 2014:10:04 18:29:43 radical.pilot.MainProcess: [INFO ] Loaded resource configurations from /users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/configs/iu.json | |
| 2014:10:04 18:29:43 radical.pilot.MainProcess: [INFO ] Loaded resource configurations from /users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/configs/xsede.json | |
| 2014:10:04 18:29:45 radical.pilot.MainProcess: [INFO ] New Session created{'database_url': 'mongodb://ec2-184-72-89-141.compute-1.amazonaws.com:27017/', 'database_name': 'radicalpilot', 'last_reconnect': None, 'uid': '54302e87e14fa24f7d66d794', 'created': datetime.datetime(2014, 10, 4, 17, 29, 44, 48190)}. | |
| Initializing Pilot Manager ... | |
| 2014:10:04 18:29:45 radical.pilot.MainProcess: [DEBUG ] Worker thread (ID: Thread-1[139822018823936]) for PilotManager 54302e89e14fa24f7d66d795 started. | |
| Submitting Compute Pilot to Pilot Manager ... | |
| 2014:10:04 18:29:45 radical.pilot.MainProcess: [DEBUG ] Connected to MongoDB. Serving requests for PilotManager 54302e89e14fa24f7d66d795. | |
| 2014:10:04 18:29:45 radical.pilot.MainProcess: [DEBUG ] saga.utils.PTYShell ('sftp://stampede.tacc.utexas.edu/') | |
| 2014:10:04 18:29:45 radical.pilot.MainProcess: [DEBUG ] PTYShell init <saga.utils.pty_shell.PTYShell object at 0x23b5d10> | |
| 2014:10:04 18:29:45 radical.pilot.MainProcess: [INFO ] PTY prompt pattern: [\$#%>\]]\s*$ | |
| 2014:10:04 18:29:45 radical.pilot.MainProcess: [DEBUG ] open master pty for [ssh] [ardi@stampede.tacc.utexas.edu] ardi: /usr/bin/env TERM=vt100 "/usr/bin/ssh" -t -o IdentityFile=/users/ardita/.ssh/id_rsa -o ControlMaster=yes -o ControlPath=/tmp/saga_ssh_ardita_%h_%p.ardi.ctrl ardi@stampede.tacc.utexas.edu' | |
| 2014:10:04 18:29:45 radical.pilot.MainProcess: [DEBUG ] PTYProcess init <saga.utils.pty_process.PTYProcess object at 0x23b5f10> | |
| 2014:10:04 18:29:45 radical.pilot.MainProcess: [INFO ] running: /usr/bin/env TERM=vt100 /usr/bin/ssh -t -o IdentityFile=/users/ardita/.ssh/id_rsa -o ControlMaster=yes -o ControlPath=/tmp/saga_ssh_ardita_%h_%p.ardi.ctrl ardi@stampede.tacc.utexas.edu | |
| 2014:10:04 18:29:45 radical.pilot.MainProcess: [DEBUG ] write: [ 6] [ 33] ( export PS1='$' ; set prompt='$'\n) | |
| 2014:10:04 18:29:46 radical.pilot.MainProcess: [DEBUG ] write: [ 6] [ 51] ( export PS1='$' > /dev/null 2>&1 || set prompt='$'\n) | |
| 2014:10:04 18:29:46 radical.pilot.MainProcess: [DEBUG ] write: [ 6] [ 28] ( printf 'HELLO_%d_SAGA\n' 1\n) | |
| 2014:10:04 18:29:47 radical.pilot.MainProcess: [DEBUG ] write: [ 6] [ 51] ( export PS1='$' > /dev/null 2>&1 || set prompt='$'\n) | |
| 2014:10:04 18:29:47 radical.pilot.MainProcess: [DEBUG ] write: [ 6] [ 28] ( printf 'HELLO_%d_SAGA\n' 2\n) | |
| 2014:10:04 18:29:48 radical.pilot.MainProcess: [DEBUG ] write: [ 6] [ 51] ( export PS1='$' > /dev/null 2>&1 || set prompt='$'\n) | |
| 2014:10:04 18:29:48 radical.pilot.MainProcess: [DEBUG ] write: [ 6] [ 28] ( printf 'HELLO_%d_SAGA\n' 3\n) | |
| 2014:10:04 18:29:49 radical.pilot.MainProcess: [DEBUG ] read : [ 6] [ 1023] (Last login: Fri Oct 3 10:39:1 ... uides/\nUser News: http:/) | |
| 2014:10:04 18:29:49 radical.pilot.MainProcess: [DEBUG ] read : [ 6] [ 1024] (/www.tacc.utexas.edu/user-serv ... as three parallel file systems) | |
| 2014:10:04 18:29:49 radical.pilot.MainProcess: [DEBUG ] got initial shell prompt (5) (Last login: Fri Oct 3 10:39:19 2014 from poirot.pharm.nottingham.ac.uk | |
| ------------------------------------------------------------------------------ | |
| Welcome to the Stampede Supercomputer | |
| Texas Advanced Computing Center, The University of Texas at Austin | |
| ------------------------------------------------------------------------------ | |
| ** Unauthorized use/access is prohibited. ** | |
| If you log on to this computer system, you acknowledge your awareness | |
| of and concurrence with the UT Austin Acceptable Use Policy. The | |
| University will prosecute violators to the full extent of the law. | |
| TACC Usage Policies: | |
| http://www.tacc.utexas.edu/user-services/usage-policies/ | |
| ______________________________________________________________________________ | |
| Questions and Problem Reports: | |
| --> XD Projects: help@xsede.org (email) | |
| --> TACC Projects: portal.tacc.utexas.edu (web) | |
| Documentation: http://www.tacc.utexas.edu/user-services/user-guides/ | |
| User News: http://www.tacc.utexas.edu/user-services/user-news/ | |
| ______________________________________________________________________________ | |
| Welcome to Stampede, *please* read these important system notes: | |
| --> Stampede is currently running the SLURM resource manager to | |
| schedule all compute resources. Example SLURM job scripts are | |
| available on the system at /share/doc/slurm | |
| To run an interactive shell, issue: | |
| srun -p development -t 0:30:00 -n 32 --pty /bin/bash -l | |
| To submit a batch job, issue: sbatch job.mpi | |
| To show all queued jobs, issue: showq | |
| To kill a queued job, issue: scancel <jobId> | |
| ) | |
| 2014:10:04 18:29:49 radical.pilot.MainProcess: [DEBUG ] waiting for prompt trigger HELLO_3_SAGA: (5) (Last login: Fri Oct 3 10:39:19 2014 from poirot.pharm.nottingham.ac.uk | |
| ------------------------------------------------------------------------------ | |
| Welcome to the Stampede Supercomputer | |
| Texas Advanced Computing Center, The University of Texas at Austin | |
| ------------------------------------------------------------------------------ | |
| ** Unauthorized use/access is prohibited. ** | |
| If you log on to this computer system, you acknowledge your awareness | |
| of and concurrence with the UT Austin Acceptable Use Policy. The | |
| University will prosecute violators to the full extent of the law. | |
| TACC Usage Policies: | |
| http://www.tacc.utexas.edu/user-services/usage-policies/ | |
| ______________________________________________________________________________ | |
| Questions and Problem Reports: | |
| --> XD Projects: help@xsede.org (email) | |
| --> TACC Projects: portal.tacc.utexas.edu (web) | |
| Documentation: http://www.tacc.utexas.edu/user-services/user-guides/ | |
| User News: http://www.tacc.utexas.edu/user-services/user-news/ | |
| ______________________________________________________________________________ | |
| Welcome to Stampede, *please* read these important system notes: | |
| --> Stampede is currently running the SLURM resource manager to | |
| schedule all compute resources. Example SLURM job scripts are | |
| available on the system at /share/doc/slurm | |
| To run an interactive shell, issue: | |
| srun -p development -t 0:30:00 -n 32 --pty /bin/bash -l | |
| To submit a batch job, issue: sbatch job.mpi | |
| To show all queued jobs, issue: showq | |
| To kill a queued job, issue: scancel <jobId> | |
| ) | |
| 2014:10:04 18:29:49 radical.pilot.MainProcess: [DEBUG ] read : [ 6] [ 349] (: $HOME (permanent,\n quota'd, backed-up) $WORK (permanent, quota'd, not backed-up) and\n $SCRATCH (high-speed purged storage). The "cdw" and "cds" aliases\n are provided as a convenience to change to your $WORK and $SCRATCH\n directories, respectively.\n______________________________________________________________________________\n\n) | |
| 2014:10:04 18:29:50 radical.pilot.MainProcess: [DEBUG ] read : [ 6] [ 244] (----------------------- Project balances for user ardi ------------------------\n| Name Avail SUs Expires | Name Avail SUs Expires |\n| TG-MCB090174 104603 2015-09-30 | TG-TRA140016 -81080 2015-05-06 | \n) | |
| 2014:10:04 18:29:50 radical.pilot.MainProcess: [DEBUG ] read : [ 6] [ 405] (-------------------------- Disk quotas for user ardi --------------------------\n| Disk Usage (GB) Limit %Used File Usage Limit %Used |\n| /home1 1.0 5.0 19.52 22438 150000 14.96 |\n| /work 0.0 1024.0 0.00 2651 3000000 0.09 |\n-------------------------------------------------------------------------------\n) | |
| 2014:10:04 18:29:50 radical.pilot.MainProcess: [DEBUG ] read : [ 6] [ 147] (\nTip 73 (See "module help tacc_tips" for features or how to disable)\n\n Want to measure how long a command takes, do:\n $ time a.out\n\n) | |
| 2014:10:04 18:29:50 radical.pilot.MainProcess: [DEBUG ] read : [ 6] [ 69] (login2.stampede(1)$ $$HELLO_1_SAGA\n$$HELLO_2_SAGA\n$$HELLO_3_SAGA\n$) | |
| 2014:10:04 18:29:50 radical.pilot.MainProcess: [DEBUG ] got shell prompt trigger (4) ( | |
| See "man slurm" or the Stampede user guide for more detailed information. | |
| --> To see all the software that is available across all compilers and | |
| mpi stacks, issue: "module spider" | |
| --> To see which software packages are available with your currently loaded | |
| compiler and mpi stack, issue: "module avail" | |
| --> Stampede has three parallel file systems: $HOME (permanent, | |
| quota'd, backed-up) $WORK (permanent, quota'd, not backed-up) and | |
| $SCRATCH (high-speed purged storage). The "cdw" and "cds" aliases | |
| are provided as a convenience to change to your $WORK and $SCRATCH | |
| directories, respectively. | |
| ______________________________________________________________________________ | |
| ----------------------- Project balances for user ardi ------------------------ | |
| | Name Avail SUs Expires | Name Avail SUs Expires | | |
| | TG-MCB090174 104603 2015-09-30 | TG-TRA140016 -81080 2015-05-06 | | |
| -------------------------- Disk quotas for user ardi -------------------------- | |
| | Disk Usage (GB) Limit %Used File Usage Limit %Used | | |
| | /home1 1.0 5.0 19.52 22438 150000 14.96 | | |
| | /work 0.0 1024.0 0.00 2651 3000000 0.09 | | |
| ------------------------------------------------------------------------------- | |
| Tip 73 (See "module help tacc_tips" for features or how to disable) | |
| Want to measure how long a command takes, do: | |
| $ time a.out | |
| login2.stampede(1)$ $$HELLO_1_SAGA | |
| $$HELLO_2_SAGA | |
| $$HELLO_3_SAGA) | |
| 2014:10:04 18:29:50 radical.pilot.MainProcess: [DEBUG ] got initial shell prompt (5) ( | |
| $) | |
| 2014:10:04 18:29:50 radical.pilot.MainProcess: [DEBUG ] Got initial shell prompt (5) ( | |
| $) | |
| 2014:10:04 18:29:50 radical.pilot.MainProcess: [DEBUG ] PTYProcess init <saga.utils.pty_process.PTYProcess object at 0x23a1e90> | |
| 2014:10:04 18:29:50 radical.pilot.MainProcess: [INFO ] running: /usr/bin/env TERM=vt100 /usr/bin/ssh -t -o IdentityFile=/users/ardita/.ssh/id_rsa -o ControlMaster=no -o ControlPath=/tmp/saga_ssh_ardita_%h_%p.ardi.ctrl ardi@stampede.tacc.utexas.edu | |
| 2014:10:04 18:29:50 radical.pilot.MainProcess: [DEBUG ] write: [ 7] [ 33] ( export PS1='$' ; set prompt='$'\n) | |
| 2014:10:04 18:29:51 radical.pilot.MainProcess: [DEBUG ] read : [ 7] [ 1023] (Last login: Sat Oct 4 12:29:4 ... uides/\nUser News: http:/) | |
| 2014:10:04 18:29:51 radical.pilot.MainProcess: [DEBUG ] read : [ 7] [ 1024] (/www.tacc.utexas.edu/user-serv ... as three parallel file systems) | |
| 2014:10:04 18:29:51 radical.pilot.MainProcess: [DEBUG ] got initial shell prompt (5) (Last login: Sat Oct 4 12:29:49 2014 from jekyll.pharm.nottingham.ac.uk | |
| ------------------------------------------------------------------------------ | |
| Welcome to the Stampede Supercomputer | |
| Texas Advanced Computing Center, The University of Texas at Austin | |
| ------------------------------------------------------------------------------ | |
| ** Unauthorized use/access is prohibited. ** | |
| If you log on to this computer system, you acknowledge your awareness | |
| of and concurrence with the UT Austin Acceptable Use Policy. The | |
| University will prosecute violators to the full extent of the law. | |
| TACC Usage Policies: | |
| http://www.tacc.utexas.edu/user-services/usage-policies/ | |
| ______________________________________________________________________________ | |
| Questions and Problem Reports: | |
| --> XD Projects: help@xsede.org (email) | |
| --> TACC Projects: portal.tacc.utexas.edu (web) | |
| Documentation: http://www.tacc.utexas.edu/user-services/user-guides/ | |
| User News: http://www.tacc.utexas.edu/user-services/user-news/ | |
| ______________________________________________________________________________ | |
| Welcome to Stampede, *please* read these important system notes: | |
| --> Stampede is currently running the SLURM resource manager to | |
| schedule all compute resources. Example SLURM job scripts are | |
| available on the system at /share/doc/slurm | |
| To run an interactive shell, issue: | |
| srun -p development -t 0:30:00 -n 32 --pty /bin/bash -l | |
| To submit a batch job, issue: sbatch job.mpi | |
| To show all queued jobs, issue: showq | |
| To kill a queued job, issue: scancel <jobId> | |
| ) | |
| 2014:10:04 18:29:51 radical.pilot.MainProcess: [DEBUG ] Got initial shell prompt (5) (Last login: Sat Oct 4 12:29:49 2014 from jekyll.pharm.nottingham.ac.uk | |
| ------------------------------------------------------------------------------ | |
| Welcome to the Stampede Supercomputer | |
| Texas Advanced Computing Center, The University of Texas at Austin | |
| ------------------------------------------------------------------------------ | |
| ** Unauthorized use/access is prohibited. ** | |
| If you log on to this computer system, you acknowledge your awareness | |
| of and concurrence with the UT Austin Acceptable Use Policy. The | |
| University will prosecute violators to the full extent of the law. | |
| TACC Usage Policies: | |
| http://www.tacc.utexas.edu/user-services/usage-policies/ | |
| ______________________________________________________________________________ | |
| Questions and Problem Reports: | |
| --> XD Projects: help@xsede.org (email) | |
| --> TACC Projects: portal.tacc.utexas.edu (web) | |
| Documentation: http://www.tacc.utexas.edu/user-services/user-guides/ | |
| User News: http://www.tacc.utexas.edu/user-services/user-news/ | |
| ______________________________________________________________________________ | |
| Welcome to Stampede, *please* read these important system notes: | |
| --> Stampede is currently running the SLURM resource manager to | |
| schedule all compute resources. Example SLURM job scripts are | |
| available on the system at /share/doc/slurm | |
| To run an interactive shell, issue: | |
| srun -p development -t 0:30:00 -n 32 --pty /bin/bash -l | |
| To submit a batch job, issue: sbatch job.mpi | |
| To show all queued jobs, issue: showq | |
| To kill a queued job, issue: scancel <jobId> | |
| ) | |
| 2014:10:04 18:29:51 radical.pilot.MainProcess: [DEBUG ] running command shell: exec /bin/sh -i | |
| 2014:10:04 18:29:51 radical.pilot.MainProcess: [DEBUG ] write: [ 7] [ 47] ( stty -echo ; unset HISTFILE ; exec /bin/sh -i\n) | |
| 2014:10:04 18:29:51 radical.pilot.MainProcess: [DEBUG ] read : [ 7] [ 349] (: $HOME (permanent,\n quota'd, backed-up) $WORK (permanent, quota'd, not backed-up) and\n $SCRATCH (high-speed purged storage). The "cdw" and "cds" aliases\n are provided as a convenience to change to your $WORK and $SCRATCH\n directories, respectively.\n______________________________________________________________________________\n\n) | |
| 2014:10:04 18:29:51 radical.pilot.MainProcess: [DEBUG ] read : [ 7] [ 244] (----------------------- Project balances for user ardi ------------------------\n| Name Avail SUs Expires | Name Avail SUs Expires |\n| TG-MCB090174 104603 2015-09-30 | TG-TRA140016 -81080 2015-05-06 | \n) | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] read : [ 7] [ 405] (-------------------------- Disk quotas for user ardi --------------------------\n| Disk Usage (GB) Limit %Used File Usage Limit %Used |\n| /home1 1.0 5.0 19.52 22438 150000 14.96 |\n| /work 0.0 1024.0 0.00 2651 3000000 0.09 |\n-------------------------------------------------------------------------------\n) | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] read : [ 7] [ 233] (\nTip 148 (See "module help tacc_tips" for features or how to disable)\n\n Linking to TACC packages? You don't have to remember their locations. Just use $TACC_THATPACKAGE_DIR, $TACC_THATPACKAGE_INC, and $TACC_THATPACKAGE_LIB\n\n) | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] read : [ 7] [ 21] (login2.stampede(1)$ $) | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] write: [ 7] [ 100] ( unset PROMPT_COMMAND ; unset HISTFILE ; PS1='PROMPT-$?->'; PS2=''; export PS1 PS2 2>&1 >/dev/null\n) | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] read : [ 7] [ 1] ($) | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] read : [ 7] [ 10] (PROMPT-0->) | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] got new shell prompt | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] run_sync: echo "WORKDIR: $PWD" | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] write: [ 7] [ 21] (echo "WORKDIR: $PWD"\n) | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] read : [ 7] [ 38] (WORKDIR: /home1/02998/ardi\nPROMPT-0->) | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] Determined remote working directory for sftp://stampede.tacc.utexas.edu/: '/home1/02998/ardi' | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] PTYShell del <saga.utils.pty_shell.PTYShell object at 0x23b5d10> | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [INFO ] Launching ComputePilot {u'state': u'PendingLaunch', u'commands': [], u'description': {u'project': u'TG-MCB090174', u'resource': u'stampede.tacc.utexas.edu', u'queue': None, u'sandbox': None, u'cleanup': True, u'pilot_agent_priv': None, u'memory': None, u'cores': 1, u'runtime': 10}, u'sagajobid': None, u'started': None, u'cores_per_node': None, u'output_transfer_started': None, u'finished': None, u'submitted': datetime.datetime(2014, 10, 4, 17, 29, 52, 438000), u'output_transfer_finished': None, u'sandbox': u'sftp://stampede.tacc.utexas.edu/home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796/', u'pilotmanager': u'54302e89e14fa24f7d66d795', u'unitmanager': None, u'heartbeat': None, u'statehistory': [{u'timestamp': datetime.datetime(2014, 10, 4, 17, 29, 52, 437000), u'state': u'PendingLaunch'}], u'input_transfer_started': None, u'_id': ObjectId('54302e89e14fa24f7d66d796'), u'input_transfer_finished': None, u'nodes': None, u'log': []} | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [INFO ] Using pilot agent /users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/agent/radical-pilot-agent-multicore.py | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [INFO ] Using bootstrapper /users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/bootstrapper/default_bootstrapper.sh | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] Creating agent sandbox 'sftp://stampede.tacc.utexas.edu/home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796/'. | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] saga.fs.Directory ('sftp://stampede.tacc.utexas.edu/home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796/') | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] PTYProcess init <saga.utils.pty_process.PTYProcess object at 0x2e2eed0> | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [INFO ] running: /usr/bin/env TERM=vt100 /usr/bin/ssh -t -o IdentityFile=/users/ardita/.ssh/id_rsa -o ControlMaster=no -o ControlPath=/tmp/saga_ssh_ardita_%h_%p.ardi.ctrl ardi@stampede.tacc.utexas.edu | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] write: [ 8] [ 33] ( export PS1='$' ; set prompt='$'\n) | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] PTYProcess del <saga.utils.pty_process.PTYProcess object at 0x23a1e90> | |
| Initializing Unit Manager ... | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [INFO ] Starting InputFileTransferWorker | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [INFO ] Starting InputFileTransferWorker | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] Worker thread (ID: Thread-3[139821756434176]) for UnitManager 54302e90e14fa24f7d66d797 started. | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [INFO ] Loaded scheduler: DirectSubmissionScheduler. | |
| Registering Compute Pilot with Unit Manager ... | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] Connected to MongoDB. Serving requests for UnitManager 54302e90e14fa24f7d66d797. | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] Connected to MongoDB. Serving requests for UnitManager 54302e90e14fa24f7d66d797. | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] Connected to MongoDB. Serving requests for UnitManager 54302e90e14fa24f7d66d797. | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] Connected to MongoDB. Serving requests for UnitManager 54302e90e14fa24f7d66d797. | |
| Submit Compute Units to Unit Manager ... | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] read : [ 8] [ 1023] (Last login: Sat Oct 4 12:29:5 ... uides/\nUser News: http:/) | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] read : [ 8] [ 1024] (/www.tacc.utexas.edu/user-serv ... as three parallel file systems) | |
| 2014:10:04 18:29:52 radical.pilot.MainProcess: [DEBUG ] got initial shell prompt (5) (Last login: Sat Oct 4 12:29:51 2014 from jekyll.pharm.nottingham.ac.uk | |
| ------------------------------------------------------------------------------ | |
| Welcome to the Stampede Supercomputer | |
| Texas Advanced Computing Center, The University of Texas at Austin | |
| ------------------------------------------------------------------------------ | |
| ** Unauthorized use/access is prohibited. ** | |
| If you log on to this computer system, you acknowledge your awareness | |
| of and concurrence with the UT Austin Acceptable Use Policy. The | |
| University will prosecute violators to the full extent of the law. | |
| TACC Usage Policies: | |
| http://www.tacc.utexas.edu/user-services/usage-policies/ | |
| ______________________________________________________________________________ | |
| Questions and Problem Reports: | |
| --> XD Projects: help@xsede.org (email) | |
| --> TACC Projects: portal.tacc.utexas.edu (web) | |
| Documentation: http://www.tacc.utexas.edu/user-services/user-guides/ | |
| User News: http://www.tacc.utexas.edu/user-services/user-news/ | |
| ______________________________________________________________________________ | |
| Welcome to Stampede, *please* read these important system notes: | |
| --> Stampede is currently running the SLURM resource manager to | |
| schedule all compute resources. Example SLURM job scripts are | |
| available on the system at /share/doc/slurm | |
| To run an interactive shell, issue: | |
| srun -p development -t 0:30:00 -n 32 --pty /bin/bash -l | |
| To submit a batch job, issue: sbatch job.mpi | |
| To show all queued jobs, issue: showq | |
| To kill a queued job, issue: scancel <jobId> | |
| ) | |
| 2014:10:04 18:29:53 radical.pilot.MainProcess: [DEBUG ] Got initial shell prompt (5) (Last login: Sat Oct 4 12:29:51 2014 from jekyll.pharm.nottingham.ac.uk | |
| ------------------------------------------------------------------------------ | |
| Welcome to the Stampede Supercomputer | |
| Texas Advanced Computing Center, The University of Texas at Austin | |
| ------------------------------------------------------------------------------ | |
| ** Unauthorized use/access is prohibited. ** | |
| If you log on to this computer system, you acknowledge your awareness | |
| of and concurrence with the UT Austin Acceptable Use Policy. The | |
| University will prosecute violators to the full extent of the law. | |
| TACC Usage Policies: | |
| http://www.tacc.utexas.edu/user-services/usage-policies/ | |
| ______________________________________________________________________________ | |
| Questions and Problem Reports: | |
| --> XD Projects: help@xsede.org (email) | |
| --> TACC Projects: portal.tacc.utexas.edu (web) | |
| Documentation: http://www.tacc.utexas.edu/user-services/user-guides/ | |
| User News: http://www.tacc.utexas.edu/user-services/user-news/ | |
| ______________________________________________________________________________ | |
| Welcome to Stampede, *please* read these important system notes: | |
| --> Stampede is currently running the SLURM resource manager to | |
| schedule all compute resources. Example SLURM job scripts are | |
| available on the system at /share/doc/slurm | |
| To run an interactive shell, issue: | |
| srun -p development -t 0:30:00 -n 32 --pty /bin/bash -l | |
| To submit a batch job, issue: sbatch job.mpi | |
| To show all queued jobs, issue: showq | |
| To kill a queued job, issue: scancel <jobId> | |
| ) | |
| 2014:10:04 18:29:53 radical.pilot.MainProcess: [DEBUG ] write: [ 8] [ 47] ( stty -echo ; unset HISTFILE ; exec /bin/sh -i\n) | |
| 2014:10:04 18:29:53 radical.pilot.MainProcess: [DEBUG ] read : [ 8] [ 349] (: $HOME (permanent,\n quota'd, backed-up) $WORK (permanent, quota'd, not backed-up) and\n $SCRATCH (high-speed purged storage). The "cdw" and "cds" aliases\n are provided as a convenience to change to your $WORK and $SCRATCH\n directories, respectively.\n______________________________________________________________________________\n\n) | |
| 2014:10:04 18:29:53 radical.pilot.MainProcess: [INFO ] ComputePilot '54302e89e14fa24f7d66d796' state changed from 'PendingLaunch' to 'Launching'. | |
| [Callback]: ComputePilot '54302e89e14fa24f7d66d796' state: Launching. | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [DEBUG ] read : [ 8] [ 244] (----------------------- Project balances for user ardi ------------------------\n| Name Avail SUs Expires | Name Avail SUs Expires |\n| TG-MCB090174 104603 2015-09-30 | TG-TRA140016 -81080 2015-05-06 | \n) | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [DEBUG ] read : [ 8] [ 405] (-------------------------- Disk quotas for user ardi --------------------------\n| Disk Usage (GB) Limit %Used File Usage Limit %Used |\n| /home1 1.0 5.0 19.52 22438 150000 14.96 |\n| /work 0.0 1024.0 0.00 2651 3000000 0.09 |\n-------------------------------------------------------------------------------\n) | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [DEBUG ] read : [ 8] [ 147] (\nTip 73 (See "module help tacc_tips" for features or how to disable)\n\n Want to measure how long a command takes, do:\n $ time a.out\n\n) | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [DEBUG ] read : [ 8] [ 21] (login2.stampede(1)$ $) | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [DEBUG ] write: [ 8] [ 100] ( unset PROMPT_COMMAND ; unset HISTFILE ; PS1='PROMPT-$?->'; PS2=''; export PS1 PS2 2>&1 >/dev/null\n) | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d799' state changed from 'New' to 'PendingExecution'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d799' state: PendingExecution. | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [DEBUG ] read : [ 8] [ 1] ($) | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d79b' state changed from 'New' to 'PendingExecution'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d79b' state: PendingExecution. | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d79c' state changed from 'New' to 'PendingExecution'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d79c' state: PendingExecution. | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [DEBUG ] read : [ 8] [ 10] (PROMPT-0->) | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [DEBUG ] write: [ 8] [ 161] (mkdir -p '/home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796' ; cd '/home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796'\n) | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d79f' state changed from 'New' to 'PendingExecution'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d79f' state: PendingExecution. | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d79d' state changed from 'New' to 'PendingExecution'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d79d' state: PendingExecution. | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d798' state changed from 'New' to 'PendingExecution'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d798' state: PendingExecution. | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [DEBUG ] read : [ 8] [ 10] (PROMPT-0->) | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d7a0' state changed from 'New' to 'PendingExecution'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d7a0' state: PendingExecution. | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d7a1' state changed from 'New' to 'PendingExecution'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d7a1' state: PendingExecution. | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d79e' state changed from 'New' to 'PendingExecution'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d79e' state: PendingExecution. | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [INFO ] Scheduled ComputeUnits [<radical.pilot.compute_unit.ComputeUnit object at 0x2e27e10>, <radical.pilot.compute_unit.ComputeUnit object at 0x2e27fd0>, <radical.pilot.compute_unit.ComputeUnit object at 0x2e27810>, <radical.pilot.compute_unit.ComputeUnit object at 0x2e11090>, <radical.pilot.compute_unit.ComputeUnit object at 0x2e11110>, <radical.pilot.compute_unit.ComputeUnit object at 0x2e27d50>, <radical.pilot.compute_unit.ComputeUnit object at 0x23afd10>, <radical.pilot.compute_unit.ComputeUnit object at 0x2e11250>, <radical.pilot.compute_unit.ComputeUnit object at 0x2e2ee90>, <radical.pilot.compute_unit.ComputeUnit object at 0x2e27ed0>] for execution on ComputePilot '54302e89e14fa24f7d66d796'. | |
| 2014:10:04 18:29:54 radical.pilot.MainProcess: [INFO ] 0 units remain unscheduled | |
| Waiting for CUs to complete ... | |
| 2014:10:04 18:29:55 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d79a' state changed from 'New' to 'PendingExecution'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d79a' state: PendingExecution. | |
| 2014:10:04 18:29:55 radical.pilot.MainProcess: [DEBUG ] PTYProcess del <saga.utils.pty_process.PTYProcess object at 0x2e2eed0> | |
| 2014:10:04 18:29:55 radical.pilot.MainProcess: [DEBUG ] Copying bootstrapper 'file://localhost//users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/bootstrapper/default_bootstrapper.sh' to agent sandbox (sftp://stampede.tacc.utexas.edu/home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796/). | |
| 2014:10:04 18:29:56 radical.pilot.MainProcess: [DEBUG ] PTYProcess init <saga.utils.pty_process.PTYProcess object at 0x2e3f610> | |
| 2014:10:04 18:29:56 radical.pilot.MainProcess: [INFO ] running: /usr/bin/env TERM=vt100 /usr/bin/ssh -t -o IdentityFile=/users/ardita/.ssh/id_rsa -o ControlMaster=no -o ControlPath=/tmp/saga_ssh_ardita_%h_%p.ardi.ctrl ardi@stampede.tacc.utexas.edu | |
| 2014:10:04 18:29:56 radical.pilot.MainProcess: [DEBUG ] write: [ 15] [ 33] ( export PS1='$' ; set prompt='$'\n) | |
| 2014:10:04 18:29:56 radical.pilot.MainProcess: [DEBUG ] read : [ 15] [ 1023] (Last login: Sat Oct 4 12:29:5 ... uides/\nUser News: http:/) | |
| 2014:10:04 18:29:56 radical.pilot.MainProcess: [DEBUG ] read : [ 15] [ 1024] (/www.tacc.utexas.edu/user-serv ... as three parallel file systems) | |
| 2014:10:04 18:29:56 radical.pilot.MainProcess: [DEBUG ] got initial shell prompt (5) (Last login: Sat Oct 4 12:29:52 2014 from jekyll.pharm.nottingham.ac.uk | |
| ------------------------------------------------------------------------------ | |
| Welcome to the Stampede Supercomputer | |
| Texas Advanced Computing Center, The University of Texas at Austin | |
| ------------------------------------------------------------------------------ | |
| ** Unauthorized use/access is prohibited. ** | |
| If you log on to this computer system, you acknowledge your awareness | |
| of and concurrence with the UT Austin Acceptable Use Policy. The | |
| University will prosecute violators to the full extent of the law. | |
| TACC Usage Policies: | |
| http://www.tacc.utexas.edu/user-services/usage-policies/ | |
| ______________________________________________________________________________ | |
| Questions and Problem Reports: | |
| --> XD Projects: help@xsede.org (email) | |
| --> TACC Projects: portal.tacc.utexas.edu (web) | |
| Documentation: http://www.tacc.utexas.edu/user-services/user-guides/ | |
| User News: http://www.tacc.utexas.edu/user-services/user-news/ | |
| ______________________________________________________________________________ | |
| Welcome to Stampede, *please* read these important system notes: | |
| --> Stampede is currently running the SLURM resource manager to | |
| schedule all compute resources. Example SLURM job scripts are | |
| available on the system at /share/doc/slurm | |
| To run an interactive shell, issue: | |
| srun -p development -t 0:30:00 -n 32 --pty /bin/bash -l | |
| To submit a batch job, issue: sbatch job.mpi | |
| To show all queued jobs, issue: showq | |
| To kill a queued job, issue: scancel <jobId> | |
| ) | |
| 2014:10:04 18:29:56 radical.pilot.MainProcess: [DEBUG ] Got initial shell prompt (5) (Last login: Sat Oct 4 12:29:52 2014 from jekyll.pharm.nottingham.ac.uk | |
| ------------------------------------------------------------------------------ | |
| Welcome to the Stampede Supercomputer | |
| Texas Advanced Computing Center, The University of Texas at Austin | |
| ------------------------------------------------------------------------------ | |
| ** Unauthorized use/access is prohibited. ** | |
| If you log on to this computer system, you acknowledge your awareness | |
| of and concurrence with the UT Austin Acceptable Use Policy. The | |
| University will prosecute violators to the full extent of the law. | |
| TACC Usage Policies: | |
| http://www.tacc.utexas.edu/user-services/usage-policies/ | |
| ______________________________________________________________________________ | |
| Questions and Problem Reports: | |
| --> XD Projects: help@xsede.org (email) | |
| --> TACC Projects: portal.tacc.utexas.edu (web) | |
| Documentation: http://www.tacc.utexas.edu/user-services/user-guides/ | |
| User News: http://www.tacc.utexas.edu/user-services/user-news/ | |
| ______________________________________________________________________________ | |
| Welcome to Stampede, *please* read these important system notes: | |
| --> Stampede is currently running the SLURM resource manager to | |
| schedule all compute resources. Example SLURM job scripts are | |
| available on the system at /share/doc/slurm | |
| To run an interactive shell, issue: | |
| srun -p development -t 0:30:00 -n 32 --pty /bin/bash -l | |
| To submit a batch job, issue: sbatch job.mpi | |
| To show all queued jobs, issue: showq | |
| To kill a queued job, issue: scancel <jobId> | |
| ) | |
| 2014:10:04 18:29:56 radical.pilot.MainProcess: [DEBUG ] write: [ 15] [ 47] ( stty -echo ; unset HISTFILE ; exec /bin/sh -i\n) | |
| 2014:10:04 18:29:56 radical.pilot.MainProcess: [DEBUG ] read : [ 15] [ 349] (: $HOME (permanent,\n quota'd, backed-up) $WORK (permanent, quota'd, not backed-up) and\n $SCRATCH (high-speed purged storage). The "cdw" and "cds" aliases\n are provided as a convenience to change to your $WORK and $SCRATCH\n directories, respectively.\n______________________________________________________________________________\n\n) | |
| 2014:10:04 18:29:57 radical.pilot.MainProcess: [DEBUG ] read : [ 15] [ 244] (----------------------- Project balances for user ardi ------------------------\n| Name Avail SUs Expires | Name Avail SUs Expires |\n| TG-MCB090174 104603 2015-09-30 | TG-TRA140016 -81080 2015-05-06 | \n) | |
| 2014:10:04 18:29:57 radical.pilot.MainProcess: [DEBUG ] read : [ 15] [ 405] (-------------------------- Disk quotas for user ardi --------------------------\n| Disk Usage (GB) Limit %Used File Usage Limit %Used |\n| /home1 1.0 5.0 19.52 22440 150000 14.96 |\n| /work 0.0 1024.0 0.00 2651 3000000 0.09 |\n-------------------------------------------------------------------------------\n) | |
| 2014:10:04 18:29:57 radical.pilot.MainProcess: [DEBUG ] read : [ 15] [ 163] (\nTip 21 (See "module help tacc_tips" for features or how to disable)\n\n You can find all installed bio codes by executing\n $ module keyword bio\n\n) | |
| 2014:10:04 18:29:57 radical.pilot.MainProcess: [DEBUG ] read : [ 15] [ 21] (login2.stampede(1)$ $) | |
| 2014:10:04 18:29:57 radical.pilot.MainProcess: [DEBUG ] write: [ 15] [ 100] ( unset PROMPT_COMMAND ; unset HISTFILE ; PS1='PROMPT-$?->'; PS2=''; export PS1 PS2 2>&1 >/dev/null\n) | |
| 2014:10:04 18:29:57 radical.pilot.MainProcess: [DEBUG ] read : [ 15] [ 1] ($) | |
| 2014:10:04 18:29:57 radical.pilot.MainProcess: [DEBUG ] read : [ 15] [ 10] (PROMPT-0->) | |
| 2014:10:04 18:29:57 radical.pilot.MainProcess: [DEBUG ] PTYProcess init <saga.utils.pty_process.PTYProcess object at 0x2e3cd10> | |
| 2014:10:04 18:29:57 radical.pilot.MainProcess: [INFO ] running: /usr/bin/env TERM=vt100 /usr/bin/sftp -o IdentityFile=/users/ardita/.ssh/id_rsa -o ControlMaster=no -o ControlPath=/tmp/saga_ssh_ardita_%h_%p.ardi.ctrl ardi@stampede.tacc.utexas.edu | |
| 2014:10:04 18:29:57 radical.pilot.MainProcess: [DEBUG ] write: [ 16] [ 33] ( export PS1='$' ; set prompt='$'\n) | |
| 2014:10:04 18:29:57 radical.pilot.MainProcess: [DEBUG ] read : [ 16] [ 43] (Connecting to stampede.tacc.utexas.edu...\n) | |
| 2014:10:04 18:29:59 radical.pilot.MainProcess: [DEBUG ] read : [ 16] [ 64] (sftp> export PS1='$' ; set prompt='$'\nInvalid command.\nsftp> ) | |
| 2014:10:04 18:29:59 radical.pilot.MainProcess: [DEBUG ] got initial shell prompt (5) (Connecting to stampede.tacc.utexas.edu... | |
| sftp> export PS1='$' ; set prompt='$' | |
| Invalid command. | |
| sftp> ) | |
| 2014:10:04 18:29:59 radical.pilot.MainProcess: [DEBUG ] Got initial shell prompt (5) (Connecting to stampede.tacc.utexas.edu... | |
| sftp> export PS1='$' ; set prompt='$' | |
| Invalid command. | |
| sftp> ) | |
| 2014:10:04 18:29:59 radical.pilot.MainProcess: [DEBUG ] write: [ 16] [ 194] (mput "//users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/bootstrapper/default_bootstrapper.sh" "/home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796" \n\n) | |
| 2014:10:04 18:29:59 radical.pilot.MainProcess: [DEBUG ] read : [ 16] [ 198] (mput "//users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/bootstrapper/default_bootstrapper.sh" "/home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796" \n) | |
| 2014:10:04 18:29:59 radical.pilot.MainProcess: [DEBUG ] read : [ 16] [ 220] (Uploading //users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/bootstrapper/default_bootstrapper.sh to /home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796/default_bootstrapper.sh\n) | |
| 2014:10:04 18:29:59 radical.pilot.MainProcess: [DEBUG ] read : [ 16] [ 79] (//users/ardita/ExTASY-toolsOct2/lib/python2.6 0% 0 0.0KB/s --:-- ETA) | |
| 2014:10:04 18:29:59 radical.pilot.MainProcess: [DEBUG ] read : [ 16] [ 81] (//users/ardita/ExTASY-toolsOct2/lib/python2.6 100% 14KB 14.1KB/s 00:00 \n) | |
| 2014:10:04 18:29:59 radical.pilot.MainProcess: [DEBUG ] read : [ 16] [ 14] (sftp> \nsftp> ) | |
| 2014:10:04 18:29:59 radical.pilot.MainProcess: [DEBUG ] copy done: ['sftp>'] | |
| 2014:10:04 18:29:59 radical.pilot.MainProcess: [DEBUG ] Copying agent 'file://localhost//users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/agent/radical-pilot-agent-multicore.py' to agent sandbox (sftp://stampede.tacc.utexas.edu/home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796/). | |
| 2014:10:04 18:30:00 radical.pilot.MainProcess: [DEBUG ] write: [ 16] [ 219] (mput "//users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/agent/radical-pilot-agent-multicore.py" "/home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796/radical-pilot-agent.py" \n\n) | |
| 2014:10:04 18:30:00 radical.pilot.MainProcess: [DEBUG ] read : [ 16] [ 223] (mput "//users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/agent/radical-pilot-agent-multicore.py" "/home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796/radical-pilot-agent.py" \n) | |
| 2014:10:04 18:30:00 radical.pilot.MainProcess: [DEBUG ] read : [ 16] [ 221] (Uploading //users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/agent/radical-pilot-agent-multicore.py to /home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796/radical-pilot-agent.py\n) | |
| 2014:10:04 18:30:00 radical.pilot.MainProcess: [DEBUG ] read : [ 16] [ 79] (//users/ardita/ExTASY-toolsOct2/lib/python2.6 0% 0 0.0KB/s --:-- ETA) | |
| 2014:10:04 18:30:01 radical.pilot.MainProcess: [DEBUG ] read : [ 16] [ 81] (//users/ardita/ExTASY-toolsOct2/lib/python2.6 100% 98KB 97.9KB/s 00:01 \n) | |
| 2014:10:04 18:30:01 radical.pilot.MainProcess: [DEBUG ] read : [ 16] [ 14] (sftp> \nsftp> ) | |
| 2014:10:04 18:30:01 radical.pilot.MainProcess: [DEBUG ] copy done: ['sftp>'] | |
| 2014:10:04 18:30:01 radical.pilot.MainProcess: [DEBUG ] saga.job.Service ('slurm+ssh://stampede.tacc.utexas.edu') | |
| 2014:10:04 18:30:01 radical.pilot.MainProcess: [DEBUG ] PTYProcess init <saga.utils.pty_process.PTYProcess object at 0x2e3fd10> | |
| 2014:10:04 18:30:01 radical.pilot.MainProcess: [INFO ] running: /usr/bin/env TERM=vt100 /usr/bin/ssh -t -o IdentityFile=/users/ardita/.ssh/id_rsa -o ControlMaster=no -o ControlPath=/tmp/saga_ssh_ardita_%h_%p.ardi.ctrl ardi@stampede.tacc.utexas.edu | |
| 2014:10:04 18:30:01 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 33] ( export PS1='$' ; set prompt='$'\n) | |
| 2014:10:04 18:30:01 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 1023] (Last login: Sat Oct 4 12:29:5 ... uides/\nUser News: http:/) | |
| 2014:10:04 18:30:01 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 1024] (/www.tacc.utexas.edu/user-serv ... as three parallel file systems) | |
| 2014:10:04 18:30:01 radical.pilot.MainProcess: [DEBUG ] got initial shell prompt (5) (Last login: Sat Oct 4 12:29:56 2014 from jekyll.pharm.nottingham.ac.uk | |
| ------------------------------------------------------------------------------ | |
| Welcome to the Stampede Supercomputer | |
| Texas Advanced Computing Center, The University of Texas at Austin | |
| ------------------------------------------------------------------------------ | |
| ** Unauthorized use/access is prohibited. ** | |
| If you log on to this computer system, you acknowledge your awareness | |
| of and concurrence with the UT Austin Acceptable Use Policy. The | |
| University will prosecute violators to the full extent of the law. | |
| TACC Usage Policies: | |
| http://www.tacc.utexas.edu/user-services/usage-policies/ | |
| ______________________________________________________________________________ | |
| Questions and Problem Reports: | |
| --> XD Projects: help@xsede.org (email) | |
| --> TACC Projects: portal.tacc.utexas.edu (web) | |
| Documentation: http://www.tacc.utexas.edu/user-services/user-guides/ | |
| User News: http://www.tacc.utexas.edu/user-services/user-news/ | |
| ______________________________________________________________________________ | |
| Welcome to Stampede, *please* read these important system notes: | |
| --> Stampede is currently running the SLURM resource manager to | |
| schedule all compute resources. Example SLURM job scripts are | |
| available on the system at /share/doc/slurm | |
| To run an interactive shell, issue: | |
| srun -p development -t 0:30:00 -n 32 --pty /bin/bash -l | |
| To submit a batch job, issue: sbatch job.mpi | |
| To show all queued jobs, issue: showq | |
| To kill a queued job, issue: scancel <jobId> | |
| ) | |
| 2014:10:04 18:30:01 radical.pilot.MainProcess: [DEBUG ] Got initial shell prompt (5) (Last login: Sat Oct 4 12:29:56 2014 from jekyll.pharm.nottingham.ac.uk | |
| ------------------------------------------------------------------------------ | |
| Welcome to the Stampede Supercomputer | |
| Texas Advanced Computing Center, The University of Texas at Austin | |
| ------------------------------------------------------------------------------ | |
| ** Unauthorized use/access is prohibited. ** | |
| If you log on to this computer system, you acknowledge your awareness | |
| of and concurrence with the UT Austin Acceptable Use Policy. The | |
| University will prosecute violators to the full extent of the law. | |
| TACC Usage Policies: | |
| http://www.tacc.utexas.edu/user-services/usage-policies/ | |
| ______________________________________________________________________________ | |
| Questions and Problem Reports: | |
| --> XD Projects: help@xsede.org (email) | |
| --> TACC Projects: portal.tacc.utexas.edu (web) | |
| Documentation: http://www.tacc.utexas.edu/user-services/user-guides/ | |
| User News: http://www.tacc.utexas.edu/user-services/user-news/ | |
| ______________________________________________________________________________ | |
| Welcome to Stampede, *please* read these important system notes: | |
| --> Stampede is currently running the SLURM resource manager to | |
| schedule all compute resources. Example SLURM job scripts are | |
| available on the system at /share/doc/slurm | |
| To run an interactive shell, issue: | |
| srun -p development -t 0:30:00 -n 32 --pty /bin/bash -l | |
| To submit a batch job, issue: sbatch job.mpi | |
| To show all queued jobs, issue: showq | |
| To kill a queued job, issue: scancel <jobId> | |
| ) | |
| 2014:10:04 18:30:01 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 47] ( stty -echo ; unset HISTFILE ; exec /bin/sh -i\n) | |
| 2014:10:04 18:30:01 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 349] (: $HOME (permanent,\n quota'd, backed-up) $WORK (permanent, quota'd, not backed-up) and\n $SCRATCH (high-speed purged storage). The "cdw" and "cds" aliases\n are provided as a convenience to change to your $WORK and $SCRATCH\n directories, respectively.\n______________________________________________________________________________\n\n) | |
| 2014:10:04 18:30:02 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 244] (----------------------- Project balances for user ardi ------------------------\n| Name Avail SUs Expires | Name Avail SUs Expires |\n| TG-MCB090174 104603 2015-09-30 | TG-TRA140016 -81080 2015-05-06 | \n) | |
| 2014:10:04 18:30:02 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 405] (-------------------------- Disk quotas for user ardi --------------------------\n| Disk Usage (GB) Limit %Used File Usage Limit %Used |\n| /home1 1.0 5.0 19.52 22442 150000 14.96 |\n| /work 0.0 1024.0 0.00 2651 3000000 0.09 |\n-------------------------------------------------------------------------------\n) | |
| 2014:10:04 18:30:02 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 318] (\nTip 130 (See "module help tacc_tips" for features or how to disable)\n\n This up shell function can change directory quickly up the directory tree: "up 2" is the same as "cd ../..". Extra bonus points if you understand how the printf trick works.\n up() { cd $(eval "printf '../'%.0s {1..$1}") && pwd }\n\n) | |
| 2014:10:04 18:30:02 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 21] (login2.stampede(1)$ $) | |
| 2014:10:04 18:30:02 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 100] ( unset PROMPT_COMMAND ; unset HISTFILE ; PS1='PROMPT-$?->'; PS2=''; export PS1 PS2 2>&1 >/dev/null\n) | |
| 2014:10:04 18:30:02 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 1] ($) | |
| 2014:10:04 18:30:02 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 10] (PROMPT-0->) | |
| 2014:10:04 18:30:02 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 13] (which squeue\n) | |
| 2014:10:04 18:30:02 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 27] (/usr/bin/squeue\nPROMPT-0->) | |
| 2014:10:04 18:30:02 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 13] (which sbatch\n) | |
| 2014:10:04 18:30:02 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 27] (/usr/bin/sbatch\nPROMPT-0->) | |
| 2014:10:04 18:30:02 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 14] (which scancel\n) | |
| 2014:10:04 18:30:03 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 28] (/usr/bin/scancel\nPROMPT-0->) | |
| 2014:10:04 18:30:03 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 15] (which scontrol\n) | |
| 2014:10:04 18:30:03 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 29] (/usr/bin/scontrol\nPROMPT-0->) | |
| 2014:10:04 18:30:03 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 7] (whoami\n) | |
| 2014:10:04 18:30:03 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 16] (ardi\nPROMPT-0->) | |
| 2014:10:04 18:30:03 radical.pilot.MainProcess: [INFO ] request cleanup for pilot 54302e89e14fa24f7d66d796 | |
| 2014:10:04 18:30:03 radical.pilot.MainProcess: [DEBUG ] Bootstrap command line: /bin/bash ['-l', 'default_bootstrapper.sh', "-n radicalpilot -s 54302e87e14fa24f7d66d794 -p 54302e89e14fa24f7d66d796 -t 10 -d 10 -c 1 -v 0.20 -m ec2-184-72-89-141.compute-1.amazonaws.com:27017 -i /opt/apps/python/epd/7.3.2/bin/python -e 'module purge' -e 'module load TACC' -e 'module load cluster' -e 'module load Linux' -e 'module load mvapich2' -e 'module load python/2.7.3-epd-7.3.2' -e 'module unload xalt' -l SLURM -j SSH -k IBRUN -x luve"] | |
| 2014:10:04 18:30:03 radical.pilot.MainProcess: [DEBUG ] Submitting SAGA job with description: {'Queue': 'normal', 'Executable': '/bin/bash', 'WorkingDirectory': '/home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796', 'Project': 'TG-MCB090174', 'WallTimeLimit': 10, 'Arguments': ['-l', 'default_bootstrapper.sh', "-n radicalpilot -s 54302e87e14fa24f7d66d794 -p 54302e89e14fa24f7d66d796 -t 10 -d 10 -c 1 -v 0.20 -m ec2-184-72-89-141.compute-1.amazonaws.com:27017 -i /opt/apps/python/epd/7.3.2/bin/python -e 'module purge' -e 'module load TACC' -e 'module load cluster' -e 'module load Linux' -e 'module load mvapich2' -e 'module load python/2.7.3-epd-7.3.2' -e 'module unload xalt' -l SLURM -j SSH -k IBRUN -x luve"], 'Error': 'AGENT.STDERR', 'Output': 'AGENT.STDOUT', 'TotalCPUCount': 1} | |
| 2014:10:04 18:30:03 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 80] (mkdir -p /home1/02998/ardi/radical.pilot.sandbox/pilot-54302e89e14fa24f7d66d796\n) | |
| 2014:10:04 18:30:03 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 10] (PROMPT-0->) | |
| 2014:10:04 18:30:03 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 753] (echo "#"'!'"/bin/bash\n#SBATCH ... -k IBRUN -x luve" | sbatch\n) | |
| 2014:10:04 18:30:03 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 203] (-----------------------------------------------------------------\n Welcome to the Stampede Supercomputer \n-----------------------------------------------------------------\n\n) | |
| 2014:10:04 18:30:03 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 81] (--> Verifying valid submit host (login2)...OK\n--> Verifying valid jobname...OK\n) | |
| 2014:10:04 18:30:05 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 38] (--> Enforcing max jobs per user...OK\n) | |
| 2014:10:04 18:30:05 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 70] (--> Verifying availability of your home dir (/home1/02998/ardi)...OK\n) | |
| 2014:10:04 18:30:05 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 69] (--> Verifying availability of your work dir (/work/02998/ardi)...OK\n) | |
| 2014:10:04 18:30:05 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 75] (--> Verifying availability of your scratch dir (/scratch/02998/ardi)...OK\n) | |
| 2014:10:04 18:30:05 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 202] (--> Verifying valid ssh keys...OK\n--> Verifying access to desired queue (normal)...OK\n--> Verifying job request is within current queue limits...OK\n--> Checking available allocation (TG-MCB090174)...) | |
| 2014:10:04 18:30:05 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 4] (OK\n) | |
| 2014:10:04 18:30:06 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 39] (Submitted batch job 4219680\nPROMPT-0->) | |
| 2014:10:04 18:30:06 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 26] (scontrol show job 4219680\n) | |
| 2014:10:04 18:30:06 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 937] (JobId=4219680 Name=SAGAPythonS ... e14fa24f7d66d796\n\nPROMPT-0->) | |
| 2014:10:04 18:30:06 radical.pilot.MainProcess: [DEBUG ] SAGA job submitted with job id [slurm+ssh://stampede.tacc.utexas.edu]-[4219680] | |
| 2014:10:04 18:30:06 radical.pilot.MainProcess: [INFO ] ComputePilot '54302e89e14fa24f7d66d796' state changed from 'Launching' to 'PendingActive'. | |
| [Callback]: ComputePilot '54302e89e14fa24f7d66d796' state: PendingActive. | |
| 2014:10:04 18:30:46 radical.pilot.MainProcess: [INFO ] Performing periodical health check for 54302e89e14fa24f7d66d796 (SAGA job id [slurm+ssh://stampede.tacc.utexas.edu]-[4219680]) | |
| 2014:10:04 18:30:46 radical.pilot.MainProcess: [DEBUG ] PTYProcess init <saga.utils.pty_process.PTYProcess object at 0x2021350> | |
| 2014:10:04 18:30:46 radical.pilot.MainProcess: [INFO ] running: /usr/bin/env TERM=vt100 /usr/bin/ssh -t -o IdentityFile=/users/ardita/.ssh/id_rsa -o ControlMaster=no -o ControlPath=/tmp/saga_ssh_ardita_%h_%p.ardi.ctrl ardi@stampede.tacc.utexas.edu | |
| 2014:10:04 18:30:46 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 33] ( export PS1='$' ; set prompt='$'\n) | |
| 2014:10:04 18:30:46 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 1023] (Last login: Sat Oct 4 12:30:0 ... uides/\nUser News: http:/) | |
| 2014:10:04 18:30:46 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 1024] (/www.tacc.utexas.edu/user-serv ... as three parallel file systems) | |
| 2014:10:04 18:30:46 radical.pilot.MainProcess: [DEBUG ] got initial shell prompt (5) (Last login: Sat Oct 4 12:30:01 2014 from jekyll.pharm.nottingham.ac.uk | |
| ------------------------------------------------------------------------------ | |
| Welcome to the Stampede Supercomputer | |
| Texas Advanced Computing Center, The University of Texas at Austin | |
| ------------------------------------------------------------------------------ | |
| ** Unauthorized use/access is prohibited. ** | |
| If you log on to this computer system, you acknowledge your awareness | |
| of and concurrence with the UT Austin Acceptable Use Policy. The | |
| University will prosecute violators to the full extent of the law. | |
| TACC Usage Policies: | |
| http://www.tacc.utexas.edu/user-services/usage-policies/ | |
| ______________________________________________________________________________ | |
| Questions and Problem Reports: | |
| --> XD Projects: help@xsede.org (email) | |
| --> TACC Projects: portal.tacc.utexas.edu (web) | |
| Documentation: http://www.tacc.utexas.edu/user-services/user-guides/ | |
| User News: http://www.tacc.utexas.edu/user-services/user-news/ | |
| ______________________________________________________________________________ | |
| Welcome to Stampede, *please* read these important system notes: | |
| --> Stampede is currently running the SLURM resource manager to | |
| schedule all compute resources. Example SLURM job scripts are | |
| available on the system at /share/doc/slurm | |
| To run an interactive shell, issue: | |
| srun -p development -t 0:30:00 -n 32 --pty /bin/bash -l | |
| To submit a batch job, issue: sbatch job.mpi | |
| To show all queued jobs, issue: showq | |
| To kill a queued job, issue: scancel <jobId> | |
| ) | |
| 2014:10:04 18:30:46 radical.pilot.MainProcess: [DEBUG ] Got initial shell prompt (5) (Last login: Sat Oct 4 12:30:01 2014 from jekyll.pharm.nottingham.ac.uk | |
| ------------------------------------------------------------------------------ | |
| Welcome to the Stampede Supercomputer | |
| Texas Advanced Computing Center, The University of Texas at Austin | |
| ------------------------------------------------------------------------------ | |
| ** Unauthorized use/access is prohibited. ** | |
| If you log on to this computer system, you acknowledge your awareness | |
| of and concurrence with the UT Austin Acceptable Use Policy. The | |
| University will prosecute violators to the full extent of the law. | |
| TACC Usage Policies: | |
| http://www.tacc.utexas.edu/user-services/usage-policies/ | |
| ______________________________________________________________________________ | |
| Questions and Problem Reports: | |
| --> XD Projects: help@xsede.org (email) | |
| --> TACC Projects: portal.tacc.utexas.edu (web) | |
| Documentation: http://www.tacc.utexas.edu/user-services/user-guides/ | |
| User News: http://www.tacc.utexas.edu/user-services/user-news/ | |
| ______________________________________________________________________________ | |
| Welcome to Stampede, *please* read these important system notes: | |
| --> Stampede is currently running the SLURM resource manager to | |
| schedule all compute resources. Example SLURM job scripts are | |
| available on the system at /share/doc/slurm | |
| To run an interactive shell, issue: | |
| srun -p development -t 0:30:00 -n 32 --pty /bin/bash -l | |
| To submit a batch job, issue: sbatch job.mpi | |
| To show all queued jobs, issue: showq | |
| To kill a queued job, issue: scancel <jobId> | |
| ) | |
| 2014:10:04 18:30:46 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 47] ( stty -echo ; unset HISTFILE ; exec /bin/sh -i\n) | |
| 2014:10:04 18:30:46 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 349] (: $HOME (permanent,\n quota'd, backed-up) $WORK (permanent, quota'd, not backed-up) and\n $SCRATCH (high-speed purged storage). The "cdw" and "cds" aliases\n are provided as a convenience to change to your $WORK and $SCRATCH\n directories, respectively.\n______________________________________________________________________________\n\n) | |
| 2014:10:04 18:30:47 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 244] (----------------------- Project balances for user ardi ------------------------\n| Name Avail SUs Expires | Name Avail SUs Expires |\n| TG-MCB090174 104603 2015-09-30 | TG-TRA140016 -81080 2015-05-06 | \n) | |
| 2014:10:04 18:30:47 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 405] (-------------------------- Disk quotas for user ardi --------------------------\n| Disk Usage (GB) Limit %Used File Usage Limit %Used |\n| /home1 1.0 5.0 19.55 24716 150000 16.48 |\n| /work 0.0 1024.0 0.00 2651 3000000 0.09 |\n-------------------------------------------------------------------------------\n) | |
| 2014:10:04 18:30:47 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 248] (\nTip 26 (See "module help tacc_tips" for features or how to disable)\n\n The "settarg" module stores key environment information in the variable TARG that dynamically changes as modules are changed. See "module help settarg" for more info.\n\n) | |
| 2014:10:04 18:30:47 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 21] (login2.stampede(1)$ $) | |
| 2014:10:04 18:30:47 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 100] ( unset PROMPT_COMMAND ; unset HISTFILE ; PS1='PROMPT-$?->'; PS2=''; export PS1 PS2 2>&1 >/dev/null\n) | |
| 2014:10:04 18:30:47 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 1] ($) | |
| 2014:10:04 18:30:47 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 10] (PROMPT-0->) | |
| 2014:10:04 18:30:47 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 13] (which squeue\n) | |
| 2014:10:04 18:30:48 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 27] (/usr/bin/squeue\nPROMPT-0->) | |
| 2014:10:04 18:30:48 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 13] (which sbatch\n) | |
| 2014:10:04 18:30:48 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 27] (/usr/bin/sbatch\nPROMPT-0->) | |
| 2014:10:04 18:30:48 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 14] (which scancel\n) | |
| 2014:10:04 18:30:48 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 28] (/usr/bin/scancel\nPROMPT-0->) | |
| 2014:10:04 18:30:48 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 15] (which scontrol\n) | |
| 2014:10:04 18:30:48 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 19] (/usr/bin/scontrol\n) | |
| 2014:10:04 18:30:48 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 10] (PROMPT-0->) | |
| 2014:10:04 18:30:48 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 7] (whoami\n) | |
| 2014:10:04 18:30:48 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 16] (ardi\nPROMPT-0->) | |
| 2014:10:04 18:30:48 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 26] (scontrol show job 4219680\n) | |
| 2014:10:04 18:30:49 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 937] (JobId=4219680 Name=SAGAPythonS ... e14fa24f7d66d796\n\nPROMPT-0->) | |
| 2014:10:04 18:30:49 radical.pilot.MainProcess: [DEBUG ] write: [ 17] [ 26] (scontrol show job 4219680\n) | |
| 2014:10:04 18:30:49 radical.pilot.MainProcess: [DEBUG ] read : [ 17] [ 937] (JobId=4219680 Name=SAGAPythonS ... e14fa24f7d66d796\n\nPROMPT-0->) | |
| 2014:10:04 18:30:49 radical.pilot.MainProcess: [INFO ] pilot 54302e89e14fa24f7d66d796 seems alive and well | |
| 2014:10:04 18:30:51 radical.pilot.MainProcess: [INFO ] ComputePilot '54302e89e14fa24f7d66d796' state changed from 'PendingActive' to 'Active'. | |
| [Callback]: ComputePilot '54302e89e14fa24f7d66d796' state: Active. | |
| 2014:10:04 18:30:51 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d799' state changed from 'PendingExecution' to 'Scheduling'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d799' state: Scheduling. | |
| 2014:10:04 18:30:51 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d799' state changed from 'Scheduling' to 'Executing'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d799' state: Executing. | |
| 2014:10:04 18:30:51 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d79b' state changed from 'PendingExecution' to 'Scheduling'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d79b' state: Scheduling. | |
| 2014:10:04 18:30:51 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d79b' state changed from 'Scheduling' to 'Executing'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d79b' state: Executing. | |
| 2014:10:04 18:30:51 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d79c' state changed from 'PendingExecution' to 'Scheduling'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d79c' state: Scheduling. | |
| 2014:10:04 18:30:51 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d79c' state changed from 'Scheduling' to 'Executing'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d79c' state: Executing. | |
| 2014:10:04 18:30:51 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d79f' state changed from 'PendingExecution' to 'Scheduling'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d79f' state: Scheduling. | |
| 2014:10:04 18:30:51 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d79f' state changed from 'Scheduling' to 'Executing'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d79f' state: Executing. | |
| 2014:10:04 18:30:51 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d79d' state changed from 'PendingExecution' to 'Scheduling'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d79d' state: Scheduling. | |
| 2014:10:04 18:30:51 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d79d' state changed from 'Scheduling' to 'Executing'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d79d' state: Executing. | |
| 2014:10:04 18:30:51 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d798' state changed from 'PendingExecution' to 'Scheduling'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d798' state: Scheduling. | |
| 2014:10:04 18:30:52 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d798' state changed from 'Scheduling' to 'Executing'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d798' state: Executing. | |
| 2014:10:04 18:30:52 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d7a0' state changed from 'PendingExecution' to 'Scheduling'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d7a0' state: Scheduling. | |
| 2014:10:04 18:30:52 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d7a0' state changed from 'Scheduling' to 'Executing'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d7a0' state: Executing. | |
| 2014:10:04 18:30:52 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d7a1' state changed from 'PendingExecution' to 'Scheduling'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d7a1' state: Scheduling. | |
| 2014:10:04 18:30:52 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d7a1' state changed from 'Scheduling' to 'Executing'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d7a1' state: Executing. | |
| 2014:10:04 18:30:52 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d79e' state changed from 'PendingExecution' to 'Scheduling'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d79e' state: Scheduling. | |
| 2014:10:04 18:30:52 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d799' state changed from 'Executing' to 'Done'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d799' state: Done. | |
| 2014:10:04 18:30:52 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d79e' state changed from 'Scheduling' to 'Executing'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d79e' state: Executing. | |
| 2014:10:04 18:30:52 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d79a' state changed from 'PendingExecution' to 'Scheduling'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d79a' state: Scheduling. | |
| 2014:10:04 18:30:52 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d79b' state changed from 'Executing' to 'Done'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d79b' state: Done. | |
| 2014:10:04 18:30:52 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d79c' state changed from 'Executing' to 'Done'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d79c' state: Done. | |
| 2014:10:04 18:30:52 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d79f' state changed from 'Executing' to 'Done'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d79f' state: Done. | |
| 2014:10:04 18:30:52 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d79d' state changed from 'Executing' to 'Done'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d79d' state: Done. | |
| 2014:10:04 18:30:53 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d798' state changed from 'Executing' to 'Done'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d798' state: Done. | |
| 2014:10:04 18:30:53 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d7a0' state changed from 'Executing' to 'Done'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d7a0' state: Done. | |
| 2014:10:04 18:30:53 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d7a1' state changed from 'Executing' to 'Done'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d7a1' state: Done. | |
| 2014:10:04 18:30:53 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d79a' state changed from 'Scheduling' to 'Executing'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d79a' state: Executing. | |
| 2014:10:04 18:30:54 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e91e14fa24f7d66d79e' state changed from 'Executing' to 'Done'. | |
| [Callback]: ComputeUnit '54302e91e14fa24f7d66d79e' state: Done. | |
| 2014:10:04 18:30:54 radical.pilot.MainProcess: [INFO ] RUN ComputeUnit '54302e90e14fa24f7d66d79a' state changed from 'Executing' to 'Done'. | |
| [Callback]: ComputeUnit '54302e90e14fa24f7d66d79a' state: Done. | |
| All CUs completed successfully! | |
| Closing session, exiting now ... | |
| 2014:10:04 18:30:54 radical.pilot.MainProcess: [INFO ] Sent 'COMMAND_CANCEL_PILOT' command to all pilots. | |
| 2014:10:04 18:30:56 radical.pilot.MainProcess: [INFO ] ComputePilot '54302e89e14fa24f7d66d796' state changed from 'Active' to 'Canceled'. | |
| [Callback]: ComputePilot '54302e89e14fa24f7d66d796' state: Canceled. | |
| 2014:10:04 18:30:56 radical.pilot.MainProcess: [INFO ] Sent 'COMMAND_CANCEL_PILOT' command to all pilots. | |
| 2014:10:04 18:30:56 radical.pilot.MainProcess: [DEBUG ] PilotManager.close(): PilotLauncherWorker-1 terminated. | |
| 2014:10:04 18:30:56 radical.pilot.MainProcess: [DEBUG ] Worker thread (ID: Thread-1[139822018823936]) for PilotManager 54302e89e14fa24f7d66d795 stopped. | |
| 2014:10:04 18:30:56 radical.pilot.MainProcess: [INFO ] Closed PilotManager 54302e89e14fa24f7d66d795. | |
| 2014:10:04 18:30:56 radical.pilot.MainProcess: [DEBUG ] UnitManager.close(): InputFileTransferWorker-1 terminated. | |
| 2014:10:04 18:30:56 radical.pilot.MainProcess: [DEBUG ] UnitManager.close(): InputFileTransferWorker-2 terminated. | |
| 2014:10:04 18:30:56 radical.pilot.MainProcess: [DEBUG ] UnitManager.close(): OutputFileTransferWorker-1 terminated. | |
| 2014:10:04 18:30:56 radical.pilot.MainProcess: [DEBUG ] UnitManager.close(): OutputFileTransferWorker-2 terminated. | |
| 2014:10:04 18:30:56 radical.pilot.MainProcess: [DEBUG ] Worker thread (ID: Thread-3[139821756434176]) for UnitManager 54302e90e14fa24f7d66d797 stopped. | |
| 2014:10:04 18:30:56 radical.pilot.MainProcess: [INFO ] Closed UnitManager 54302e90e14fa24f7d66d797. | |
| 2014:10:04 18:30:57 radical.pilot.MainProcess: [INFO ] Deleted session 54302e87e14fa24f7d66d794 from database. | |
| 2014:10:04 18:30:57 radical.pilot.MainProcess: [INFO ] Closed Session 54302e87e14fa24f7d66d794. | |
| Exception in thread InputFileTransferWorker-1 (most likely raised during interpreter shutdown): | |
| Traceback (most recent call last): | |
| File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/controller/input_file_transfer_worker.py", line 232, in run | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/collection.py", line 1677, in find_and_modify | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/database.py", line 445, in command | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/database.py", line 345, in _command | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/cursor.py", line 1058, in next | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/cursor.py", line 1002, in _refresh | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/cursor.py", line 915, in __send_message | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/mongo_client.py", line 1207, in _send_message_with_response | |
| <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'error' | |
| Exception in thread OutputFileTransferWorker-1 (most likely raised during interpreter shutdown): | |
| Traceback (most recent call last): | |
| File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/radical/pilot/controller/output_file_transfer_worker.py", line 236, in run | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/collection.py", line 1677, in find_and_modify | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/database.py", line 445, in command | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/database.py", line 345, in _command | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/cursor.py", line 1058, in next | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/cursor.py", line 1002, in _refresh | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/cursor.py", line 915, in __send_message | |
| File "/users/ardita/ExTASY-toolsOct2/lib/python2.6/site-packages/pymongo/mongo_client.py", line 1207, in _send_message_with_response | |
| <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'error' |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment