Created
December 15, 2014 03:55
-
-
Save stharrold/f176a9071fe20a02678a to your computer and use it in GitHub Desktop.
20141213T102000_example_TESTING_5GB_fastq_again.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ubuntu@20140918t192000-irodsv331:~$ du --bytes /tmp/5GB.fastq | |
| 5000000065 /tmp/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ # Test iRODS temporary directory | |
| ubuntu@20140918t192000-irodsv331:~$ du --bytes /tmp/5GB.fastq | |
| 5000000065 /tmp/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:00:57 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ iput /tmp/5GB.fastq $ITMP_IPLANT/. | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:02:09 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ icp $ITMP_IPLANT/5GB.fastq $ITMP_IPLANT/5GB_copied.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:05:10 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ imv $ITMP_IPLANT/5GB_copied.fastq $ITMP_IPLANT/5GB_moved.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:05:10 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ iget $ITMP_IPLANT/5GB_moved.fastq $TMP_IPLANT/. | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:07:54 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ cksum /tmp/5GB.fastq | |
| 2123744973 5000000065 /tmp/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:08:23 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ cksum $TMP_IPLANT/5GB_moved.fastq | |
| 2123744973 5000000065 /tmp/iplant/5GB_moved.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:09:11 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ irm -f $ITMP_IPLANT/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:09:11 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ irm -f $ITMP_IPLANT/5GB_moved.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:09:12 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ rm -f $TMP_IPLANT/5GB_moved.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:09:12 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ # Read `$IRODS/server/log/rodsLog.YYYY.MM.DD` between the timestamps from `date` to check execution. | |
| ubuntu@20140918t192000-irodsv331:~$ less ~/iRODS/server/log/rodsLog.2014.12.11 | |
| ubuntu@20140918t192000-irodsv331:~$ tail -40 /home/ubuntu/iRODS/server/log/rodsLog.2014.12.11 | |
| Dec 13 03:17:23 pid:29364 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 13 03:17:23 pid:29364 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 13 03:17:23 pid:29364 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 13 03:17:23 pid:29364 NOTICE: Agent exiting with status = 0 | |
| Dec 13 16:00:57 pid:1649 NOTICE: Agent process 29364 exited with status 0 | |
| Dec 13 16:00:57 pid:1649 NOTICE: Agent process 9270 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 13 16:00:57 pid:9270 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 13 16:00:57 pid:9270 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 13 16:02:09 pid:9270 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 13 16:02:09 pid:9270 NOTICE: Agent exiting with status = 0 | |
| Dec 13 16:02:09 pid:1649 NOTICE: Agent process 9270 exited with status 0 | |
| Dec 13 16:02:09 pid:1649 NOTICE: Agent process 9329 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 13 16:02:09 pid:9329 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 13 16:02:09 pid:9329 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 13 16:05:10 pid:9329 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 13 16:05:10 pid:9329 NOTICE: Agent exiting with status = 0 | |
| Dec 13 16:05:10 pid:1649 NOTICE: Agent process 9329 exited with status 0 | |
| Dec 13 16:05:10 pid:1649 NOTICE: Agent process 9385 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 13 16:05:10 pid:9385 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 13 16:05:10 pid:9385 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 13 16:05:10 pid:9385 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 13 16:05:10 pid:9385 NOTICE: Agent exiting with status = 0 | |
| Dec 13 16:05:10 pid:1649 NOTICE: Agent process 9385 exited with status 0 | |
| Dec 13 16:05:10 pid:1649 NOTICE: Agent process 9389 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 13 16:05:10 pid:9389 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 13 16:05:10 pid:9389 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 13 16:07:54 pid:9389 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 13 16:07:54 pid:9389 NOTICE: Agent exiting with status = 0 | |
| Dec 13 16:09:11 pid:1649 NOTICE: Agent process 9389 exited with status 0 | |
| Dec 13 16:09:11 pid:1649 NOTICE: Agent process 9499 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 13 16:09:11 pid:9499 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 13 16:09:11 pid:9499 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 13 16:09:11 pid:9499 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 13 16:09:11 pid:9499 NOTICE: Agent exiting with status = 0 | |
| Dec 13 16:09:11 pid:1649 NOTICE: Agent process 9499 exited with status 0 | |
| Dec 13 16:09:11 pid:1649 NOTICE: Agent process 9503 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 13 16:09:11 pid:9503 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 13 16:09:11 pid:9503 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 13 16:09:12 pid:9503 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 13 16:09:12 pid:9503 NOTICE: Agent exiting with status = 0 | |
| ubuntu@20140918t192000-irodsv331:~$ # SUCCESS | |
| ubuntu@20140918t192000-irodsv331:~$ | |
| ubuntu@20140918t192000-irodsv331:~$ cp -i /home/ubuntu/iRODS/server/config/reConfigs/core.re /home/ubuntu/iRODS/server/config/reConfigs/core.re_BACKUP_20141213T102000 | |
| ubuntu@20140918t192000-irodsv331:~$ cp -i /home/ubuntu/iRODS/server/config/reConfigs/core.re_BACKUP_ORIGINAL_20141011T135000 /home/ubuntu/iRODS/server/config/reConfigs/core.re | |
| cp: overwrite ‘/home/ubuntu/iRODS/server/config/reConfigs/core.re’? y | |
| ubuntu@20140918t192000-irodsv331:~$ | |
| ubuntu@20140918t192000-irodsv331:~$ REPO=~/irods_module_iplant # Example local directory for cloned git repository. | |
| ubuntu@20140918t192000-irodsv331:~$ IRODS=~/iRODS # Example local root directory for iRODS installation. | |
| ubuntu@20140918t192000-irodsv331:~$ IPLANT=/tempZone/home/rods/iplant # Example iRODS directory to save iplant data. | |
| ubuntu@20140918t192000-irodsv331:~$ ITMP_IPLANT=/tempZone/tmp/iplant # Example iRODS directory to save temporary files. | |
| ubuntu@20140918t192000-irodsv331:~$ imkdir -p $ITMP_IPLANT # Create iRODS temporary directory. | |
| ubuntu@20140918t192000-irodsv331:~$ TMP_IPLANT=/tmp/iplant # Example local directory to save tempoary files. | |
| ubuntu@20140918t192000-irodsv331:~$ mkdir -p $TMP_IPLANT # Create local temporary directory. | |
| ubuntu@20140918t192000-irodsv331:~$ IPLANT_LOG=/tmp/iplant/iplant.log # Example local path to save iPlant log file. | |
| ubuntu@20140918t192000-irodsv331:~$ | |
| ubuntu@20140918t192000-irodsv331:~$ # Test iplant.py without automatic calls from iRODS | |
| ubuntu@20140918t192000-irodsv331:~$ # Use the original version of `core.re` from before following `INSTALL.md`: $IRODS/server/config/reConfigs/core.re_BACKUP_YYYYMMDDTHHMMSS | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:34:31 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ iput /tmp/5GB.fastq $IPLANT/. | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:35:58 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ $IRODS/server/bin/cmd/iplant.py --ipath $IPLANT/5GB.fastq --iplant $IPLANT --action compress --itmp_iplant $ITMP_IPLANT --tmp_iplant $TMP_IPLANT --logging_level DEBUG --log_file $IPLANT_LOG | |
| INFO: Arguments: | |
| Namespace(action='compress', delete_itmp_files=False, delete_tmp_files=False, ipath='/tempZone/home/rods/iplant/5GB.fastq', iplant='/tempZone/home/rods/iplant', itmp_iplant='/tempZone/tmp/iplant', log_file='/tmp/iplant/iplant.log', logging_level='DEBUG', test=False, tmp_iplant='/tmp/iplant') | |
| INFO: --ipath is contained within --iplant. | |
| --ipath /tempZone/home/rods/iplant/5GB.fastq | |
| --iplant /tempZone/home/rods/iplant | |
| "2014-12-13 16:35:59,160","__main__","INFO","main: BEGIN_LOGGING" | |
| "2014-12-13 16:35:59,189","__main__","INFO","main: Log format: '%(asctime)s','%(name)s','%(levelname)s','%(message)s'" | |
| "2014-12-13 16:35:59,189","__main__","INFO","main: Log date format: default ISO 8601, UTC" | |
| "2014-12-13 16:35:59,189","__main__","INFO","main: Compressing file." | |
| "2014-12-13 16:35:59,190","__main__","DEBUG","main: compress(ipath=/tempZone/home/rods/iplant/5GB.fastq, itmp_iplant=/tempZone/tmp/iplant, tmp_iplant=/tmp/iplant, delete_itmp_files=False, delete_tmp_files=False)" | |
| "2014-12-13 16:35:59,190","__main__","DEBUG","compress: imeta ls -d /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-13 16:35:59,255","__main__","DEBUG","compress: imeta_stdout = | |
| AVUs defined for dataObj /tempZone/home/rods/iplant/5GB.fastq: | |
| None | |
| " | |
| "2014-12-13 16:35:59,256","__main__","DEBUG","compress: _imeta_to_dict(imeta_stdout) = | |
| {}" | |
| "2014-12-13 16:35:59,257","__main__","DEBUG","compress: do_compress = True" | |
| "2014-12-13 16:35:59,257","__main__","DEBUG","compress: imv /tempZone/home/rods/iplant/5GB.fastq /tempZone/tmp/iplant/20141213T163559.257774_5GB.fastq" | |
| "2014-12-13 16:35:59,410","__main__","DEBUG","compress: iget -f -T /tempZone/tmp/iplant/20141213T163559.257774_5GB.fastq /tmp/iplant/20141213T163559.257774_5GB.fastq" | |
| "2014-12-13 16:38:45,052","__main__","DEBUG","compress: uncompressed_size = os.path.getsize(/tmp/iplant/20141213T163559.257774_5GB.fastq)" | |
| "2014-12-13 16:38:45,052","__main__","DEBUG","compress: uncompressed_size = 5000000065" | |
| "2014-12-13 16:38:45,053","__main__","DEBUG","compress: uncompressed_hash = _compute_hash(fpath=/tmp/iplant/20141213T163559.257774_5GB.fastq, algorithm=SHA1)" | |
| "2014-12-13 16:39:24,685","__main__","DEBUG","compress: uncompressed_hash = a13fa24d16155850e85fb94aad6417e830f963dc" | |
| "2014-12-13 16:39:24,686","__main__","DEBUG","compress: gzip --fast --force --keep /tmp/iplant/20141213T163559.257774_5GB.fastq" | |
| "2014-12-13 16:41:17,342","__main__","DEBUG","compress: iput -T /tmp/iplant/20141213T163559.257774_5GB.fastq.gz /tempZone/tmp/iplant/20141213T163559.257774_5GB.fastq.gz" | |
| "2014-12-13 16:41:39,849","__main__","DEBUG","compress: icp /tempZone/tmp/iplant/20141213T163559.257774_5GB.fastq.gz /tempZone/tmp/iplant/20141213T163559.257774_5GB.fastq.gz_copy" | |
| "2014-12-13 16:41:44,361","__main__","DEBUG","compress: imv /tempZone/tmp/iplant/20141213T163559.257774_5GB.fastq.gz_copy /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-13 16:41:46,798","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq IS_COMPRESSED TRUE BOOL" | |
| "2014-12-13 16:41:48,520","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMPRESSION_METHOD GZIP NONE" | |
| "2014-12-13 16:41:49,223","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_SIZE 5000000065 BYTES" | |
| "2014-12-13 16:41:49,890","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_HASH a13fa24d16155850e85fb94aad6417e830f963dc NONE" | |
| "2014-12-13 16:41:49,981","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq HASH_METHOD SHA1 NONE" | |
| "2014-12-13 16:41:50,048","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq PARENT_FILE /tempZone/tmp/iplant/20141213T163559.257774_5GB.fastq NONE" | |
| "2014-12-13 16:41:50,148","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMMENTS 'This file is registered under the extension .fastq but is stored internally to iRODS with compression as .fastq.gz. This file will be decompressed upon retrieval (e.g. with iget, isync).' NONE" | |
| "2014-12-13 16:41:50,214","__main__","INFO","main: END_LOGGING" | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:41:50 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ $IRODS/server/bin/cmd/iplant.py --ipath $IPLANT/5GB.fastq --iplant $IPLANT --action decompress --itmp_iplant $ITMP_IPLANT --tmp_iplant $TMP_IPLANT --logging_level DEBUG --log_file $IPLANT_LOG | |
| INFO: Arguments: | |
| Namespace(action='decompress', delete_itmp_files=False, delete_tmp_files=False, ipath='/tempZone/home/rods/iplant/5GB.fastq', iplant='/tempZone/home/rods/iplant', itmp_iplant='/tempZone/tmp/iplant', log_file='/tmp/iplant/iplant.log', logging_level='DEBUG', test=False, tmp_iplant='/tmp/iplant') | |
| INFO: --ipath is contained within --iplant. | |
| --ipath /tempZone/home/rods/iplant/5GB.fastq | |
| --iplant /tempZone/home/rods/iplant | |
| "2014-12-13 16:41:50,770","__main__","INFO","main: BEGIN_LOGGING" | |
| "2014-12-13 16:41:50,771","__main__","INFO","main: Log format: '%(asctime)s','%(name)s','%(levelname)s','%(message)s'" | |
| "2014-12-13 16:41:50,772","__main__","INFO","main: Log date format: default ISO 8601, UTC" | |
| "2014-12-13 16:41:50,772","__main__","INFO","main: Decompressing file." | |
| "2014-12-13 16:41:50,772","__main__","DEBUG","main: decompress(ipath=/tempZone/home/rods/iplant/5GB.fastq, itmp_iplant=/tempZone/tmp/iplant, tmp_iplant=/tmp/iplant, delete_itmp_files=False, delete_tmp_files=False)" | |
| "2014-12-13 16:41:50,773","__main__","DEBUG","decompress: imeta ls -d /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-13 16:41:50,819","__main__","DEBUG","decompress: imeta_stdout = | |
| AVUs defined for dataObj /tempZone/home/rods/iplant/5GB.fastq: | |
| attribute: IS_COMPRESSED | |
| value: TRUE | |
| units: BOOL | |
| ---- | |
| attribute: PARENT_FILE | |
| value: /tempZone/tmp/iplant/20141213T163559.257774_5GB.fastq | |
| units: NONE | |
| ---- | |
| attribute: COMMENTS | |
| value: 'This file is registered under the extension .fastq but is stored internally to iRODS with compression as .fastq.gz. This file will be decompressed upon retrieval (e.g. with iget, isync).' | |
| units: NONE | |
| ---- | |
| attribute: UNCOMPRESSED_HASH | |
| value: a13fa24d16155850e85fb94aad6417e830f963dc | |
| units: NONE | |
| ---- | |
| attribute: HASH_METHOD | |
| value: SHA1 | |
| units: NONE | |
| ---- | |
| attribute: COMPRESSION_METHOD | |
| value: GZIP | |
| units: NONE | |
| ---- | |
| attribute: UNCOMPRESSED_SIZE | |
| value: 5000000065 | |
| units: BYTES | |
| " | |
| "2014-12-13 16:41:50,820","__main__","DEBUG","decompress: _imeta_to_dict(imeta_stdout) = | |
| {'PARENT_FILE': {'units': 'NONE', 'value': '/tempZone/tmp/iplant/20141213T163559.257774_5GB.fastq'}, 'COMMENTS': {'units': 'NONE', 'value': "'This file is registered under the extension .fastq but is stored internally to iRODS with compression as .fastq.gz. This file will be decompressed upon retrieval (e.g. with iget, isync).'"}, 'COMPRESSION_METHOD': {'units': 'NONE', 'value': 'GZIP'}, 'IS_COMPRESSED': {'units': 'BOOL', 'value': True}, 'HASH_METHOD': {'units': 'NONE', 'value': 'SHA1'}, 'UNCOMPRESSED_HASH': {'units': 'NONE', 'value': 'a13fa24d16155850e85fb94aad6417e830f963dc'}, 'UNCOMPRESSED_SIZE': {'units': 'BYTES', 'value': 5000000065}}" | |
| "2014-12-13 16:41:50,820","__main__","DEBUG","decompress: do_decompress = True" | |
| "2014-12-13 16:41:50,820","__main__","DEBUG","decompress: imv /tempZone/home/rods/iplant/5GB.fastq /tempZone/tmp/iplant/20141213T164150.820566_5GB.fastq.gz" | |
| "2014-12-13 16:41:50,907","__main__","DEBUG","decompress: iget -f -T /tempZone/tmp/iplant/20141213T164150.820566_5GB.fastq.gz /tmp/iplant/20141213T164150.820566_5GB.fastq.gz" | |
| "2014-12-13 16:41:57,766","__main__","DEBUG","decompress: gunzip --force --keep /tmp/iplant/20141213T164150.820566_5GB.fastq.gz" | |
| "2014-12-13 16:43:39,140","__main__","DEBUG","decompress: uncompressed_size = os.path.getsize(/tmp/iplant/20141213T164150.820566_5GB.fastq)" | |
| "2014-12-13 16:43:39,141","__main__","DEBUG","decompress: uncompressed_size = 5000000065" | |
| "2014-12-13 16:43:39,141","__main__","DEBUG","decompress: Uncompressed file size matches 'UNCOMPRESSED_SIZE' from imeta." | |
| "2014-12-13 16:43:39,142","__main__","DEBUG","decompress: uncompressed_hash = _compute_hash(fpath=/tmp/iplant/20141213T164150.820566_5GB.fastq, algorithm=SHA1)" | |
| "2014-12-13 16:44:21,280","__main__","DEBUG","decompress: uncompressed_hash = a13fa24d16155850e85fb94aad6417e830f963dc" | |
| "2014-12-13 16:44:21,280","__main__","DEBUG","decompress: Uncompressed hash matches 'UNCOMPRESSED_HASH' from imeta." | |
| "2014-12-13 16:44:21,281","__main__","DEBUG","decompress: iput -T /tmp/iplant/20141213T164150.820566_5GB.fastq /tempZone/tmp/iplant/20141213T164150.820566_5GB.fastq" | |
| "2014-12-13 16:46:24,442","__main__","DEBUG","decompress: icp /tempZone/tmp/iplant/20141213T164150.820566_5GB.fastq /tempZone/tmp/iplant/20141213T164150.820566_5GB.fastq_copy" | |
| "2014-12-13 16:49:06,464","__main__","DEBUG","decompress: imv /tempZone/tmp/iplant/20141213T164150.820566_5GB.fastq_copy /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-13 16:49:06,764","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq IS_COMPRESSED FALSE BOOL" | |
| "2014-12-13 16:49:06,847","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMPRESSION_METHOD NONE NONE" | |
| "2014-12-13 16:49:06,911","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_SIZE 5000000065 BYTES" | |
| "2014-12-13 16:49:06,980","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_HASH a13fa24d16155850e85fb94aad6417e830f963dc NONE" | |
| "2014-12-13 16:49:07,044","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq HASH_METHOD SHA1 NONE" | |
| "2014-12-13 16:49:07,111","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq PARENT_FILE /tempZone/tmp/iplant/20141213T164150.820566_5GB.fastq.gz NONE" | |
| "2014-12-13 16:49:07,178","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMMENTS 'This file is registered under the extension .fastq and is stored internally to iRODS without compression as .fastq.' NONE" | |
| "2014-12-13 16:49:07,244","__main__","INFO","main: END_LOGGING" | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:49:07 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ iget $IPLANT/5GB.fastq $TMP_IPLANT/. | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 2251784 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 1980 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 1540 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 220 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 880 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 1100 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 2200 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 1760 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 660 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 2860 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 2640 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 3300 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 440 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 1320 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toWrite 4194304, bytesWritten 2420 status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| ERROR: rcPartialDataGet: toGet 25165824, bytesRead 0 status = -27000 SYS_COPY_LEN_ERR | |
| Client Caught broken pipe signal. Connection to server may be down | |
| ERROR: writeMsgHeader: wrote 0 bytes, expect 139, status = -5032 | |
| ERROR: sendApiRequest: sendRodsMsg error, status = -5032 status = -5032 SYS_HEADER_WRITE_LEN_ERR, Broken pipe | |
| ERROR: getUtil: get error for /tmp/iplant/./5GB.fastq status = -27028 SYS_COPY_LEN_ERR, No space left on device | |
| Client Caught broken pipe signal. Connection to server may be down | |
| ERROR: writeMsgHeader: wrote 0 bytes for myLen , expect 4, status = -5032 | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:51:01 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ cksum /tmp/5GB.fastq | |
| 2123744973 5000000065 /tmp/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:51:45 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ cksum $TMP_IPLANT/5GB.fastq | |
| 509273444 4867858432 /tmp/iplant/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:52:30 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ irm -f $IPLANT/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:52:31 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ rm -f $TMP_IPLANT/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 16:52:31 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ # Check file sizes of tmp files. | |
| ubuntu@20140918t192000-irodsv331:~$ # Read `$IPLANT_LOG` between the timestamps from `date` to check execution. | |
| ubuntu@20140918t192000-irodsv331:~$ # Replace `core.re` with the iPlant version from after following `INSTALL.md`. | |
| ubuntu@20140918t192000-irodsv331:~$ # REDO TEST BECAUSE OUT OF SPACE | |
| ubuntu@20140918t192000-irodsv331:~$ | |
| ubuntu@20140918t192000-irodsv331:~$ ils $IPLANT | |
| /tempZone/home/rods/iplant: | |
| ubuntu@20140918t192000-irodsv331:~$ ils $ITMP_IPLANT | |
| /tempZone/tmp/iplant: | |
| 20141213T163559.257774_5GB.fastq | |
| 20141213T163559.257774_5GB.fastq.gz | |
| 20141213T164150.820566_5GB.fastq | |
| 20141213T164150.820566_5GB.fastq.gz | |
| ubuntu@20140918t192000-irodsv331:~$ irm -f # Check file sizes of tmp files. | |
| ERROR: irm: no input | |
| Use -h for help. | |
| ubuntu@20140918t192000-irodsv331:~$ # Read `$IPLANT_LOG` between the timestamps from `date` to check execution. | |
| ubuntu@20140918t192000-irodsv331:~$ | |
| ubuntu@20140918t192000-irodsv331:~$ irm -f $ITMP_IPLANT/20141213T163559.257774_5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ irm -f $ITMP_IPLANT/20141213T163559.257774_5GB.fastq.gz | |
| ubuntu@20140918t192000-irodsv331:~$ irm -f $ITMP_IPLANT/20141213T164150.820566_5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ irm -f $ITMP_IPLANT/20141213T164150.820566_5GB.fastq.gz | |
| ubuntu@20140918t192000-irodsv331:~$ ils $ITMP_IPLANT | |
| /tempZone/tmp/iplant: | |
| ubuntu@20140918t192000-irodsv331:~$ ls $TMP_IPLANT | |
| 20141213T163559.257774_5GB.fastq 20141213T163559.257774_5GB.fastq.gz 20141213T164150.820566_5GB.fastq 20141213T164150.820566_5GB.fastq.gz iplant.log | |
| ubuntu@20140918t192000-irodsv331:~$ ls /tmp/iplant/20141213T16* | |
| /tmp/iplant/20141213T163559.257774_5GB.fastq /tmp/iplant/20141213T163559.257774_5GB.fastq.gz /tmp/iplant/20141213T164150.820566_5GB.fastq /tmp/iplant/20141213T164150.820566_5GB.fastq.gz | |
| ubuntu@20140918t192000-irodsv331:~$ rm -f /tmp/iplant/20141213T16* | |
| ubuntu@20140918t192000-irodsv331:~$ ls /tmp/iplant | |
| iplant.log | |
| ubuntu@20140918t192000-irodsv331:~$ | |
| ubuntu@20140918t192000-irodsv331:~$ | |
| ubuntu@20140918t192000-irodsv331:~$ diff /home/ubuntu/iRODS/server/config/reConfigs/core.re /home/ubuntu/iRODS/server/config/reConfigs/core.re_BACKUP_ORIGINAL_20141011T135000 | |
| ubuntu@20140918t192000-irodsv331:~$ # Test iplant.py without automatic calls from iRODS | |
| ubuntu@20140918t192000-irodsv331:~$ # Use the original version of `core.re` from before following `INSTALL.md`: $IRODS/server/config/reConfigs/core.re_BACKUP_YYYYMMDDTHHMMSS | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 17:01:13 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ iput /tmp/5GB.fastq $IPLANT/. | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 17:03:57 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ $IRODS/server/bin/cmd/iplant.py --ipath $IPLANT/5GB.fastq --iplant $IPLANT --action compress --itmp_iplant $ITMP_IPLANT --tmp_iplant $TMP_IPLANT --logging_level DEBUG --log_file $IPLANT_LOG | |
| INFO: Arguments: | |
| Namespace(action='compress', delete_itmp_files=False, delete_tmp_files=False, ipath='/tempZone/home/rods/iplant/5GB.fastq', iplant='/tempZone/home/rods/iplant', itmp_iplant='/tempZone/tmp/iplant', log_file='/tmp/iplant/iplant.log', logging_level='DEBUG', test=False, tmp_iplant='/tmp/iplant') | |
| INFO: --ipath is contained within --iplant. | |
| --ipath /tempZone/home/rods/iplant/5GB.fastq | |
| --iplant /tempZone/home/rods/iplant | |
| "2014-12-13 17:03:58,373","__main__","INFO","main: BEGIN_LOGGING" | |
| "2014-12-13 17:03:58,373","__main__","INFO","main: Log format: '%(asctime)s','%(name)s','%(levelname)s','%(message)s'" | |
| "2014-12-13 17:03:58,374","__main__","INFO","main: Log date format: default ISO 8601, UTC" | |
| "2014-12-13 17:03:58,374","__main__","INFO","main: Compressing file." | |
| "2014-12-13 17:03:58,374","__main__","DEBUG","main: compress(ipath=/tempZone/home/rods/iplant/5GB.fastq, itmp_iplant=/tempZone/tmp/iplant, tmp_iplant=/tmp/iplant, delete_itmp_files=False, delete_tmp_files=False)" | |
| "2014-12-13 17:03:58,375","__main__","DEBUG","compress: imeta ls -d /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-13 17:03:58,424","__main__","DEBUG","compress: imeta_stdout = | |
| AVUs defined for dataObj /tempZone/home/rods/iplant/5GB.fastq: | |
| None | |
| " | |
| "2014-12-13 17:03:58,425","__main__","DEBUG","compress: _imeta_to_dict(imeta_stdout) = | |
| {}" | |
| "2014-12-13 17:03:58,425","__main__","DEBUG","compress: do_compress = True" | |
| "2014-12-13 17:03:58,425","__main__","DEBUG","compress: imv /tempZone/home/rods/iplant/5GB.fastq /tempZone/tmp/iplant/20141213T170358.425638_5GB.fastq" | |
| "2014-12-13 17:03:58,554","__main__","DEBUG","compress: iget -f -T /tempZone/tmp/iplant/20141213T170358.425638_5GB.fastq /tmp/iplant/20141213T170358.425638_5GB.fastq" | |
| "2014-12-13 17:06:49,753","__main__","DEBUG","compress: uncompressed_size = os.path.getsize(/tmp/iplant/20141213T170358.425638_5GB.fastq)" | |
| "2014-12-13 17:06:49,754","__main__","DEBUG","compress: uncompressed_size = 5000000065" | |
| "2014-12-13 17:06:49,755","__main__","DEBUG","compress: uncompressed_hash = _compute_hash(fpath=/tmp/iplant/20141213T170358.425638_5GB.fastq, algorithm=SHA1)" | |
| "2014-12-13 17:07:18,157","__main__","DEBUG","compress: uncompressed_hash = a13fa24d16155850e85fb94aad6417e830f963dc" | |
| "2014-12-13 17:07:18,157","__main__","DEBUG","compress: gzip --fast --force --keep /tmp/iplant/20141213T170358.425638_5GB.fastq" | |
| "2014-12-13 17:08:45,230","__main__","DEBUG","compress: iput -T /tmp/iplant/20141213T170358.425638_5GB.fastq.gz /tempZone/tmp/iplant/20141213T170358.425638_5GB.fastq.gz" | |
| "2014-12-13 17:08:49,941","__main__","DEBUG","compress: icp /tempZone/tmp/iplant/20141213T170358.425638_5GB.fastq.gz /tempZone/tmp/iplant/20141213T170358.425638_5GB.fastq.gz_copy" | |
| "2014-12-13 17:09:02,049","__main__","DEBUG","compress: imv /tempZone/tmp/iplant/20141213T170358.425638_5GB.fastq.gz_copy /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-13 17:09:05,172","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq IS_COMPRESSED TRUE BOOL" | |
| "2014-12-13 17:09:06,069","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMPRESSION_METHOD GZIP NONE" | |
| "2014-12-13 17:09:06,903","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_SIZE 5000000065 BYTES" | |
| "2014-12-13 17:09:06,993","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_HASH a13fa24d16155850e85fb94aad6417e830f963dc NONE" | |
| "2014-12-13 17:09:07,060","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq HASH_METHOD SHA1 NONE" | |
| "2014-12-13 17:09:07,135","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq PARENT_FILE /tempZone/tmp/iplant/20141213T170358.425638_5GB.fastq NONE" | |
| "2014-12-13 17:09:07,201","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMMENTS 'This file is registered under the extension .fastq but is stored internally to iRODS with compression as .fastq.gz. This file will be decompressed upon retrieval (e.g. with iget, isync).' NONE" | |
| "2014-12-13 17:09:07,268","__main__","INFO","main: END_LOGGING" | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 17:09:07 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ # Check file sizes of tmp files. | |
| ubuntu@20140918t192000-irodsv331:~$ # Read `$IPLANT_LOG` between the timestamps from `date` to check execution. | |
| ubuntu@20140918t192000-irodsv331:~$ ls $TMP_IPLANT | |
| 20141213T170358.425638_5GB.fastq 20141213T170358.425638_5GB.fastq.gz iplant.log | |
| ubuntu@20140918t192000-irodsv331:~$ cksum $TMP_IPLANT/20141213T170358.425638_5GB.fastq | |
| 2123744973 5000000065 /tmp/iplant/20141213T170358.425638_5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ cksum 20141213T170358.425638_5GB.fastq | |
| cksum: 20141213T170358.425638_5GB.fastq: No such file or directory | |
| ubuntu@20140918t192000-irodsv331:~$ cksum /tmp/5GB.fastq | |
| 2123744973 5000000065 /tmp/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ # tmp file is same as input file | |
| ubuntu@20140918t192000-irodsv331:~$ ils $ITMP_IPLANT | |
| /tempZone/tmp/iplant: | |
| 20141213T170358.425638_5GB.fastq | |
| 20141213T170358.425638_5GB.fastq.gz | |
| ubuntu@20140918t192000-irodsv331:~$ irm -f $ITMP_IPLANT/20141213T170358.425638_5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ irm -f $ITMP_IPLANT/20141213T170358.425638_5GB.fastq.gz | |
| ubuntu@20140918t192000-irodsv331:~$ ls $TMP_IPLANT | |
| 20141213T170358.425638_5GB.fastq 20141213T170358.425638_5GB.fastq.gz iplant.log | |
| ubuntu@20140918t192000-irodsv331:~$ rm -f /tmp/iplant/20141213T170358.425638_5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ rm -f /tmp/iplant/20141213T170358.425638_5GB.fastq.gz | |
| ubuntu@20140918t192000-irodsv331:~$ | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 17:41:53 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ $IRODS/server/bin/cmd/iplant.py --ipath $IPLANT/5GB.fastq --iplant $IPLANT --action decompress --itmp_iplant $ITMP_IPLANT --tmp_iplant $TMP_IPLANT --logging_level DEBUG --log_file $IPLANT_LOG | |
| INFO: Arguments: | |
| Namespace(action='decompress', delete_itmp_files=False, delete_tmp_files=False, ipath='/tempZone/home/rods/iplant/5GB.fastq', iplant='/tempZone/home/rods/iplant', itmp_iplant='/tempZone/tmp/iplant', log_file='/tmp/iplant/iplant.log', logging_level='DEBUG', test=False, tmp_iplant='/tmp/iplant') | |
| INFO: --ipath is contained within --iplant. | |
| --ipath /tempZone/home/rods/iplant/5GB.fastq | |
| --iplant /tempZone/home/rods/iplant | |
| "2014-12-13 17:41:54,145","__main__","INFO","main: BEGIN_LOGGING" | |
| "2014-12-13 17:41:54,146","__main__","INFO","main: Log format: '%(asctime)s','%(name)s','%(levelname)s','%(message)s'" | |
| "2014-12-13 17:41:54,146","__main__","INFO","main: Log date format: default ISO 8601, UTC" | |
| "2014-12-13 17:41:54,147","__main__","INFO","main: Decompressing file." | |
| "2014-12-13 17:41:54,147","__main__","DEBUG","main: decompress(ipath=/tempZone/home/rods/iplant/5GB.fastq, itmp_iplant=/tempZone/tmp/iplant, tmp_iplant=/tmp/iplant, delete_itmp_files=False, delete_tmp_files=False)" | |
| "2014-12-13 17:41:54,148","__main__","DEBUG","decompress: imeta ls -d /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-13 17:41:54,201","__main__","DEBUG","decompress: imeta_stdout = | |
| AVUs defined for dataObj /tempZone/home/rods/iplant/5GB.fastq: | |
| attribute: IS_COMPRESSED | |
| value: TRUE | |
| units: BOOL | |
| ---- | |
| attribute: COMMENTS | |
| value: 'This file is registered under the extension .fastq but is stored internally to iRODS with compression as .fastq.gz. This file will be decompressed upon retrieval (e.g. with iget, isync).' | |
| units: NONE | |
| ---- | |
| attribute: UNCOMPRESSED_HASH | |
| value: a13fa24d16155850e85fb94aad6417e830f963dc | |
| units: NONE | |
| ---- | |
| attribute: HASH_METHOD | |
| value: SHA1 | |
| units: NONE | |
| ---- | |
| attribute: PARENT_FILE | |
| value: /tempZone/tmp/iplant/20141213T170358.425638_5GB.fastq | |
| units: NONE | |
| ---- | |
| attribute: COMPRESSION_METHOD | |
| value: GZIP | |
| units: NONE | |
| ---- | |
| attribute: UNCOMPRESSED_SIZE | |
| value: 5000000065 | |
| units: BYTES | |
| " | |
| "2014-12-13 17:41:54,209","__main__","DEBUG","decompress: _imeta_to_dict(imeta_stdout) = | |
| {'PARENT_FILE': {'units': 'NONE', 'value': '/tempZone/tmp/iplant/20141213T170358.425638_5GB.fastq'}, 'COMMENTS': {'units': 'NONE', 'value': "'This file is registered under the extension .fastq but is stored internally to iRODS with compression as .fastq.gz. This file will be decompressed upon retrieval (e.g. with iget, isync).'"}, 'COMPRESSION_METHOD': {'units': 'NONE', 'value': 'GZIP'}, 'IS_COMPRESSED': {'units': 'BOOL', 'value': True}, 'HASH_METHOD': {'units': 'NONE', 'value': 'SHA1'}, 'UNCOMPRESSED_HASH': {'units': 'NONE', 'value': 'a13fa24d16155850e85fb94aad6417e830f963dc'}, 'UNCOMPRESSED_SIZE': {'units': 'BYTES', 'value': 5000000065}}" | |
| "2014-12-13 17:41:54,210","__main__","DEBUG","decompress: do_decompress = True" | |
| "2014-12-13 17:41:54,210","__main__","DEBUG","decompress: imv /tempZone/home/rods/iplant/5GB.fastq /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq.gz" | |
| "2014-12-13 17:41:54,343","__main__","DEBUG","decompress: iget -f -T /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq.gz /tmp/iplant/20141213T174154.210195_5GB.fastq.gz" | |
| "2014-12-13 17:42:35,790","__main__","DEBUG","decompress: gunzip --force --keep /tmp/iplant/20141213T174154.210195_5GB.fastq.gz" | |
| "2014-12-13 17:43:29,553","__main__","DEBUG","decompress: uncompressed_size = os.path.getsize(/tmp/iplant/20141213T174154.210195_5GB.fastq)" | |
| "2014-12-13 17:43:29,553","__main__","DEBUG","decompress: uncompressed_size = 5000000065" | |
| "2014-12-13 17:43:29,553","__main__","DEBUG","decompress: Uncompressed file size matches 'UNCOMPRESSED_SIZE' from imeta." | |
| "2014-12-13 17:43:29,554","__main__","DEBUG","decompress: uncompressed_hash = _compute_hash(fpath=/tmp/iplant/20141213T174154.210195_5GB.fastq, algorithm=SHA1)" | |
| "2014-12-13 17:43:39,382","__main__","DEBUG","decompress: uncompressed_hash = a13fa24d16155850e85fb94aad6417e830f963dc" | |
| "2014-12-13 17:43:39,382","__main__","DEBUG","decompress: Uncompressed hash matches 'UNCOMPRESSED_HASH' from imeta." | |
| "2014-12-13 17:43:39,382","__main__","DEBUG","decompress: iput -T /tmp/iplant/20141213T174154.210195_5GB.fastq /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq" | |
| "2014-12-13 17:44:38,185","__main__","DEBUG","decompress: icp /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq_copy" | |
| "2014-12-13 17:47:38,865","__main__","DEBUG","decompress: imv /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq_copy /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-13 17:47:39,201","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq IS_COMPRESSED FALSE BOOL" | |
| "2014-12-13 17:47:39,283","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMPRESSION_METHOD NONE NONE" | |
| "2014-12-13 17:47:39,350","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_SIZE 5000000065 BYTES" | |
| "2014-12-13 17:47:39,417","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_HASH a13fa24d16155850e85fb94aad6417e830f963dc NONE" | |
| "2014-12-13 17:47:39,483","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq HASH_METHOD SHA1 NONE" | |
| "2014-12-13 17:47:39,550","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq PARENT_FILE /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq.gz NONE" | |
| "2014-12-13 17:47:39,617","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMMENTS 'This file is registered under the extension .fastq and is stored internally to iRODS without compression as .fastq.' NONE" | |
| "2014-12-13 17:47:39,684","__main__","INFO","main: END_LOGGING" | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 17:47:39 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ iget $IPLANT/5GB.fastq $TMP_IPLANT/. | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 17:50:29 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ cksum /tmp/5GB.fastq | |
| 2123744973 5000000065 /tmp/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 17:51:14 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ cksum $TMP_IPLANT/5GB.fastq | |
| 2123744973 5000000065 /tmp/iplant/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 17:52:04 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ irm -f $IPLANT/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 17:52:04 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ rm -f $TMP_IPLANT/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Sat Dec 13 17:52:05 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ # Check file sizes of tmp files. | |
| ubuntu@20140918t192000-irodsv331:~$ # Read `$IPLANT_LOG` between the timestamps from `date` to check execution. | |
| ubuntu@20140918t192000-irodsv331:~$ ls $TMP_IPLANT | |
| 20141213T174154.210195_5GB.fastq 20141213T174154.210195_5GB.fastq.gz iplant.log | |
| ubuntu@20140918t192000-irodsv331:~$ cksum /tmp/iplant/20141213T174154.210195_5GB.fastq | |
| 2123744973 5000000065 /tmp/iplant/20141213T174154.210195_5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ # local files sizes are correct | |
| ubuntu@20140918t192000-irodsv331:~$ | |
| ubuntu@20140918t192000-irodsv331:~$ $IRODS/server/bin/cmd/iplant.py --ipath $IPLANT/5GB.fastq --iplant $IPLANT --action decompress --itmp_iplant $ITMP_IPLANT --tmp_iplant $TMP_IPLANT --logging_level DEBUG --log_file $IPLANT_LOG --help | |
| usage: iplant.py [-h] --ipath IPATH --iplant IPLANT --action | |
| {compress,decompress} --itmp_iplant ITMP_IPLANT --tmp_iplant | |
| TMP_IPLANT [--delete_itmp_files] [--delete_tmp_files] | |
| [--logging_level {DEBUG,INFO,WARNING,ERROR,CRITICAL}] | |
| [--log_file LOG_FILE] [-t] | |
| Compress or decompress .fastq file in iPlant collection. | |
| optional arguments: | |
| -h, --help show this help message and exit | |
| --ipath IPATH iRODS path to .fastq file for (de)compression. | |
| --iplant IPLANT iRODS path to iplant root directory. Only files within | |
| this directory will be (de)compressed. | |
| --action {compress,decompress} | |
| Action to take on the file from `ipath`. | |
| --itmp_iplant ITMP_IPLANT | |
| iRODS path to temporary directory for moving files | |
| during (de)compression. | |
| --tmp_iplant TMP_IPLANT | |
| Local path to temporary directory for moving files | |
| during (de)compression. | |
| --delete_itmp_files Delete iRODS temporary files made during | |
| (de)compression. | |
| --delete_tmp_files Delete local temporary files made during | |
| (de)compression. | |
| --logging_level {DEBUG,INFO,WARNING,ERROR,CRITICAL} | |
| Verbosity of logging level. 'DEBUG' is the most | |
| verbose; 'CRITICAL' is the least. Default: INFO | |
| --log_file LOG_FILE Local path for writing log in addition to stdout. Use | |
| to debug while executing module with iRODS icommands. | |
| -t, --test Test that module is being called correctly. Checks | |
| input then prints message to stdout. No actions are | |
| taken. | |
| ubuntu@20140918t192000-irodsv331:~$ cp -i /home/ubuntu/iRODS/server/config/reConfigs/core.re_BACKUP_2014121 /home/ubuntu/iRODS/server/config/reConfigs/core.recore.re_BACKUP_20141210T214000 core.re_BACKUP_20141211T213000 core.re_BACKUP_20141212T203000 | |
| core.re_BACKUP_20141211T120000 core.re_BACKUP_20141212T180000 core.re_BACKUP_20141213T102000 | |
| ubuntu@20140918t192000-irodsv331:~$ cp -i /home/ubuntu/iRODS/server/config/reConfigs/core.re_BACKUP_20141213T102000 /home/ubuntu/iRODS/server/config/reConfigs/core.re | |
| cp: overwrite ‘/home/ubuntu/iRODS/server/config/reConfigs/core.re’? y | |
| ubuntu@20140918t192000-irodsv331:~$ diff /home/ubuntu/iRODS/server/config/reConfigs/core.re /home/ubuntu/iRODS/server/config/reConfigs/core.re_BACKUP_ORIGINAL_20141011T135000 198,201d197 | |
| < # ORIGINAL: | |
| < # acPreprocForDataObjOpen { } | |
| < # IPLANT: | |
| < acPreprocForDataObjOpen {ON($objPath like "/tempZone/home/rods/iplant/*") {iplantPreprocForDataObjOpen;}} | |
| 256,259d251 | |
| < # ORIGINAL: | |
| < # acPostProcForPut { } | |
| < # IPLANT: | |
| < acPostProcForPut {ON($objPath like "/tempZone/home/rods/iplant/*") {iplantPostProcForPut;}} | |
| 265,268d256 | |
| < # ORIGINAL: | |
| < # acPostProcForOpen { } | |
| < # IPLANT: | |
| < acPostProcForOpen {ON($objPath like "/tempZone/home/rods/iplant/*") {iplantPostProcForOpen;}} | |
| ubuntu@20140918t192000-irodsv331:~$ | |
| ubuntu@20140918t192000-irodsv331:~$ # TODO: resume here, run test with automatic pipeline | |
| ubuntu@20140918t192000-irodsv331:~$ # TODO: modify iplant.re and remove --delete_tmp_files option | |
| ubuntu@20140918t192000-irodsv331:~$ ils $ITMP_IPLANT | |
| /tempZone/tmp/iplant: | |
| 20141213T174154.210195_5GB.fastq | |
| 20141213T174154.210195_5GB.fastq.gz | |
| ubuntu@20140918t192000-irodsv331:~$ irm -f $ITMP_IPLANT/20141213T174154.210195_5GB.fastqubuntu@20140918t192000-irodsv331:~$ irm -f $ITMP_IPLANT/20141213T174154.210195_5GB.fastq.gz | |
| ubuntu@20140918t192000-irodsv331:~$ ls /tmp/iplant/ | |
| 20141213T174154.210195_5GB.fastq 20141213T174154.210195_5GB.fastq.gz iplant.log | |
| ubuntu@20140918t192000-irodsv331:~$ rm -f /tmp/iplant/20141213T174154.210195_5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ rm -f /tmp/iplant/20141213T174154.210195_5GB.fastq.gz | |
| ubuntu@20140918t192000-irodsv331:~$ | |
| ubuntu@20140918t192000-irodsv331:~$ # REINSTALL | |
| ubuntu@20140918t192000-irodsv331:~$ # because omitted --delete_tmp_files options | |
| ubuntu@20140918t192000-irodsv331:~$ cd ~ | |
| ubuntu@20140918t192000-irodsv331:~$ git clone https://github.com/stharrold/irods_module_iplant.git | |
| fatal: destination path 'irods_module_iplant' already exists and is not an empty directory. | |
| ubuntu@20140918t192000-irodsv331:~$ IRODS=~/iRODS # example root iRODS directory | |
| ubuntu@20140918t192000-irodsv331:~$ chmod +x ~/irods_module_iplant/iplant/rules/* | |
| ubuntu@20140918t192000-irodsv331:~$ cp -r ~/irods_module_iplant/iplant $IRODS/modules/. | |
| ubuntu@20140918t192000-irodsv331:~$ cp $IRODS/modules/iplant/rules/*.re $IRODS/server/config/reConfigs/. | |
| ubuntu@20140918t192000-irodsv331:~$ cp $IRODS/modules/iplant/rules/*.py $IRODS/server/bin/cmd/. | |
| ubuntu@20140918t192000-irodsv331:~$ | |
| ubuntu@20140918t192000-irodsv331:~$ diff /home/ubuntu/iRODS/server/config/server.config /home/ubuntu/iRODS/server/config/server.config_BACKUP_ORIGINAL_20141011T125000 | |
| 31,34c31 | |
| < # ORIGINAL: 2014-10-18 | |
| < # reRuleSet core | |
| < # IPLANT: 2014-12-06 | |
| < reRuleSet core,iplant | |
| --- | |
| > reRuleSet core | |
| ubuntu@20140918t192000-irodsv331:~$ diff /home/ubuntu/iRODS/server/config/reConfigs/core.re /home/ubuntu/iRODS/server/config/reConfigs/core.re_BACKUP_ORIGINAL_20141011T135000 | |
| 198,201d197 | |
| < # ORIGINAL: | |
| < # acPreprocForDataObjOpen { } | |
| < # IPLANT: | |
| < acPreprocForDataObjOpen {ON($objPath like "/tempZone/home/rods/iplant/*") {iplantPreprocForDataObjOpen;}} | |
| 256,259d251 | |
| < # ORIGINAL: | |
| < # acPostProcForPut { } | |
| < # IPLANT: | |
| < acPostProcForPut {ON($objPath like "/tempZone/home/rods/iplant/*") {iplantPostProcForPut;}} | |
| 265,268d256 | |
| < # ORIGINAL: | |
| < # acPostProcForOpen { } | |
| < # IPLANT: | |
| < acPostProcForOpen {ON($objPath like "/tempZone/home/rods/iplant/*") {iplantPostProcForOpen;}} | |
| ubuntu@20140918t192000-irodsv331:~$ # Test iplant.py with automatic calls from iRODS | |
| ubuntu@20140918t192000-irodsv331:~$ # Use the iPlant version of `core.re` from following `INSTALL.md`. | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Mon Dec 15 02:41:20 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ iput /tmp/5GB.fastq $IPLANT/. | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Mon Dec 15 02:49:04 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ iget $IPLANT/5GB.fastq $TMP_IPLANT/. | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Mon Dec 15 03:01:33 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ cksum /tmp/5GB.fastq | |
| 2123744973 5000000065 /tmp/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Mon Dec 15 03:02:22 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ cksum $TMP_IPLANT/5GB.fastq | |
| 4124808789 1375278113 /tmp/iplant/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Mon Dec 15 03:02:36 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ irm -f $IPLANT/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Mon Dec 15 03:02:36 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ rm -f $TMP_IPLANT/5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ date | |
| Mon Dec 15 03:02:37 UTC 2014 | |
| ubuntu@20140918t192000-irodsv331:~$ # Read `$IRODS/server/log/rodsLog.YYYY.MM.DD` between the timestamps from `date` to check execution. | |
| ubuntu@20140918t192000-irodsv331:~$ # Read `$IPLANT_LOG` between the timestamps from `date` to check execution. | |
| ubuntu@20140918t192000-irodsv331:~$ # Check file sizes of tmp files. | |
| ubuntu@20140918t192000-irodsv331:~$ # File was not fully fetched from cksums | |
| ubuntu@20140918t192000-irodsv331:~$ # FAIL | |
| ubuntu@20140918t192000-irodsv331:~$ ls $TMP_IPLANT | |
| 20141215T024355.675266_5GB.fastq 20141215T024905.519428_5GB.fastq 20141215T025630.340886_5GB.fastq iplant.log | |
| 20141215T024355.675266_5GB.fastq.gz 20141215T024905.519428_5GB.fastq.gz 20141215T025630.340886_5GB.fastq.gz | |
| ubuntu@20140918t192000-irodsv331:~$ cksum $TMP_IPLANT/*.fastq | |
| 2123744973 5000000065 /tmp/iplant/20141215T024355.675266_5GB.fastq | |
| 2123744973 5000000065 /tmp/iplant/20141215T024905.519428_5GB.fastq | |
| 2123744973 5000000065 /tmp/iplant/20141215T025630.340886_5GB.fastq | |
| ubuntu@20140918t192000-irodsv331:~$ # tempoarary files have correct size. The iplant.py module worked, but iget did not. | |
| ubuntu@20140918t192000-irodsv331:~$ less ~/iRODS/server/log/rodsLog.2014.12.11 | |
| ubuntu@20140918t192000-irodsv331:~$ less ~/iRODS/server/log/rodsLog.2014.12.11 | |
| ubuntu@20140918t192000-irodsv331:~$ tail -400 ~/iRODS/server/log/rodsLog.2014.12.11 | |
| Dec 15 02:32:36 pid:10407 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:32:36 pid:10407 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:32:36 pid:10407 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:32:39 pid:1649 NOTICE: Agent process 10407 exited with status 0 | |
| Dec 15 02:32:39 pid:1649 NOTICE: Agent process 10410 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:32:39 pid:10410 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:32:39 pid:10410 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:32:39 pid:10410 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:32:39 pid:10410 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:41:20 pid:1649 NOTICE: Agent process 10410 exited with status 0 | |
| Dec 15 02:41:20 pid:1649 NOTICE: Agent process 10567 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:41:20 pid:10567 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:41:20 pid:10567 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:43:54 pid:10567 NOTICE: writeLine: inString = iplant.re:iplantPostProcForPut: Calling iplant.py to compress /tempZone/home/rods/iplant/5GB.fastq | |
| Dec 15 02:43:54 pid:10653 NOTICE: execCmd:cmd/iplant.py argv:--ipath /tempZone/home/rods/iplant/5GB.fastq --iplant /tempZone/home/rods/iplant --action compress --itmp_iplant /tempZone/tmp/iplant --tmp_iplant /tmp/iplant --delete_itmp_files --logging_level DEBUG --log_file /tmp/iplant/iplant.log | |
| Dec 15 02:43:54 pid:10653 NOTICE: getServerHostByFileInx: fileInx 3 not active | |
| Dec 15 02:43:55 pid:1649 NOTICE: Agent process 10655 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:43:55 pid:10655 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:43:55 pid:10655 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:43:55 pid:10655 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:43:55 pid:10655 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:43:55 pid:1649 NOTICE: Agent process 10655 exited with status 0 | |
| Dec 15 02:43:55 pid:1649 NOTICE: Agent process 10658 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:43:55 pid:10658 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:43:55 pid:10658 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:43:55 pid:10658 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:43:55 pid:10658 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:43:55 pid:1649 NOTICE: Agent process 10658 exited with status 0 | |
| Dec 15 02:43:55 pid:1649 NOTICE: Agent process 10661 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:43:55 pid:10661 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:43:55 pid:10661 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:43:55 pid:10661 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:43:55 pid:10661 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:43:55 pid:1649 NOTICE: Agent process 10661 exited with status 0 | |
| Dec 15 02:43:55 pid:1649 NOTICE: Agent process 10664 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:43:55 pid:10664 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:43:55 pid:10664 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:43:55 pid:10664 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:43:55 pid:10664 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:43:55 pid:1649 NOTICE: Agent process 10664 exited with status 0 | |
| Dec 15 02:43:55 pid:1649 NOTICE: Agent process 10667 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:43:55 pid:10667 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:43:55 pid:10667 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:46:45 pid:10667 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:46:45 pid:10667 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:48:41 pid:1649 NOTICE: Agent process 10667 exited with status 0 | |
| Dec 15 02:48:41 pid:1649 NOTICE: Agent process 10776 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:48:41 pid:10776 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:48:41 pid:10776 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:48:46 pid:10776 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:48:46 pid:10776 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:48:48 pid:1649 NOTICE: Agent process 10776 exited with status 0 | |
| Dec 15 02:48:48 pid:1649 NOTICE: Agent process 10813 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:48:48 pid:10813 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:48:48 pid:10813 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:48:57 pid:10813 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:48:57 pid:10813 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:48:57 pid:1649 NOTICE: Agent process 10813 exited with status 0 | |
| Dec 15 02:48:57 pid:1649 NOTICE: Agent process 10843 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:48:58 pid:10843 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:48:58 pid:10843 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:00 pid:10843 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:00 pid:10843 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:01 pid:1649 NOTICE: Agent process 10843 exited with status 0 | |
| Dec 15 02:49:01 pid:1649 NOTICE: Agent process 10846 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:01 pid:10846 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:01 pid:10846 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:01 pid:10846 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:01 pid:10846 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:01 pid:1649 NOTICE: Agent process 10846 exited with status 0 | |
| Dec 15 02:49:01 pid:1649 NOTICE: Agent process 10849 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:02 pid:10849 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:02 pid:10849 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:02 pid:10849 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:02 pid:10849 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:02 pid:1649 NOTICE: Agent process 10849 exited with status 0 | |
| Dec 15 02:49:02 pid:1649 NOTICE: Agent process 10852 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:02 pid:10852 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:02 pid:10852 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:03 pid:10852 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:03 pid:10852 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:03 pid:1649 NOTICE: Agent process 10852 exited with status 0 | |
| Dec 15 02:49:03 pid:1649 NOTICE: Agent process 10855 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:03 pid:10855 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:03 pid:10855 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:03 pid:10855 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:03 pid:10855 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:03 pid:1649 NOTICE: Agent process 10855 exited with status 0 | |
| Dec 15 02:49:03 pid:1649 NOTICE: Agent process 10858 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:03 pid:10858 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:03 pid:10858 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:04 pid:10858 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:04 pid:10858 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:04 pid:1649 NOTICE: Agent process 10858 exited with status 0 | |
| Dec 15 02:49:04 pid:1649 NOTICE: Agent process 10861 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:04 pid:10861 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:04 pid:10861 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:04 pid:10861 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:04 pid:10861 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:04 pid:1649 NOTICE: Agent process 10861 exited with status 0 | |
| Dec 15 02:49:04 pid:1649 NOTICE: Agent process 10864 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:04 pid:10864 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:04 pid:10864 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:04 pid:10864 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:04 pid:10864 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:04 pid:1649 NOTICE: Agent process 10864 exited with status 0 | |
| Dec 15 02:49:04 pid:1649 NOTICE: Agent process 10867 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:04 pid:10867 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:04 pid:10867 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:04 pid:10867 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:04 pid:10867 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:04 pid:1649 NOTICE: Agent process 10867 exited with status 0 | |
| Dec 15 02:49:04 pid:1649 NOTICE: Agent process 10870 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:04 pid:10870 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:04 pid:10870 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:04 pid:10870 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:04 pid:10870 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:04 pid:10567 NOTICE: writeLine: inString = iplant.py:stdout:INFO: Arguments: | |
| Namespace(action='compress', delete_itmp_files=True, delete_tmp_files=False, ipath='/tempZone/home/rods/iplant/5GB.fastq', iplant='/tempZone/home/rods/iplant', itmp_iplant='/tempZone/tmp/iplant', log_file='/tmp/iplant/iplant.log', logging_level='DEBUG', test=False, tmp_iplant='/tmp/iplant') | |
| INFO: --ipath is contained within --iplant. | |
| --ipath /tempZone/home/rods/iplant/5GB.fastq | |
| --iplant /tempZone/home/rods/iplant | |
| "2014-12-15 02:43:55,642","__main__","INFO","main: BEGIN_LOGGING" | |
| "2014-12-15 02:43:55,642","__main__","INFO","main: Log format: '%(asctime)s','%(name)s','%(levelname)s','%(message)s'" | |
| "2014-12-15 02:43:55,642","__main__","INFO","main: Log date format: default ISO 8601, UTC" | |
| "2014-12-15 02:43:55,642","__main__","INFO","main: Compressing file." | |
| "2014-12-15 02:43:55,642","__main__","DEBUG","main: compress(ipath=/tempZone/home/rods/iplant/5GB.fastq, itmp_iplant=/tempZone/tmp/iplant, tmp_iplant=/tmp/iplant, delete_itmp_files=True, delete_tmp_files=False)" | |
| "2014-12-15 02:43:55,642","__main__","DEBUG","compress: imeta ls -d /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-15 02:43:55,674","__main__","DEBUG","compress: imeta_stdout = | |
| AVUs defined for dataObj /tempZone/home/rods/iplant/5GB.fastq: | |
| None | |
| " | |
| "2014-12-15 02:43:55,675","__main__","DEBUG","compress: _imeta_to_dict(imeta_stdout) = | |
| {}" | |
| "2014-12-15 02:43 | |
| Dec 15 02:49:04 pid:10567 NOTICE: writeLine: inString = iplant.py:stderr: | |
| Dec 15 02:49:04 pid:10567 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:04 pid:10567 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:04 pid:1649 NOTICE: Agent process 10567 exited with status 0 | |
| Dec 15 02:49:04 pid:1649 NOTICE: Agent process 10870 exited with status 0 | |
| Dec 15 02:49:04 pid:1649 NOTICE: Agent process 10874 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:04 pid:10874 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:04 pid:10874 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:04 pid:10874 NOTICE: writeLine: inString = iplant.re:iplantPreprocForDataObjOpen: Calling iplant.py to decompress /tempZone/home/rods/iplant/5GB.fastq | |
| Dec 15 02:49:04 pid:10876 NOTICE: execCmd:cmd/iplant.py argv:--ipath /tempZone/home/rods/iplant/5GB.fastq --iplant /tempZone/home/rods/iplant --action decompress --itmp_iplant /tempZone/tmp/iplant --tmp_iplant /tmp/iplant --delete_itmp_files --logging_level DEBUG --log_file /tmp/iplant/iplant.log | |
| Dec 15 02:49:05 pid:1649 NOTICE: Agent process 10878 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:05 pid:10878 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:05 pid:10878 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:05 pid:10878 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:05 pid:10878 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:05 pid:1649 NOTICE: Agent process 10878 exited with status 0 | |
| Dec 15 02:49:05 pid:1649 NOTICE: Agent process 10881 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:05 pid:10881 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:05 pid:10881 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:05 pid:10881 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:05 pid:10881 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:05 pid:1649 NOTICE: Agent process 10881 exited with status 0 | |
| Dec 15 02:49:05 pid:1649 NOTICE: Agent process 10884 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:05 pid:10884 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:05 pid:10884 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:05 pid:10884 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:05 pid:10884 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:05 pid:1649 NOTICE: Agent process 10884 exited with status 0 | |
| Dec 15 02:49:05 pid:1649 NOTICE: Agent process 10887 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:05 pid:10887 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:05 pid:10887 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:05 pid:10887 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:05 pid:10887 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:49:05 pid:1649 NOTICE: Agent process 10887 exited with status 0 | |
| Dec 15 02:49:05 pid:1649 NOTICE: Agent process 10890 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:49:05 pid:10890 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:49:05 pid:10890 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:49:13 pid:10890 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:49:13 pid:10890 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:51:17 pid:1649 NOTICE: Agent process 10890 exited with status 0 | |
| Dec 15 02:51:17 pid:1649 NOTICE: Agent process 10963 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:51:17 pid:10963 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:51:17 pid:10963 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:53:03 pid:10963 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:53:03 pid:10963 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:53:03 pid:1649 NOTICE: Agent process 10963 exited with status 0 | |
| Dec 15 02:53:03 pid:1649 NOTICE: Agent process 11024 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:53:03 pid:11024 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:53:03 pid:11024 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:55:55 pid:11024 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:55:55 pid:11024 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:55:56 pid:1649 NOTICE: Agent process 11024 exited with status 0 | |
| Dec 15 02:55:56 pid:1649 NOTICE: Agent process 11090 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:55:58 pid:11090 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:55:58 pid:11090 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:00 pid:11090 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:00 pid:11090 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:00 pid:1649 NOTICE: Agent process 11090 exited with status 0 | |
| Dec 15 02:56:00 pid:1649 NOTICE: Agent process 11093 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:00 pid:11093 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:00 pid:11093 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:00 pid:11093 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:00 pid:11093 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:00 pid:1649 NOTICE: Agent process 11093 exited with status 0 | |
| Dec 15 02:56:00 pid:1649 NOTICE: Agent process 11096 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:00 pid:11096 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:00 pid:11096 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:00 pid:11096 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:00 pid:11096 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:00 pid:1649 NOTICE: Agent process 11096 exited with status 0 | |
| Dec 15 02:56:00 pid:1649 NOTICE: Agent process 11099 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:00 pid:11099 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:00 pid:11099 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:00 pid:11099 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:00 pid:11099 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:00 pid:1649 NOTICE: Agent process 11099 exited with status 0 | |
| Dec 15 02:56:00 pid:1649 NOTICE: Agent process 11102 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:00 pid:11102 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:00 pid:11102 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:00 pid:11102 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:00 pid:11102 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:01 pid:1649 NOTICE: Agent process 11102 exited with status 0 | |
| Dec 15 02:56:01 pid:1649 NOTICE: Agent process 11105 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:01 pid:11105 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:01 pid:11105 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:01 pid:11105 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:01 pid:11105 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:01 pid:1649 NOTICE: Agent process 11105 exited with status 0 | |
| Dec 15 02:56:01 pid:1649 NOTICE: Agent process 11108 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:01 pid:11108 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:01 pid:11108 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:01 pid:11108 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:01 pid:11108 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:01 pid:1649 NOTICE: Agent process 11108 exited with status 0 | |
| Dec 15 02:56:01 pid:1649 NOTICE: Agent process 11111 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:01 pid:11111 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:01 pid:11111 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:01 pid:11111 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:01 pid:11111 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:01 pid:1649 NOTICE: Agent process 11111 exited with status 0 | |
| Dec 15 02:56:01 pid:1649 NOTICE: Agent process 11114 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:01 pid:11114 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:01 pid:11114 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:01 pid:11114 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:01 pid:11114 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:01 pid:1649 NOTICE: Agent process 11114 exited with status 0 | |
| Dec 15 02:56:01 pid:1649 NOTICE: Agent process 11117 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:01 pid:11117 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:01 pid:11117 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:01 pid:11117 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:01 pid:11117 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:01 pid:10874 NOTICE: writeLine: inString = iplant.py:stdout:INFO: Arguments: | |
| Namespace(action='decompress', delete_itmp_files=True, delete_tmp_files=False, ipath='/tempZone/home/rods/iplant/5GB.fastq', iplant='/tempZone/home/rods/iplant', itmp_iplant='/tempZone/tmp/iplant', log_file='/tmp/iplant/iplant.log', logging_level='DEBUG', test=False, tmp_iplant='/tmp/iplant') | |
| INFO: --ipath is contained within --iplant. | |
| --ipath /tempZone/home/rods/iplant/5GB.fastq | |
| --iplant /tempZone/home/rods/iplant | |
| "2014-12-15 02:49:05,473","__main__","INFO","main: BEGIN_LOGGING" | |
| "2014-12-15 02:49:05,473","__main__","INFO","main: Log format: '%(asctime)s','%(name)s','%(levelname)s','%(message)s'" | |
| "2014-12-15 02:49:05,473","__main__","INFO","main: Log date format: default ISO 8601, UTC" | |
| "2014-12-15 02:49:05,473","__main__","INFO","main: Decompressing file." | |
| "2014-12-15 02:49:05,474","__main__","DEBUG","main: decompress(ipath=/tempZone/home/rods/iplant/5GB.fastq, itmp_iplant=/tempZone/tmp/iplant, tmp_iplant=/tmp/iplant, delete_itmp_files=True, delete_tmp_files=False)" | |
| "2014-12-15 02:49:05,474","__main__","DEBUG","decompress: imeta ls -d /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-15 02:49:05,497","__main__","DEBUG","decompress: imeta_stdout = | |
| AVUs defined for dataObj /tempZone/home/rods/iplant/5GB.fastq: | |
| attribute: IS_COMPRESSED | |
| value: TRUE | |
| units: BOOL | |
| ---- | |
| attribute: COMMENTS | |
| value: 'This file is registered | |
| Dec 15 02:56:01 pid:10874 NOTICE: writeLine: inString = iplant.py:stderr: | |
| Dec 15 02:56:29 pid:10874 NOTICE: writeLine: inString = iplant.re:iplantPostProcForOpen: Calling iplant.py to compress /tempZone/home/rods/iplant/5GB.fastq | |
| Dec 15 02:56:29 pid:11152 NOTICE: execCmd:cmd/iplant.py argv:--ipath /tempZone/home/rods/iplant/5GB.fastq --iplant /tempZone/home/rods/iplant --action compress --itmp_iplant /tempZone/tmp/iplant --tmp_iplant /tmp/iplant --delete_itmp_files --logging_level DEBUG --log_file /tmp/iplant/iplant.log | |
| Dec 15 02:56:29 pid:11152 NOTICE: getServerHostByFileInx: fileInx 3 not active | |
| Dec 15 02:56:30 pid:1649 NOTICE: Agent process 11117 exited with status 0 | |
| Dec 15 02:56:30 pid:1649 NOTICE: Agent process 11154 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:30 pid:11154 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:30 pid:11154 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:30 pid:11154 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:30 pid:11154 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:30 pid:1649 NOTICE: Agent process 11154 exited with status 0 | |
| Dec 15 02:56:30 pid:1649 NOTICE: Agent process 11157 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:30 pid:11157 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:30 pid:11157 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:30 pid:11157 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:30 pid:11157 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:30 pid:1649 NOTICE: Agent process 11157 exited with status 0 | |
| Dec 15 02:56:30 pid:1649 NOTICE: Agent process 11160 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:30 pid:11160 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:30 pid:11160 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:30 pid:11160 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:30 pid:11160 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:30 pid:1649 NOTICE: Agent process 11160 exited with status 0 | |
| Dec 15 02:56:30 pid:1649 NOTICE: Agent process 11163 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:30 pid:11163 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:30 pid:11163 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:56:30 pid:11163 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:56:30 pid:11163 NOTICE: Agent exiting with status = 0 | |
| Dec 15 02:56:30 pid:1649 NOTICE: Agent process 11163 exited with status 0 | |
| Dec 15 02:56:30 pid:1649 NOTICE: Agent process 11166 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 02:56:30 pid:11166 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 02:56:30 pid:11166 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 02:58:21 pid:11166 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 02:58:21 pid:11166 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:00:57 pid:1649 NOTICE: Agent process 11166 exited with status 0 | |
| Dec 15 03:00:57 pid:1649 NOTICE: Agent process 11274 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 03:00:57 pid:11274 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 03:00:57 pid:11274 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 03:01:24 pid:11274 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:01:24 pid:11274 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:01:25 pid:1649 NOTICE: Agent process 11274 exited with status 0 | |
| Dec 15 03:01:25 pid:1649 NOTICE: Agent process 11312 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 03:01:25 pid:11312 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 03:01:25 pid:11312 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 03:01:28 pid:11312 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:01:28 pid:11312 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:01:29 pid:1649 NOTICE: Agent process 11312 exited with status 0 | |
| Dec 15 03:01:29 pid:1649 NOTICE: Agent process 11331 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 03:01:31 pid:11331 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 03:01:31 pid:11331 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 03:01:32 pid:11331 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:01:32 pid:11331 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:01:32 pid:1649 NOTICE: Agent process 11331 exited with status 0 | |
| Dec 15 03:01:32 pid:1649 NOTICE: Agent process 11334 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 03:01:32 pid:11334 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 03:01:32 pid:11334 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 03:01:33 pid:11334 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:01:33 pid:11334 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11334 exited with status 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11337 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 03:01:33 pid:11337 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 03:01:33 pid:11337 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 03:01:33 pid:11337 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:01:33 pid:11337 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11337 exited with status 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11340 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 03:01:33 pid:11340 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 03:01:33 pid:11340 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 03:01:33 pid:11340 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:01:33 pid:11340 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11340 exited with status 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11343 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 03:01:33 pid:11343 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 03:01:33 pid:11343 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 03:01:33 pid:11343 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:01:33 pid:11343 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11343 exited with status 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11346 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 03:01:33 pid:11346 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 03:01:33 pid:11346 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 03:01:33 pid:11346 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:01:33 pid:11346 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11346 exited with status 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11349 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 03:01:33 pid:11349 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 03:01:33 pid:11349 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 03:01:33 pid:11349 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:01:33 pid:11349 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11349 exited with status 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11352 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 03:01:33 pid:11352 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 03:01:33 pid:11352 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 03:01:33 pid:11352 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:01:33 pid:11352 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11352 exited with status 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11355 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 03:01:33 pid:11355 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 03:01:33 pid:11355 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 03:01:33 pid:11355 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:01:33 pid:11355 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11355 exited with status 0 | |
| Dec 15 03:01:33 pid:1649 NOTICE: Agent process 11358 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 03:01:33 pid:11358 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 03:01:33 pid:11358 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 03:01:33 pid:11358 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:01:33 pid:11358 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:01:33 pid:10874 NOTICE: writeLine: inString = iplant.py:stdout:INFO: Arguments: | |
| Namespace(action='compress', delete_itmp_files=True, delete_tmp_files=False, ipath='/tempZone/home/rods/iplant/5GB.fastq', iplant='/tempZone/home/rods/iplant', itmp_iplant='/tempZone/tmp/iplant', log_file='/tmp/iplant/iplant.log', logging_level='DEBUG', test=False, tmp_iplant='/tmp/iplant') | |
| INFO: --ipath is contained within --iplant. | |
| --ipath /tempZone/home/rods/iplant/5GB.fastq | |
| --iplant /tempZone/home/rods/iplant | |
| "2014-12-15 02:56:30,317","__main__","INFO","main: BEGIN_LOGGING" | |
| "2014-12-15 02:56:30,317","__main__","INFO","main: Log format: '%(asctime)s','%(name)s','%(levelname)s','%(message)s'" | |
| "2014-12-15 02:56:30,317","__main__","INFO","main: Log date format: default ISO 8601, UTC" | |
| "2014-12-15 02:56:30,317","__main__","INFO","main: Compressing file." | |
| "2014-12-15 02:56:30,317","__main__","DEBUG","main: compress(ipath=/tempZone/home/rods/iplant/5GB.fastq, itmp_iplant=/tempZone/tmp/iplant, tmp_iplant=/tmp/iplant, delete_itmp_files=True, delete_tmp_files=False)" | |
| "2014-12-15 02:56:30,317","__main__","DEBUG","compress: imeta ls -d /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-15 02:56:30,340","__main__","DEBUG","compress: imeta_stdout = | |
| AVUs defined for dataObj /tempZone/home/rods/iplant/5GB.fastq: | |
| attribute: IS_COMPRESSED | |
| value: FALSE | |
| units: BOOL | |
| ---- | |
| attribute: COMMENTS | |
| value: 'This file is registered under th | |
| Dec 15 03:01:33 pid:10874 NOTICE: writeLine: inString = iplant.py:stderr: | |
| Dec 15 03:01:33 pid:10874 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:01:33 pid:10874 NOTICE: Agent exiting with status = 0 | |
| Dec 15 03:02:36 pid:1649 NOTICE: Agent process 10874 exited with status 0 | |
| Dec 15 03:02:36 pid:1649 NOTICE: Agent process 11358 exited with status 0 | |
| Dec 15 03:02:36 pid:1649 NOTICE: Agent process 11391 started for puser=rods and cuser=rods from 127.0.0.1 | |
| Dec 15 03:02:36 pid:11391 NOTICE: rsAuthCheck user rods#tempZone | |
| Dec 15 03:02:36 pid:11391 NOTICE: rsAuthResponse set proxy authFlag to 5, client authFlag to 5, user:rods#tempZone proxy:rods client:rods | |
| Dec 15 03:02:36 pid:11391 NOTICE: readAndProcClientMsg: received disconnect msg from client | |
| Dec 15 03:02:36 pid:11391 NOTICE: Agent exiting with status = 0 | |
| ubuntu@20140918t192000-irodsv331:~$ # | |
| ubuntu@20140918t192000-irodsv331:~$ # stdout was cutoff. | |
| ubuntu@20140918t192000-irodsv331:~$ less $IPLANT_LOG | |
| ubuntu@20140918t192000-irodsv331:~$ less $IPLANT_LOG | |
| ubuntu@20140918t192000-irodsv331:~$ tail -180 $IPLANT_LOG | |
| " | |
| "2014-12-13 17:41:54,209","__main__","DEBUG","decompress: _imeta_to_dict(imeta_stdout) = | |
| {'PARENT_FILE': {'units': 'NONE', 'value': '/tempZone/tmp/iplant/20141213T170358.425638_5GB.fastq'}, 'COMMENTS': {'units': 'NONE', 'value': "'This file is registered under the extension .fastq but is stored internally to iRODS with compression as .fastq.gz. This file will be decompressed upon retrieval (e.g. with iget, isync).'"}, 'COMPRESSION_METHOD': {'units': 'NONE', 'value': 'GZIP'}, 'IS_COMPRESSED': {'units': 'BOOL', 'value': True}, 'HASH_METHOD': {'units': 'NONE', 'value': 'SHA1'}, 'UNCOMPRESSED_HASH': {'units': 'NONE', 'value': 'a13fa24d16155850e85fb94aad6417e830f963dc'}, 'UNCOMPRESSED_SIZE': {'units': 'BYTES', 'value': 5000000065}}" | |
| "2014-12-13 17:41:54,210","__main__","DEBUG","decompress: do_decompress = True" | |
| "2014-12-13 17:41:54,210","__main__","DEBUG","decompress: imv /tempZone/home/rods/iplant/5GB.fastq /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq.gz" | |
| "2014-12-13 17:41:54,343","__main__","DEBUG","decompress: iget -f -T /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq.gz /tmp/iplant/20141213T174154.210195_5GB.fastq.gz" | |
| "2014-12-13 17:42:35,790","__main__","DEBUG","decompress: gunzip --force --keep /tmp/iplant/20141213T174154.210195_5GB.fastq.gz" | |
| "2014-12-13 17:43:29,553","__main__","DEBUG","decompress: uncompressed_size = os.path.getsize(/tmp/iplant/20141213T174154.210195_5GB.fastq)" | |
| "2014-12-13 17:43:29,553","__main__","DEBUG","decompress: uncompressed_size = 5000000065" | |
| "2014-12-13 17:43:29,553","__main__","DEBUG","decompress: Uncompressed file size matches 'UNCOMPRESSED_SIZE' from imeta." | |
| "2014-12-13 17:43:29,554","__main__","DEBUG","decompress: uncompressed_hash = _compute_hash(fpath=/tmp/iplant/20141213T174154.210195_5GB.fastq, algorithm=SHA1)" | |
| "2014-12-13 17:43:39,382","__main__","DEBUG","decompress: uncompressed_hash = a13fa24d16155850e85fb94aad6417e830f963dc" | |
| "2014-12-13 17:43:39,382","__main__","DEBUG","decompress: Uncompressed hash matches 'UNCOMPRESSED_HASH' from imeta." | |
| "2014-12-13 17:43:39,382","__main__","DEBUG","decompress: iput -T /tmp/iplant/20141213T174154.210195_5GB.fastq /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq" | |
| "2014-12-13 17:44:38,185","__main__","DEBUG","decompress: icp /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq_copy" | |
| "2014-12-13 17:47:38,865","__main__","DEBUG","decompress: imv /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq_copy /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-13 17:47:39,201","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq IS_COMPRESSED FALSE BOOL" | |
| "2014-12-13 17:47:39,283","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMPRESSION_METHOD NONE NONE" | |
| "2014-12-13 17:47:39,350","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_SIZE 5000000065 BYTES" | |
| "2014-12-13 17:47:39,417","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_HASH a13fa24d16155850e85fb94aad6417e830f963dc NONE" | |
| "2014-12-13 17:47:39,483","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq HASH_METHOD SHA1 NONE" | |
| "2014-12-13 17:47:39,550","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq PARENT_FILE /tempZone/tmp/iplant/20141213T174154.210195_5GB.fastq.gz NONE" | |
| "2014-12-13 17:47:39,617","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMMENTS 'This file is registered under the extension .fastq and is stored internally to iRODS without compression as .fastq.' NONE" | |
| "2014-12-13 17:47:39,684","__main__","INFO","main: END_LOGGING" | |
| "2014-12-15 02:43:55,642","__main__","INFO","main: BEGIN_LOGGING" | |
| "2014-12-15 02:43:55,642","__main__","INFO","main: Log format: '%(asctime)s','%(name)s','%(levelname)s','%(message)s'" | |
| "2014-12-15 02:43:55,642","__main__","INFO","main: Log date format: default ISO 8601, UTC" | |
| "2014-12-15 02:43:55,642","__main__","INFO","main: Compressing file." | |
| "2014-12-15 02:43:55,642","__main__","DEBUG","main: compress(ipath=/tempZone/home/rods/iplant/5GB.fastq, itmp_iplant=/tempZone/tmp/iplant, tmp_iplant=/tmp/iplant, delete_itmp_files=True, delete_tmp_files=False)" | |
| "2014-12-15 02:43:55,642","__main__","DEBUG","compress: imeta ls -d /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-15 02:43:55,674","__main__","DEBUG","compress: imeta_stdout = | |
| AVUs defined for dataObj /tempZone/home/rods/iplant/5GB.fastq: | |
| None | |
| " | |
| "2014-12-15 02:43:55,675","__main__","DEBUG","compress: _imeta_to_dict(imeta_stdout) = | |
| {}" | |
| "2014-12-15 02:43:55,675","__main__","DEBUG","compress: do_compress = True" | |
| "2014-12-15 02:43:55,675","__main__","DEBUG","compress: imv /tempZone/home/rods/iplant/5GB.fastq /tempZone/tmp/iplant/20141215T024355.675266_5GB.fastq" | |
| "2014-12-15 02:43:55,958","__main__","DEBUG","compress: iget -f -T /tempZone/tmp/iplant/20141215T024355.675266_5GB.fastq /tmp/iplant/20141215T024355.675266_5GB.fastq" | |
| "2014-12-15 02:46:45,183","__main__","DEBUG","compress: uncompressed_size = os.path.getsize(/tmp/iplant/20141215T024355.675266_5GB.fastq)" | |
| "2014-12-15 02:46:45,183","__main__","DEBUG","compress: uncompressed_size = 5000000065" | |
| "2014-12-15 02:46:45,183","__main__","DEBUG","compress: uncompressed_hash = _compute_hash(fpath=/tmp/iplant/20141215T024355.675266_5GB.fastq, algorithm=SHA1)" | |
| "2014-12-15 02:47:09,297","__main__","DEBUG","compress: uncompressed_hash = a13fa24d16155850e85fb94aad6417e830f963dc" | |
| "2014-12-15 02:47:09,298","__main__","DEBUG","compress: gzip --fast --force --keep /tmp/iplant/20141215T024355.675266_5GB.fastq" | |
| "2014-12-15 02:48:41,142","__main__","DEBUG","compress: iput -T /tmp/iplant/20141215T024355.675266_5GB.fastq.gz /tempZone/tmp/iplant/20141215T024355.675266_5GB.fastq.gz" | |
| "2014-12-15 02:48:47,148","__main__","DEBUG","compress: icp /tempZone/tmp/iplant/20141215T024355.675266_5GB.fastq.gz /tempZone/tmp/iplant/20141215T024355.675266_5GB.fastq.gz_copy" | |
| "2014-12-15 02:48:57,867","__main__","DEBUG","compress: imv /tempZone/tmp/iplant/20141215T024355.675266_5GB.fastq.gz_copy /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-15 02:49:00,793","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq IS_COMPRESSED TRUE BOOL" | |
| "2014-12-15 02:49:01,996","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMPRESSION_METHOD GZIP NONE" | |
| "2014-12-15 02:49:02,630","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_SIZE 5000000065 BYTES" | |
| "2014-12-15 02:49:03,246","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_HASH a13fa24d16155850e85fb94aad6417e830f963dc NONE" | |
| "2014-12-15 02:49:03,805","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq HASH_METHOD SHA1 NONE" | |
| "2014-12-15 02:49:04,147","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq PARENT_FILE /tempZone/tmp/iplant/20141215T024355.675266_5GB.fastq NONE" | |
| "2014-12-15 02:49:04,245","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMMENTS 'This file is registered under the extension .fastq but is stored internally to iRODS with compression as .fastq.gz. This file will be decompressed upon retrieval (e.g. with iget, isync).' NONE" | |
| "2014-12-15 02:49:04,288","__main__","DEBUG","compress: delete_itmp_files = True" | |
| "2014-12-15 02:49:04,288","__main__","DEBUG","compress: irm -f /tempZone/tmp/iplant/20141215T024355.675266_5GB.fastq" | |
| "2014-12-15 02:49:04,545","__main__","DEBUG","compress: irm -f /tempZone/tmp/iplant/20141215T024355.675266_5GB.fastq.gz" | |
| "2014-12-15 02:49:04,733","__main__","INFO","main: END_LOGGING" | |
| "2014-12-15 02:49:05,473","__main__","INFO","main: BEGIN_LOGGING" | |
| "2014-12-15 02:49:05,473","__main__","INFO","main: Log format: '%(asctime)s','%(name)s','%(levelname)s','%(message)s'" | |
| "2014-12-15 02:49:05,473","__main__","INFO","main: Log date format: default ISO 8601, UTC" | |
| "2014-12-15 02:49:05,473","__main__","INFO","main: Decompressing file." | |
| "2014-12-15 02:49:05,474","__main__","DEBUG","main: decompress(ipath=/tempZone/home/rods/iplant/5GB.fastq, itmp_iplant=/tempZone/tmp/iplant, tmp_iplant=/tmp/iplant, delete_itmp_files=True, delete_tmp_files=False)" | |
| "2014-12-15 02:49:05,474","__main__","DEBUG","decompress: imeta ls -d /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-15 02:49:05,497","__main__","DEBUG","decompress: imeta_stdout = | |
| AVUs defined for dataObj /tempZone/home/rods/iplant/5GB.fastq: | |
| attribute: IS_COMPRESSED | |
| value: TRUE | |
| units: BOOL | |
| ---- | |
| attribute: COMMENTS | |
| value: 'This file is registered under the extension .fastq but is stored internally to iRODS with compression as .fastq.gz. This file will be decompressed upon retrieval (e.g. with iget, isync).' | |
| units: NONE | |
| ---- | |
| attribute: UNCOMPRESSED_HASH | |
| value: a13fa24d16155850e85fb94aad6417e830f963dc | |
| units: NONE | |
| ---- | |
| attribute: HASH_METHOD | |
| value: SHA1 | |
| units: NONE | |
| ---- | |
| attribute: COMPRESSION_METHOD | |
| value: GZIP | |
| units: NONE | |
| ---- | |
| attribute: PARENT_FILE | |
| value: /tempZone/tmp/iplant/20141215T024355.675266_5GB.fastq | |
| units: NONE | |
| ---- | |
| attribute: UNCOMPRESSED_SIZE | |
| value: 5000000065 | |
| units: BYTES | |
| " | |
| "2014-12-15 02:49:05,508","__main__","DEBUG","decompress: _imeta_to_dict(imeta_stdout) = | |
| {'PARENT_FILE': {'units': 'NONE', 'value': '/tempZone/tmp/iplant/20141215T024355.675266_5GB.fastq'}, 'COMMENTS': {'units': 'NONE', 'value': "'This file is registered under the extension .fastq but is stored internally to iRODS with compression as .fastq.gz. This file will be decompressed upon retrieval (e.g. with iget, isync).'"}, 'COMPRESSION_METHOD': {'units': 'NONE', 'value': 'GZIP'}, 'IS_COMPRESSED': {'units': 'BOOL', 'value': True}, 'HASH_METHOD': {'units': 'NONE', 'value': 'SHA1'}, 'UNCOMPRESSED_HASH': {'units': 'NONE', 'value': 'a13fa24d16155850e85fb94aad6417e830f963dc'}, 'UNCOMPRESSED_SIZE': {'units': 'BYTES', 'value': 5000000065}}" | |
| "2014-12-15 02:49:05,519","__main__","DEBUG","decompress: do_decompress = True" | |
| "2014-12-15 02:49:05,519","__main__","DEBUG","decompress: imv /tempZone/home/rods/iplant/5GB.fastq /tempZone/tmp/iplant/20141215T024905.519428_5GB.fastq.gz" | |
| "2014-12-15 02:49:05,572","__main__","DEBUG","decompress: iget -f -T /tempZone/tmp/iplant/20141215T024905.519428_5GB.fastq.gz /tmp/iplant/20141215T024905.519428_5GB.fastq.gz" | |
| "2014-12-15 02:49:13,728","__main__","DEBUG","decompress: gunzip --force --keep /tmp/iplant/20141215T024905.519428_5GB.fastq.gz" | |
| "2014-12-15 02:50:34,089","__main__","DEBUG","decompress: uncompressed_size = os.path.getsize(/tmp/iplant/20141215T024905.519428_5GB.fastq)" | |
| "2014-12-15 02:50:34,089","__main__","DEBUG","decompress: uncompressed_size = 5000000065" | |
| "2014-12-15 02:50:34,089","__main__","DEBUG","decompress: Uncompressed file size matches 'UNCOMPRESSED_SIZE' from imeta." | |
| "2014-12-15 02:50:34,089","__main__","DEBUG","decompress: uncompressed_hash = _compute_hash(fpath=/tmp/iplant/20141215T024905.519428_5GB.fastq, algorithm=SHA1)" | |
| "2014-12-15 02:51:17,137","__main__","DEBUG","decompress: uncompressed_hash = a13fa24d16155850e85fb94aad6417e830f963dc" | |
| "2014-12-15 02:51:17,137","__main__","DEBUG","decompress: Uncompressed hash matches 'UNCOMPRESSED_HASH' from imeta." | |
| "2014-12-15 02:51:17,137","__main__","DEBUG","decompress: iput -T /tmp/iplant/20141215T024905.519428_5GB.fastq /tempZone/tmp/iplant/20141215T024905.519428_5GB.fastq" | |
| "2014-12-15 02:53:03,658","__main__","DEBUG","decompress: icp /tempZone/tmp/iplant/20141215T024905.519428_5GB.fastq /tempZone/tmp/iplant/20141215T024905.519428_5GB.fastq_copy" | |
| "2014-12-15 02:55:56,030","__main__","DEBUG","decompress: imv /tempZone/tmp/iplant/20141215T024905.519428_5GB.fastq_copy /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-15 02:56:00,290","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq IS_COMPRESSED FALSE BOOL" | |
| "2014-12-15 02:56:00,865","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMPRESSION_METHOD NONE NONE" | |
| "2014-12-15 02:56:00,918","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_SIZE 5000000065 BYTES" | |
| "2014-12-15 02:56:00,959","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_HASH a13fa24d16155850e85fb94aad6417e830f963dc NONE" | |
| "2014-12-15 02:56:01,001","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq HASH_METHOD SHA1 NONE" | |
| "2014-12-15 02:56:01,043","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq PARENT_FILE /tempZone/tmp/iplant/20141215T024905.519428_5GB.fastq.gz NONE" | |
| "2014-12-15 02:56:01,110","__main__","DEBUG","decompress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMMENTS 'This file is registered under the extension .fastq and is stored internally to iRODS without compression as .fastq.' NONE" | |
| "2014-12-15 02:56:01,151","__main__","DEBUG","decompress: delete_itmp_files = True" | |
| "2014-12-15 02:56:01,152","__main__","DEBUG","decompress: irm -f /tempZone/tmp/iplant/20141215T024905.519428_5GB.fastq" | |
| "2014-12-15 02:56:01,365","__main__","DEBUG","decompress: irm -f /tempZone/tmp/iplant/20141215T024905.519428_5GB.fastq.gz" | |
| "2014-12-15 02:56:01,592","__main__","INFO","main: END_LOGGING" | |
| "2014-12-15 02:56:30,317","__main__","INFO","main: BEGIN_LOGGING" | |
| "2014-12-15 02:56:30,317","__main__","INFO","main: Log format: '%(asctime)s','%(name)s','%(levelname)s','%(message)s'" | |
| "2014-12-15 02:56:30,317","__main__","INFO","main: Log date format: default ISO 8601, UTC" | |
| "2014-12-15 02:56:30,317","__main__","INFO","main: Compressing file." | |
| "2014-12-15 02:56:30,317","__main__","DEBUG","main: compress(ipath=/tempZone/home/rods/iplant/5GB.fastq, itmp_iplant=/tempZone/tmp/iplant, tmp_iplant=/tmp/iplant, delete_itmp_files=True, delete_tmp_files=False)" | |
| "2014-12-15 02:56:30,317","__main__","DEBUG","compress: imeta ls -d /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-15 02:56:30,340","__main__","DEBUG","compress: imeta_stdout = | |
| AVUs defined for dataObj /tempZone/home/rods/iplant/5GB.fastq: | |
| attribute: IS_COMPRESSED | |
| value: FALSE | |
| units: BOOL | |
| ---- | |
| attribute: COMMENTS | |
| value: 'This file is registered under the extension .fastq and is stored internally to iRODS without compression as .fastq.' | |
| units: NONE | |
| ---- | |
| attribute: UNCOMPRESSED_HASH | |
| value: a13fa24d16155850e85fb94aad6417e830f963dc | |
| units: NONE | |
| ---- | |
| attribute: HASH_METHOD | |
| value: SHA1 | |
| units: NONE | |
| ---- | |
| attribute: PARENT_FILE | |
| value: /tempZone/tmp/iplant/20141215T024905.519428_5GB.fastq.gz | |
| units: NONE | |
| ---- | |
| attribute: COMPRESSION_METHOD | |
| value: NONE | |
| units: NONE | |
| ---- | |
| attribute: UNCOMPRESSED_SIZE | |
| value: 5000000065 | |
| units: BYTES | |
| " | |
| "2014-12-15 02:56:30,340","__main__","DEBUG","compress: _imeta_to_dict(imeta_stdout) = | |
| {'PARENT_FILE': {'units': 'NONE', 'value': '/tempZone/tmp/iplant/20141215T024905.519428_5GB.fastq.gz'}, 'COMMENTS': {'units': 'NONE', 'value': "'This file is registered under the extension .fastq and is stored internally to iRODS without compression as .fastq.'"}, 'COMPRESSION_METHOD': {'units': 'NONE', 'value': 'NONE'}, 'IS_COMPRESSED': {'units': 'BOOL', 'value': False}, 'HASH_METHOD': {'units': 'NONE', 'value': 'SHA1'}, 'UNCOMPRESSED_HASH': {'units': 'NONE', 'value': 'a13fa24d16155850e85fb94aad6417e830f963dc'}, 'UNCOMPRESSED_SIZE': {'units': 'BYTES', 'value': 5000000065}}" | |
| "2014-12-15 02:56:30,340","__main__","DEBUG","compress: do_compress = True" | |
| "2014-12-15 02:56:30,340","__main__","DEBUG","compress: imv /tempZone/home/rods/iplant/5GB.fastq /tempZone/tmp/iplant/20141215T025630.340886_5GB.fastq" | |
| "2014-12-15 02:56:30,410","__main__","DEBUG","compress: iget -f -T /tempZone/tmp/iplant/20141215T025630.340886_5GB.fastq /tmp/iplant/20141215T025630.340886_5GB.fastq" | |
| "2014-12-15 02:58:21,664","__main__","DEBUG","compress: uncompressed_size = os.path.getsize(/tmp/iplant/20141215T025630.340886_5GB.fastq)" | |
| "2014-12-15 02:58:21,665","__main__","DEBUG","compress: uncompressed_size = 5000000065" | |
| "2014-12-15 02:58:21,665","__main__","DEBUG","compress: uncompressed_hash = _compute_hash(fpath=/tmp/iplant/20141215T025630.340886_5GB.fastq, algorithm=SHA1)" | |
| "2014-12-15 02:59:02,451","__main__","DEBUG","compress: uncompressed_hash = a13fa24d16155850e85fb94aad6417e830f963dc" | |
| "2014-12-15 02:59:02,451","__main__","DEBUG","compress: gzip --fast --force --keep /tmp/iplant/20141215T025630.340886_5GB.fastq" | |
| "2014-12-15 03:00:57,405","__main__","DEBUG","compress: iput -T /tmp/iplant/20141215T025630.340886_5GB.fastq.gz /tempZone/tmp/iplant/20141215T025630.340886_5GB.fastq.gz" | |
| "2014-12-15 03:01:25,129","__main__","DEBUG","compress: icp /tempZone/tmp/iplant/20141215T025630.340886_5GB.fastq.gz /tempZone/tmp/iplant/20141215T025630.340886_5GB.fastq.gz_copy" | |
| "2014-12-15 03:01:28,149","__main__","DEBUG","compress: imv /tempZone/tmp/iplant/20141215T025630.340886_5GB.fastq.gz_copy /tempZone/home/rods/iplant/5GB.fastq" | |
| "2014-12-15 03:01:32,454","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq IS_COMPRESSED TRUE BOOL" | |
| "2014-12-15 03:01:33,124","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMPRESSION_METHOD GZIP NONE" | |
| "2014-12-15 03:01:33,169","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_SIZE 5000000065 BYTES" | |
| "2014-12-15 03:01:33,216","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq UNCOMPRESSED_HASH a13fa24d16155850e85fb94aad6417e830f963dc NONE" | |
| "2014-12-15 03:01:33,257","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq HASH_METHOD SHA1 NONE" | |
| "2014-12-15 03:01:33,299","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq PARENT_FILE /tempZone/tmp/iplant/20141215T025630.340886_5GB.fastq NONE" | |
| "2014-12-15 03:01:33,341","__main__","DEBUG","compress: imeta set -d /tempZone/home/rods/iplant/5GB.fastq COMMENTS 'This file is registered under the extension .fastq but is stored internally to iRODS with compression as .fastq.gz. This file will be decompressed upon retrieval (e.g. with iget, isync).' NONE" | |
| "2014-12-15 03:01:33,377","__main__","DEBUG","compress: delete_itmp_files = True" | |
| "2014-12-15 03:01:33,378","__main__","DEBUG","compress: irm -f /tempZone/tmp/iplant/20141215T025630.340886_5GB.fastq" | |
| "2014-12-15 03:01:33,594","__main__","DEBUG","compress: irm -f /tempZone/tmp/iplant/20141215T025630.340886_5GB.fastq.gz" | |
| "2014-12-15 03:01:33,783","__main__","INFO","main: END_LOGGING" | |
| ubuntu@20140918t192000-irodsv331:~$ # "iplant.py --action decompress" was called by acPreprocForDataObjOpen and completed at 2014-12-15 02:56:01,592 | |
| ubuntu@20140918t192000-irodsv331:~$ # "iplant.py --action compress" was called by acPostProcForOpen at Dec 15 02:56:29 | |
| ubuntu@20140918t192000-irodsv331:~$ # iget file transfer rate is ~2.65 GB/min, which explains the ~1.3GB file size fetched. | |
| ubuntu@20140918t192000-irodsv331:~$ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment