problem importing job outputs

gistfile1.txt

Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]: galaxy.jobs.runners.condor DEBUG 2024-06-13 22:27:12,690 [pN:handler_sn06_0,p:3772118,tN:CondorRunner.monitor_thread] (70816209/50973860) job is now running
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]: galaxy.jobs.runners.condor DEBUG 2024-06-13 22:27:12,695 [pN:handler_sn06_0,p:3772118,tN:CondorRunner.monitor_thread] (70816209/50973860) job has completed
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]: galaxy.jobs ERROR 2024-06-13 22:27:12,732 [pN:handler_sn06_0,p:3772118,tN:CondorRunner.work_thread-2] problem importing job outputs. stdout [] stderr [
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]: Traceback (most recent call last):
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/data/jwd05e/main/070/816/70816209/metadata/set.py", line 5, in <module>
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata()
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:                                                                ^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/metadata/set_metadata.py", line 153, in set_metadata
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     set_metadata_portable()
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/metadata/set_metadata.py", line 214, in set_metadata_portable
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     object_store = get_object_store(
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:                    ^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/metadata/set_metadata.py", line 173, in get_object_store
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     object_store = build_object_store_from_config(None, config_dict=config_dict, disable_process_management=True)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/objectstore/__init__.py", line 1505, in build_object_store_from_config
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     return objectstore_class(config=config, config_dict=config_dict, **objectstore_constructor_kwds)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/objectstore/__init__.py", line 1076, in __init__
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     backend = build_object_store_from_config(config, config_dict=backend_def, fsmon=fsmon)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/objectstore/__init__.py", line 1505, in build_object_store_from_config
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     return objectstore_class(config=config, config_dict=config_dict, **objectstore_constructor_kwds)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/objectstore/s3.py", line 196, in __init__
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     self._initialize()
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/objectstore/s3.py", line 215, in _initialize
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     self._bucket = self._get_bucket(self.bucket)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/objectstore/s3.py", line 257, in _get_bucket
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     bucket = self.conn.get_bucket(bucket_name)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/venv/lib/python3.11/site-packages/boto/s3/connection.py", line 509, in get_bucket
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     return self.head_bucket(bucket_name, headers=headers)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/venv/lib/python3.11/site-packages/boto/s3/connection.py", line 528, in head_bucket
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     response = self.make_request('HEAD', bucket_name, headers=headers)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/venv/lib/python3.11/site-packages/boto/s3/connection.py", line 667, in make_request
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     return super(S3Connection, self).make_request(
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/venv/lib/python3.11/site-packages/boto/connection.py", line 1070, in make_request
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     return self._mexe(http_request, sender, override_num_retries,
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/venv/lib/python3.11/site-packages/boto/connection.py", line 1030, in _mexe
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     raise ex
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/venv/lib/python3.11/site-packages/boto/connection.py", line 942, in _mexe
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     connection.request(request.method, request.path,
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/usr/local/tools/_conda/envs/__python@3.11.5/lib/python3.11/http/client.py", line 1286, in request
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     self._send_request(method, url, body, headers, encode_chunked)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/usr/local/tools/_conda/envs/__python@3.11.5/lib/python3.11/http/client.py", line 1332, in _send_request
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     self.endheaders(body, encode_chunked=encode_chunked)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/usr/local/tools/_conda/envs/__python@3.11.5/lib/python3.11/http/client.py", line 1281, in endheaders
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     self._send_output(message_body, encode_chunked=encode_chunked)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/usr/local/tools/_conda/envs/__python@3.11.5/lib/python3.11/http/client.py", line 1041, in _send_output
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     self.send(msg)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/usr/local/tools/_conda/envs/__python@3.11.5/lib/python3.11/http/client.py", line 979, in send
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     self.connect()
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/venv/lib/python3.11/site-packages/boto/https_connection.py", line 128, in connect
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     self.sock = ssl.wrap_socket(sock, keyfile=self.key_file,
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/usr/local/tools/_conda/envs/__python@3.11.5/lib/python3.11/ssl.py", line 1479, in wrap_socket
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     return context.wrap_socket(
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:            ^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/usr/local/tools/_conda/envs/__python@3.11.5/lib/python3.11/ssl.py", line 517, in wrap_socket
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     return self.sslsocket_class._create(
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/usr/local/tools/_conda/envs/__python@3.11.5/lib/python3.11/ssl.py", line 1108, in _create
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     self.do_handshake()
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/usr/local/tools/_conda/envs/__python@3.11.5/lib/python3.11/ssl.py", line 1379, in do_handshake
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     self._sslobj.do_handshake()
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]: ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1006)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]: ]
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]: Traceback (most recent call last):
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/jobs/__init__.py", line 1916, in finish
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     import_model_store.perform_import(history=job.history, job=job)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/model/store/__init__.py", line 406, in perform_import
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     datasets_attrs = self.datasets_properties()
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/model/store/__init__.py", line 1539, in datasets_properties
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     datasets_attrs = load(open(datasets_attrs_file_name))
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]: FileNotFoundError: [Errno 2] No such file or directory: '/data/jwd05e/main/070/816/70816209/metadata/outputs_populated/datasets_attrs.txt'
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]: galaxy.jobs.runners ERROR 2024-06-13 22:27:12,746 [pN:handler_sn06_0,p:3772118,tN:CondorRunner.work_thread-2] (70816209/50973860) Job wrapper finish method failed
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]: Traceback (most recent call last):
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/jobs/runners/__init__.py", line 656, in _finish_or_resubmit_job
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     job_wrapper.finish(
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/jobs/__init__.py", line 1916, in finish
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     import_model_store.perform_import(history=job.history, job=job)
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/model/store/__init__.py", line 406, in perform_import
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     datasets_attrs = self.datasets_properties()
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:   File "/opt/galaxy/server/lib/galaxy/model/store/__init__.py", line 1539, in datasets_properties
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:     datasets_attrs = load(open(datasets_attrs_file_name))
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]:                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 13 22:27:12 sn06.galaxyproject.eu python[3772118]: FileNotFoundError: [Errno 2] No such file or directory: '/data/jwd05e/main/070/816/70816209/metadata/outputs_populated/datasets_attrs.txt'