Skip to content

Instantly share code, notes, and snippets.

@hnykda
Created January 8, 2018 21:51
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save hnykda/172ee03e8f769220ffeaae3317da6805 to your computer and use it in GitHub Desktop.
Save hnykda/172ee03e8f769220ffeaae3317da6805 to your computer and use it in GitHub Desktop.
dan at think460s in ~/prac/gwi/pyxecutor (feature/add-logging-and-skip-exceptions●)
$ SKIP_ERRORS=True INIT_DEBUG=True SAVE_INTERMEDIATE_DATASETS=True pyxecutor exec_py ../pyxecutor/data/input/q4_2017/reduced/core.hdf ../lagoon/core/q4_2017/desktop_1/ data/output/q4_2017/desktop_1.hdf
20180108 22:48:54:git.cmd:DEBUG - Popen(['git', 'version'], cwd=/home/dan/prac/gwi/pyxecutor, universal_newlines=False, shell=None)
20180108 22:48:54:git.cmd:DEBUG - Popen(['git', 'version'], cwd=/home/dan/prac/gwi/pyxecutor, universal_newlines=False, shell=None)
20180108 22:48:54:s3fs.core:DEBUG - Open S3 connection. Anonymous: False
20180108 22:48:54:pyxecutor.io:INFO - Loading '...lagoon.core.q4_2017.desktop_1.main' module ...
20180108 22:48:54:pyxecutor.main:ERROR - the 'package' argument is required to perform a relative import for '...lagoon.core.q4_2017.desktop_1.main'
Traceback (most recent call last):
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/main.py", line 72, in respond
res = actions.env_exec(action, *args, **kwargs)
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/actions.py", line 152, in env_exec
res = action(*args, **kwargs)
File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/site-packages/celery/local.py", line 191, in __call__
return self._get_current_object()(*a, **kw)
File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/site-packages/celery/app/task.py", line 380, in __call__
return self.run(*args, **kwargs)
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/actions.py", line 31, in exec_py
pipeline_to_run = get_pipeline(python_resource)
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/io.py", line 134, in get_pipeline
return _load_pipeline(main_mod)
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/io.py", line 113, in _load_pipeline
mo = importlib.import_module(mod_path_py)
File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/importlib/__init__.py", line 121, in import_module
raise TypeError(msg.format(name))
TypeError: the 'package' argument is required to perform a relative import for '...lagoon.core.q4_2017.desktop_1.main'
{'status': 'failed', 'error': 'Traceback (most recent call last):\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/main.py", line 72, in respond\n res = actions.env_exec(action, *args, **kwargs)\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/actions.py", line 152, in env_exec\n res = action(*args, **kwargs)\n File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/site-packages/celery/local.py", line 191, in __call__\n return self._get_current_object()(*a, **kw)\n File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/site-packages/celery/app/task.py", line 380, in __call__\n return self.run(*args, **kwargs)\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/actions.py", line 31, in exec_py\n pipeline_to_run = get_pipeline(python_resource)\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/io.py", line 134, in get_pipeline\n return _load_pipeline(main_mod)\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/io.py", line 113, in _load_pipeline\n mo = importlib.import_module(mod_path_py)\n File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/importlib/__init__.py", line 121, in import_module\n raise TypeError(msg.format(name))\nTypeError: the \'package\' argument is required to perform a relative import for \'...lagoon.core.q4_2017.desktop_1.main\'\n', 'data': None}
(pyxecutor2)
dan at think460s in ~/prac/gwi/pyxecutor (feature/add-logging-and-skip-exceptions●)
$ cd la
(pyxecutor2)
dan at think460s in ~/prac/gwi/pyxecutor (feature/add-logging-and-skip-exceptions●)
$ cd ../lagoon/core/q4_2017
(pyxecutor2)
dan at think460s in ~/prac/gwi/lagoon/core/q4_2017 (project/core/q4_2017-testing-pyxec)
$ SKIP_ERRORS=True INIT_DEBUG=True SAVE_INTERMEDIATE_DATASETS=True pyxecutor exec_py ../../../data/input/q4_2017/reduced/core.hdf desktop_1 data/output/q4_2017/desktop_1.hdf
20180108 22:49:28:git.cmd:DEBUG - Popen(['git', 'version'], cwd=/home/dan/prac/gwi/lagoon/core/q4_2017, universal_newlines=False, shell=None)
20180108 22:49:28:git.cmd:DEBUG - Popen(['git', 'version'], cwd=/home/dan/prac/gwi/lagoon/core/q4_2017, universal_newlines=False, shell=None)
20180108 22:49:28:s3fs.core:DEBUG - Open S3 connection. Anonymous: False
20180108 22:49:28:pyxecutor.io:INFO - Loading 'desktop_1.main' module ...
20180108 22:49:28:pyxecutor.main:ERROR - No module named 'desktop_1'
Traceback (most recent call last):
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/main.py", line 72, in respond
res = actions.env_exec(action, *args, **kwargs)
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/actions.py", line 152, in env_exec
res = action(*args, **kwargs)
File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/site-packages/celery/local.py", line 191, in __call__
return self._get_current_object()(*a, **kw)
File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/site-packages/celery/app/task.py", line 380, in __call__
return self.run(*args, **kwargs)
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/actions.py", line 31, in exec_py
pipeline_to_run = get_pipeline(python_resource)
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/io.py", line 134, in get_pipeline
return _load_pipeline(main_mod)
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/io.py", line 113, in _load_pipeline
mo = importlib.import_module(mod_path_py)
File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'desktop_1'
{'status': 'failed', 'error': 'Traceback (most recent call last):\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/main.py", line 72, in respond\n res = actions.env_exec(action, *args, **kwargs)\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/actions.py", line 152, in env_exec\n res = action(*args, **kwargs)\n File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/site-packages/celery/local.py", line 191, in __call__\n return self._get_current_object()(*a, **kw)\n File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/site-packages/celery/app/task.py", line 380, in __call__\n return self.run(*args, **kwargs)\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/actions.py", line 31, in exec_py\n pipeline_to_run = get_pipeline(python_resource)\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/io.py", line 134, in get_pipeline\n return _load_pipeline(main_mod)\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/io.py", line 113, in _load_pipeline\n mo = importlib.import_module(mod_path_py)\n File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/importlib/__init__.py", line 126, in import_module\n return _bootstrap._gcd_import(name[level:], package, level)\n File "<frozen importlib._bootstrap>", line 994, in _gcd_import\n File "<frozen importlib._bootstrap>", line 971, in _find_and_load\n File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked\n File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n File "<frozen importlib._bootstrap>", line 994, in _gcd_import\n File "<frozen importlib._bootstrap>", line 971, in _find_and_load\n File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked\nModuleNotFoundError: No module named \'desktop_1\'\n', 'data': None}
(pyxecutor2)
dan at think460s in ~/prac/gwi/lagoon/core/q4_2017 (project/core/q4_2017-testing-pyxec)
$ SKIP_ERRORS=True INIT_DEBUG=True SAVE_INTERMEDIATE_DATASETS=True pyxecutor exec_py ../../../data/input/q4_2017/reduced/core.hdf desktop_1/main.py data/output/q4_2017/desktop_1.hdf
20180108 22:49:37:git.cmd:DEBUG - Popen(['git', 'version'], cwd=/home/dan/prac/gwi/lagoon/core/q4_2017, universal_newlines=False, shell=None)
20180108 22:49:37:git.cmd:DEBUG - Popen(['git', 'version'], cwd=/home/dan/prac/gwi/lagoon/core/q4_2017, universal_newlines=False, shell=None)
20180108 22:49:37:s3fs.core:DEBUG - Open S3 connection. Anonymous: False
20180108 22:49:38:pyxecutor.io:INFO - Loading 'desktop_1.main' module ...
20180108 22:49:38:pyxecutor.main:ERROR - No module named 'desktop_1'
Traceback (most recent call last):
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/main.py", line 72, in respond
res = actions.env_exec(action, *args, **kwargs)
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/actions.py", line 152, in env_exec
res = action(*args, **kwargs)
File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/site-packages/celery/local.py", line 191, in __call__
return self._get_current_object()(*a, **kw)
File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/site-packages/celery/app/task.py", line 380, in __call__
return self.run(*args, **kwargs)
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/actions.py", line 31, in exec_py
pipeline_to_run = get_pipeline(python_resource)
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/io.py", line 139, in get_pipeline
return _load_pipeline(python_resource)
File "/home/dan/prac/gwi/pyxecutor/pyxecutor/io.py", line 113, in _load_pipeline
mo = importlib.import_module(mod_path_py)
File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'desktop_1'
{'status': 'failed', 'error': 'Traceback (most recent call last):\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/main.py", line 72, in respond\n res = actions.env_exec(action, *args, **kwargs)\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/actions.py", line 152, in env_exec\n res = action(*args, **kwargs)\n File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/site-packages/celery/local.py", line 191, in __call__\n return self._get_current_object()(*a, **kw)\n File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/site-packages/celery/app/task.py", line 380, in __call__\n return self.run(*args, **kwargs)\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/actions.py", line 31, in exec_py\n pipeline_to_run = get_pipeline(python_resource)\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/io.py", line 139, in get_pipeline\n return _load_pipeline(python_resource)\n File "/home/dan/prac/gwi/pyxecutor/pyxecutor/io.py", line 113, in _load_pipeline\n mo = importlib.import_module(mod_path_py)\n File "/home/dan/.local/opt/miniconda/envs/pyxecutor2/lib/python3.6/importlib/__init__.py", line 126, in import_module\n return _bootstrap._gcd_import(name[level:], package, level)\n File "<frozen importlib._bootstrap>", line 994, in _gcd_import\n File "<frozen importlib._bootstrap>", line 971, in _find_and_load\n File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked\n File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n File "<frozen importlib._bootstrap>", line 994, in _gcd_import\n File "<frozen importlib._bootstrap>", line 971, in _find_and_load\n File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked\nModuleNotFoundError: No module named \'desktop_1\'\n', 'data': None}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment