Created
July 11, 2019 11:17
-
-
Save theSekyi/a0865524837b8741580abbe8b3e8fd05 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ docker-compose up orchestrator | |
| code-challenge-2019_dask-scheduler_1 is up-to-date | |
| code-challenge-2019_luigid_1 is up-to-date | |
| Recreating code-challenge-2019_orchestrator_1 ... done | |
| Attaching to code-challenge-2019_orchestrator_1 | |
| orchestrator_1 | DEBUG: Checking if MakeDatasets(no_remove_finished=False, out_dir=/usr/share/data/interim/) is complete | |
| orchestrator_1 | DEBUG: Checking if DownloadData(no_remove_finished=False, fname=wine_dataset, out_dir=/usr/share/data/raw/, url=https://github.com/datarevenue-berlin/code-challenge-2019/releases/download/0.1.0/dataset_sampled.csv) is complete | |
| orchestrator_1 | INFO: Informed scheduler that task MakeDatasets_False__usr_share_data__006671ec2e has status PENDING | |
| orchestrator_1 | INFO: Informed scheduler that task DownloadData_wine_dataset_False__usr_share_data__79bc385f2e has status PENDING | |
| orchestrator_1 | INFO: Done scheduling tasks | |
| orchestrator_1 | INFO: Running Worker with 1 processes | |
| orchestrator_1 | DEBUG: Asking scheduler for work... | |
| orchestrator_1 | DEBUG: Pending tasks: 2 | |
| orchestrator_1 | INFO: [pid 1] Worker Worker(salt=650343655, workers=1, host=fa2cdb3fb6c5, username=root, pid=1) running DownloadData(no_remove_finished=False, fname=wine_dataset, out_dir=/usr/share/data/raw/, url=https://github.com/datarevenue-berlin/code-challenge-2019/releases/download/0.1.0/dataset_sampled.csv) | |
| orchestrator_1 | INFO: INFO:download-data:Downloading dataset | |
| orchestrator_1 | INFO: INFO:download-data:Will write to /usr/share/data/raw/wine_dataset.csv | |
| orchestrator_1 | INFO: [pid 1] Worker Worker(salt=650343655, workers=1, host=fa2cdb3fb6c5, username=root, pid=1) done DownloadData(no_remove_finished=False, fname=wine_dataset, out_dir=/usr/share/data/raw/, url=https://github.com/datarevenue-berlin/code-challenge-2019/releases/download/0.1.0/dataset_sampled.csv) | |
| orchestrator_1 | DEBUG: 1 running tasks, waiting for next task to finish | |
| orchestrator_1 | INFO: Informed scheduler that task DownloadData_wine_dataset_False__usr_share_data__79bc385f2e has status DONE | |
| orchestrator_1 | DEBUG: Asking scheduler for work... | |
| orchestrator_1 | DEBUG: Pending tasks: 1 | |
| orchestrator_1 | INFO: [pid 1] Worker Worker(salt=650343655, workers=1, host=fa2cdb3fb6c5, username=root, pid=1) running MakeDatasets(no_remove_finished=False, out_dir=/usr/share/data/interim/) | |
| orchestrator_1 | ERROR: [pid 1] Worker Worker(salt=650343655, workers=1, host=fa2cdb3fb6c5, username=root, pid=1) failed MakeDatasets(no_remove_finished=False, out_dir=/usr/share/data/interim/) | |
| orchestrator_1 | Traceback (most recent call last): | |
| orchestrator_1 | File "/usr/local/lib/python3.6/site-packages/luigi/worker.py", line 184, in run | |
| orchestrator_1 | raise RuntimeError('Unfulfilled %s at run time: %s' % (deps, ', '.join(missing))) | |
| orchestrator_1 | RuntimeError: Unfulfilled dependency at run time: DownloadData_wine_dataset_False__usr_share_data__79bc385f2e | |
| orchestrator_1 | DEBUG: 1 running tasks, waiting for next task to finish | |
| orchestrator_1 | INFO: Informed scheduler that task MakeDatasets_False__usr_share_data__006671ec2e has status FAILED | |
| orchestrator_1 | DEBUG: Checking if MakeDatasets(no_remove_finished=False, out_dir=/usr/share/data/interim/) is complete | |
| orchestrator_1 | DEBUG: Checking if DownloadData(no_remove_finished=False, fname=wine_dataset, out_dir=/usr/share/data/raw/, url=https://github.com/datarevenue-berlin/code-challenge-2019/releases/download/0.1.0/dataset_sampled.csv) is complete | |
| orchestrator_1 | INFO: Informed scheduler that task MakeDatasets_False__usr_share_data__006671ec2e has status PENDING | |
| orchestrator_1 | INFO: Informed scheduler that task DownloadData_wine_dataset_False__usr_share_data__79bc385f2e has status PENDING | |
| orchestrator_1 | DEBUG: Asking scheduler for work... | |
| orchestrator_1 | DEBUG: Pending tasks: 2 | |
| orchestrator_1 | INFO: [pid 1] Worker Worker(salt=650343655, workers=1, host=fa2cdb3fb6c5, username=root, pid=1) running DownloadData(no_remove_finished=False, fname=wine_dataset, out_dir=/usr/share/data/raw/, url=https://github.com/datarevenue-berlin/code-challenge-2019/releases/download/0.1.0/dataset_sampled.csv) | |
| orchestrator_1 | INFO: INFO:download-data:Downloading dataset | |
| orchestrator_1 | INFO: INFO:download-data:Will write to /usr/share/data/raw/wine_dataset.csv | |
| orchestrator_1 | INFO: [pid 1] Worker Worker(salt=650343655, workers=1, host=fa2cdb3fb6c5, username=root, pid=1) done DownloadData(no_remove_finished=False, fname=wine_dataset, out_dir=/usr/share/data/raw/, url=https://github.com/datarevenue-berlin/code-challenge-2019/releases/download/0.1.0/dataset_sampled.csv) | |
| orchestrator_1 | DEBUG: 1 running tasks, waiting for next task to finish | |
| orchestrator_1 | INFO: Informed scheduler that task DownloadData_wine_dataset_False__usr_share_data__79bc385f2e has status DONE | |
| orchestrator_1 | DEBUG: Asking scheduler for work... | |
| orchestrator_1 | DEBUG: Pending tasks: 1 | |
| orchestrator_1 | INFO: [pid 1] Worker Worker(salt=650343655, workers=1, host=fa2cdb3fb6c5, username=root, pid=1) running MakeDatasets(no_remove_finished=False, out_dir=/usr/share/data/interim/) | |
| orchestrator_1 | ERROR: [pid 1] Worker Worker(salt=650343655, workers=1, host=fa2cdb3fb6c5, username=root, pid=1) failed MakeDatasets(no_remove_finished=False, out_dir=/usr/share/data/interim/) | |
| orchestrator_1 | Traceback (most recent call last): | |
| orchestrator_1 | File "/usr/local/lib/python3.6/site-packages/luigi/worker.py", line 184, in run | |
| orchestrator_1 | raise RuntimeError('Unfulfilled %s at run time: %s' % (deps, ', '.join(missing))) | |
| orchestrator_1 | RuntimeError: Unfulfilled dependency at run time: DownloadData_wine_dataset_False__usr_share_data__79bc385f2e | |
| orchestrator_1 | DEBUG: 1 running tasks, waiting for next task to finish | |
| orchestrator_1 | INFO: Informed scheduler that task MakeDatasets_False__usr_share_data__006671ec2e has status FAILED | |
| orchestrator_1 | DEBUG: Asking scheduler for work... | |
| orchestrator_1 | DEBUG: Done | |
| orchestrator_1 | DEBUG: There are no more tasks to run at this time | |
| orchestrator_1 | DEBUG: There are 1 pending tasks possibly being run by other workers | |
| orchestrator_1 | DEBUG: There are 1 pending tasks unique to this worker | |
| orchestrator_1 | DEBUG: There are 1 pending tasks last scheduled by this worker | |
| orchestrator_1 | INFO: Worker Worker(salt=650343655, workers=1, host=fa2cdb3fb6c5, username=root, pid=1) was stopped. Shutting down Keep-Alive thread | |
| orchestrator_1 | INFO: | |
| orchestrator_1 | ===== Luigi Execution Summary ===== | |
| orchestrator_1 | | |
| orchestrator_1 | Scheduled 2 tasks of which: | |
| orchestrator_1 | * 1 ran successfully: | |
| orchestrator_1 | - 1 DownloadData(no_remove_finished=False, fname=wine_dataset, out_dir=/usr/share/data/raw/, url=https://github.com/datarevenue-berlin/code-challenge-2019/releases/download/0.1.0/dataset_sampled.csv) | |
| orchestrator_1 | * 1 failed: | |
| orchestrator_1 | - 1 MakeDatasets(no_remove_finished=False, out_dir=/usr/share/data/interim/) | |
| orchestrator_1 | | |
| orchestrator_1 | This progress looks :( because there were failed tasks | |
| orchestrator_1 | | |
| orchestrator_1 | ===== Luigi Execution Summary ===== | |
| orchestrator_1 | | |
| code-challenge-2019_orchestrator_1 exited with code 0 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment