Skip to content

Instantly share code, notes, and snippets.

@7yl4r
Last active February 21, 2020 16:56
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save 7yl4r/4412ea9ba3b7817acd5b5e1df4612e5a to your computer and use it in GitHub Desktop.
Save 7yl4r/4412ea9ba3b7817acd5b5e1df4612e5a to your computer and use it in GitHub Desktop.
# === config details:
[airflow@imars-airflow01 imars_dags]$ echo $AIRFLOW_HOME
/home/airflow
# === relevant excerpt from `tree`
[airflow@imars-airflow01 ~]$ tree $AIRFLOW_HOME/dags
/home/airflow/dags
└── imars_dags
├── imars_dags
   ├── operators
   │   ├── FileWatcher
   │   │   ├── check_filesize_match.py
   │   │   ├── check_for_duplicates.py
   │   │   ├── check_for_duplicates_test.py
   │   │   ├── check_ipfs_accessible.py
   │   │   ├── check_locally_accessible.py
   │   │   ├── FileWatcherOperator.py
| | └── __init__.py
| └── __init__.py
└── __init__.py
# === import error:
[airflow@imars-airflow01 imars_dags]$ airflow list_dags
[2020-02-21 16:47:56,590] {settings.py:254} INFO - settings.configure_orm(): Using pool settings. pool_size=1000, max_overflow=10, pool_recycle=3600, pid=12126
[2020-02-21 16:47:58,019] {__init__.py:51} INFO - Using executor CeleryExecutor
[2020-02-21 16:47:58,022] {dagbag.py:403} INFO - Filling up the DagBag from /home/airflow/dags
[2020-02-21 16:47:58,169] {airflow_log_cleanup.py:50} INFO - ENABLE_DELETE_CHILD_LOG False
[2020-02-21 16:47:58,194] {dagbag.py:246} ERROR - Failed to import: /home/airflow/dags/imars_dags/imars_dags/dags/file_watcher/file_watcher.py
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/airflow/models/dagbag.py", line 243, in process_file
m = imp.load_source(mod_name, filepath)
File "/usr/lib64/python3.6/imp.py", line 172, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 684, in _load
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/airflow/dags/imars_dags/imars_dags/dags/file_watcher/file_watcher.py", line 48, in <module>
from imars_dags.operators.FileWatcher.FileWatcherOperator \
ModuleNotFoundError: No module named 'imars_dags.operators'
-------------------------------------------------------------------
DAGS
-------------------------------------------------------------------
airflow-clear-missing-dags
airflow-db-cleanup
airflow-delete-broken-dags
airflow_db_cleanup
airflow_kill_halted_tasks
airflow_log_cleanup
bouys_to_graphite_fgbnms
csvs_to_graphite
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment