Skip to content

Instantly share code, notes, and snippets.

@epifanio
Created January 26, 2023 08:03
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save epifanio/1c86ffcd1b1d1612a9478a736b47fe9b to your computer and use it in GitHub Desktop.
celery_1 | [2023-01-26 08:00:36,026: INFO/MainProcess] Task satpy_pygeoapi_plugin.process_netcdf.execute[83f43a26-9d4f-11ed-92b6-0242c0a82004] received
celery_1 | [2023-01-26 08:00:36,026: DEBUG/MainProcess] TaskPool: Apply <function fast_trace_task at 0x7f17278e9120> (args:('satpy_pygeoapi_plugin.process_netcdf.execute', '83f43a26-9d4f-11ed-92b6-0242c0a82004', {'lang': 'py', 'task': 'satpy_pygeoapi_plugin.process_netcdf.execute', 'id': '83f43a26-9d4f-11ed-92b6-0242c0a82004', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': '83f43a26-9d4f-11ed-92b6-0242c0a82004', 'parent_id': None, 'argsrepr': "({'message': 'An optional message.', 'name': 'netcdf', 'netcdf_file': '/data/noaa19-avhrr-20230124115334-20230124120327.nc'}, {'message': 'An optional message.', 'name': 'netcdf', 'netcdf_file': '/data/noaa19-avhrr-20230124115334-20230124120327.nc'})", 'kwargsrepr': '{}', 'origin': 'gen84@pygeoapi', 'ignore_result': False, 'properties': {'correlation_id': '83f43a26-9d4f-11ed-92b6-0242c0a82004', 'reply_to': 'c8a4793f-41de-36d6-a461-578f026ab8bf', 'delivery_mode': 2, 'delivery_info': {'exchange': '', 'routing_key': 'celery'}, 'priority': 0, 'body_encoding': 'base64', 'delivery_tag':... kwargs:{})
celery_1 | [2023-01-26 08:00:36,028: WARNING/ForkPoolWorker-31] MESSAGE
celery_1 | [2023-01-26 08:00:36,028: WARNING/ForkPoolWorker-31]
celery_1 | [2023-01-26 08:00:36,028: WARNING/ForkPoolWorker-31] An optional message.
celery_1 | [2023-01-26 08:00:36,028: WARNING/ForkPoolWorker-31] ########################################################
celery_1 | [2023-01-26 08:00:36,028: WARNING/ForkPoolWorker-31] /data/noaa19-avhrr-20230124115334-20230124120327.nc
celery_1 | [2023-01-26 08:00:36,028: WARNING/ForkPoolWorker-31] ########################################################
celery_1 | [2023-01-26 08:00:36,028: WARNING/ForkPoolWorker-31] Pattern match:
celery_1 | [2023-01-26 08:00:36,029: WARNING/ForkPoolWorker-31]
celery_1 | [2023-01-26 08:00:36,029: WARNING/ForkPoolWorker-31] ('/data/', 'noaa19', 'avhrr', '20230124115334', '20230124120327')
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31] START TIME:
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31]
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31] 2023-01-24 11:53:34
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31] Similar netcdf paths:
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31]
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31] ['/data/noaa19-avhrr-20230124115334-20230124120327.nc']
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31] satpy product/layer
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31]
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31] ['overview']
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31] environ({'PYGEOAPI_CONFIG': '/satpy/docker/example-config.yml', 'REDIS_PORT': '6379', 'HOSTNAME': 'celery', 'DEB_BUILD_DEPS': ' curl unzip', 'SHLVL': '1', 'HOME': '/root', 'PYTHONUNBUFFERED': '1', 'PYGEOAPI_OPENAPI': 'example-openapi.yml', 'DEB_PACKAGES': ' locales tzdata gunicorn python3-dateutil python3-flask python3-flask-cors python3-gevent python3-greenlet python3-pip python3-tz python3-unicodecsv python3-yaml libsqlite3-mod-spatialite python3-dask python3-elasticsearch python3-fiona python3-gdal python3-netcdf4 python3-pandas python3-psycopg2 python3-pymongo python3-pyproj python3-rasterio python3-scipy python3-shapely python3-tinydb python3-xarray python3-zarr python3-mapscript python3-pytest python3-pyld', '_': '/usr/bin/celery', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'LANG': 'en_US.UTF-8', 'DEBIAN_FRONTEND': 'noninteractive', 'REDIS_PASSWORD': 'FGX0Im8cROKpFpHPOP8FETmOpLPsiLwEnHHXprOgn2AR7r7cGp', 'REDIS_HOST': 'redis', 'PWD': '/pygeoapi', 'TZ': 'Etc/UTC', 'CURL_CA_BUNDLE': '/etc/ssl/certs/ca-certificates.crt', '_MP_FORK_LOGLEVEL_': '10', '_MP_FORK_LOGFILE_': '', '_MP_FORK_LOGFORMAT_': '[%(asctime)s: %(levelname)s/%(processName)s] %(message)s', 'CELERY_LOG_LEVEL': '10', 'CELERY_LOG_FILE': '', 'CELERY_LOG_REDIRECT': '1', 'CELERY_LOG_REDIRECT_LEVEL': 'WARNING'})
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31] Need to generate:
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31]
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31] ['overview']
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31] 2023-01-26 08:00:36.030770
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31]
celery_1 | [2023-01-26 08:00:36,030: WARNING/ForkPoolWorker-31] Before Scene
celery_1 | [2023-01-26 08:00:36,031: DEBUG/ForkPoolWorker-31] Reading ('/usr/lib/python3/dist-packages/satpy/etc/readers/satpy_cf_nc.yaml',)
celery_1 | [2023-01-26 08:00:36,034: DEBUG/ForkPoolWorker-31] Assigning to satpy_cf_nc: ['/data/noaa19-avhrr-20230124115334-20230124120327.nc']
celery_1 | [2023-01-26 08:00:36,054: WARNING/ForkPoolWorker-31] 2023-01-26 08:00:36.054504
celery_1 | [2023-01-26 08:00:36,054: WARNING/ForkPoolWorker-31]
celery_1 | [2023-01-26 08:00:36,054: WARNING/ForkPoolWorker-31] Before load
celery_1 | [2023-01-26 08:00:36,058: DEBUG/ForkPoolWorker-31] Looking for composites config file advanced very high resolution radiometer - 3.yaml
celery_1 | [2023-01-26 08:00:36,059: DEBUG/ForkPoolWorker-31] No composite config found called advanced very high resolution radiometer - 3.yaml
celery_1 | [2023-01-26 08:00:36,060: ERROR/ForkPoolWorker-31] Task satpy_pygeoapi_plugin.process_netcdf.execute[83f43a26-9d4f-11ed-92b6-0242c0a82004] raised unexpected: KeyError("Unknown datasets: {DataQuery(name='overview')}")
celery_1 | Traceback (most recent call last):
celery_1 | File "/usr/lib/python3/dist-packages/satpy/scene.py", line 1266, in _update_dependency_tree
celery_1 | self._dependency_tree.populate_with_keys(needed_datasets, query)
celery_1 | File "/usr/lib/python3/dist-packages/satpy/dependency_tree.py", line 262, in populate_with_keys
celery_1 | raise MissingDependencies(unknown_datasets, "Unknown datasets:")
celery_1 | satpy.node.MissingDependencies: Unknown datasets: {DataQuery(name='overview')}
celery_1 |
celery_1 | During handling of the above exception, another exception occurred:
celery_1 |
celery_1 | Traceback (most recent call last):
celery_1 | File "/usr/lib/python3/dist-packages/celery/app/trace.py", line 451, in trace_task
celery_1 | R = retval = fun(*args, **kwargs)
celery_1 | File "/usr/lib/python3/dist-packages/celery/app/trace.py", line 734, in __protected_call__
celery_1 | return self.run(*args, **kwargs)
celery_1 | File "/satpy/satpy_pygeoapi_plugin/process_netcdf.py", line 284, in execute
celery_1 | _generate_satpy_geotiff(similar_netcdf_paths, satpy_products_to_generate)
celery_1 | File "/satpy/satpy_pygeoapi_plugin/process_netcdf.py", line 165, in _generate_satpy_geotiff
celery_1 | swath_scene.load(satpy_products)
celery_1 | File "/usr/lib/python3/dist-packages/satpy/scene.py", line 1255, in load
celery_1 | self._update_dependency_tree(needed_datasets, query)
celery_1 | File "/usr/lib/python3/dist-packages/satpy/scene.py", line 1268, in _update_dependency_tree
celery_1 | raise KeyError(str(err))
celery_1 | KeyError: "Unknown datasets: {DataQuery(name='overview')}"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment