- Clean install of CentOS 7.4.1708
- Grab the AWIPS 17.2.1 RPMS and setup a repo for them
yum groupinstall "AWIPS II Standalone"
As expected it failed with netcdf dependancies:
Error: Package: awips2-postgresql-9.5.3-17.2.1.19.el7.x86_64 (awips2repo)
Requires: netcdf
Error: Package: awips2-database-17.2.1-16.noarch (awips2noarch)
Requires: netcdf-devel = 4.1.2
So grabbed them from a copy we keep from previous attempts & installed
wget https://s3-us-west-2.amazonaws.com/gina-awips2/17.2.1/x86_64/netcdf-4.1.2-1.el7.rf.x86_64.rpm
wget https://s3-us-west-2.amazonaws.com/gina-awips2/17.2.1/x86_64/netcdf-devel-4.1.2-1.el7.rf.x86_64.rpm
rpm -Uvh netcdf-4.1.2-1.el7.rf.x86_64.rpm netcdf-devel-4.1.2-1.el7.rf.x86_64.rpm
That failed due, as expected, due to missing dependancies on libgfortran:
warning: netcdf-4.1.2-1.el7.rf.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 6b8d79e6: NOKEY
error: Failed dependencies:
libgfortran.so.3()(64bit) is needed by netcdf-4.1.2-1.el7.rf.x86_64
libgfortran.so.3(GFORTRAN_1.0)(64bit) is needed by netcdf-4.1.2-1.el7.rf.x86_64
libquadmath.so.0()(64bit) is needed by netcdf-4.1.2-1.el7.rf.x86_64
yum install libgfortran
rpm -Uvh netcdf-4.1.2-1.el7.rf.x86_64.rpm netcdf-devel-4.1.2-1.el7.rf.x86_64.rpm
Successful install of netcdf-4.1.2-1 after installing libgfortran.
Install of AWIPS II Standalone again:
yum groupinstall "AWIPS II Standalone"
Error: Package: awips2-pgadmin3-1.22.1-17.2.1.13.el7.x86_64 (awips2repo)
Requires: wxGTK >= 2.8.12-1
We handled this with adding the epel extra packages repo:
cat /etc/yum.repos.d/epel.repo
[epel]
name=Extra Packages for Enterprise Linux
baseurl=https://pubmirror1.math.uh.edu/fedora-buffet/epel/7/x86_64/
enabled=1
fastestmirror_enabled=0
gpgcheck=1
gpgkey=https://pubmirror1.math.uh.edu/fedora-buffet/epel/RPM-GPG-KEY-EPEL-7
yum groupinstall "AWIPS II Standalone"
Successful this time around.
Installed:
awips2.noarch 0:17.2.1-43 awips2-adapt-native.noarch 0:17.2.1-22
awips2-aviation-shared.noarch 0:17.2.1-22 awips2-cli.noarch 0:17.2.1-42
awips2-common-base.x86_64 0:17.2.1-26 awips2-data.gfe.noarch 0:17.2.1-13
awips2-data.hdf5-topo.noarch 0:17.2.1-13 awips2-database.noarch 0:17.2.1-16
awips2-database-standalone-configuration.noarch 0:17.2.1-13 awips2-edex.x86_64 0:17.2.1-16
awips2-edex-archive.x86_64 0:17.2.1-22 awips2-edex-base.x86_64 0:17.2.1-36
awips2-edex-bufr.x86_64 0:17.2.1-22 awips2-edex-common-core.x86_64 0:17.2.1-22
awips2-edex-config-auto.x86_64 0:17.2.1-16 awips2-edex-configuration.x86_64 0:17.2.1-24
awips2-edex-convectprob.x86_64 0:17.2.1-16 awips2-edex-core.x86_64 0:17.2.1-26
awips2-edex-cots.x86_64 0:17.2.1-16 awips2-edex-dat.x86_64 0:17.2.1-22
awips2-edex-datadelivery-client.x86_64 0:17.2.1-16 awips2-edex-datadelivery-core.x86_64 0:17.2.1-16
awips2-edex-dataplugins.x86_64 0:17.2.1-42 awips2-edex-geodb.x86_64 0:17.2.1-16
awips2-edex-gfe.x86_64 0:17.2.1-43 awips2-edex-glmdecoder.x86_64 0:17.2.1-16
awips2-edex-goesr.x86_64 0:17.2.1-16 awips2-edex-grib-decoderpostprocessor.x86_64 0:17.2.1-16
awips2-edex-grid.x86_64 0:17.2.1-42 awips2-edex-hydro.x86_64 0:17.2.1-22
awips2-edex-mping.x86_64 0:17.2.1-16 awips2-edex-ncep.x86_64 0:17.2.1-16
awips2-edex-npp.x86_64 0:17.2.1-16 awips2-edex-ogc-core.x86_64 0:17.2.1-16
awips2-edex-ogc-wfs.x86_64 0:17.2.1-16 awips2-edex-ost.x86_64 0:17.2.1-16
awips2-edex-radar.x86_64 0:17.2.1-22 awips2-edex-registry-client.x86_64 0:17.2.1-16
awips2-edex-registry-request.x86_64 0:17.2.1-16 awips2-edex-remote-script.x86_64 0:17.2.1-16
awips2-edex-satellite.x86_64 0:17.2.1-16 awips2-edex-shapefiles.noarch 0:17.2.1-16
awips2-edex-sportlma.x86_64 0:17.2.1-16 awips2-edex-text.x86_64 0:17.2.1-23
awips2-gfesuite-client.noarch 0:17.2.1-36 awips2-gfesuite-server.noarch 0:17.2.1-36
awips2-httpd-pypies.x86_64 0:2.4.23-2.el7 awips2-httpd-pypies-tools.x86_64 0:2.4.23-2.el7
awips2-hydroapps-shared.x86_64 0:17.2.1-23 awips2-java.x86_64 0:1.8.0_101-17.2.1.13.el7
awips2-localapps-environment.noarch 0:17.2.1-13 awips2-maps-database.noarch 0:17.2.1-13
awips2-ncep-database.noarch 0:17.2.1-13 awips2-notification.i386 0:17.2.1-22
awips2-pgadmin3.x86_64 0:1.22.1-17.2.1.13.el7 awips2-postgresql.x86_64 0:9.5.3-17.2.1.19.el7
awips2-psql.x86_64 0:9.5.3-17.2.1.13.el7 awips2-pypies.x86_64 0:17.2.1-22
awips2-python.x86_64 0:2.7.12-17.2.1.13.el7 awips2-python-dateutil.x86_64 0:2.4.2-2.7.12.1.el7
awips2-python-dynamicserialize.noarch 0:17.2.1-2.7.12.13.el7 awips2-python-h5py.x86_64 0:1.3.0-2.7.12.1.9.2.1.el7
awips2-python-jep.x86_64 0:3.5.3-2.el7 awips2-python-matplotlib.x86_64 0:1.4.3-17.2.1.2.7.12.1.9.2.1.el7
awips2-python-nose.noarch 0:0.11.1-2.7.12.1.el7 awips2-python-numpy.x86_64 0:1.9.2-2.el7
awips2-python-pmw.x86_64 0:1.3.2-2.7.12.1.el7 awips2-python-pupynere.noarch 0:1.0.13-2.7.12.1.9.2.1.el7
awips2-python-pyparsing.x86_64 0:2.0.3-2.7.12.1.el7 awips2-python-pytz.x86_64 0:2015.4-2.7.12.1.9.2.1.el7
awips2-python-qpid.noarch 0:0.32-2.7.12.3.el7 awips2-python-scientific.x86_64 0:2.8-2.7.12.1.el7
awips2-python-scipy.x86_64 0:0.15.1-2.7.12.1.9.2.1.el7 awips2-python-setuptools.x86_64 0:18.0.1-2.7.12.1.el7
awips2-python-shapely.x86_64 0:1.4.4-2.7.12.1.el7 awips2-python-six.x86_64 0:1.9.0-2.7.12.1.el7
awips2-python-tables.x86_64 0:2.1.2-2.7.12.1.9.2.1.el7 awips2-python-thrift.noarch 0:20080411p1-2.7.12.1.el7
awips2-python-tpg.noarch 0:3.1.2-2.7.12.1.el7 awips2-python-ufpy.noarch 0:17.2.1-2.7.12.22.el7
awips2-python-werkzeug.noarch 0:0.6.2-2.7.12.1.el7 awips2-qpid-java-broker.noarch 0:0.32-9.el7
awips2-qpid-java-client.noarch 0:0.32-2 awips2-qpid-java-common.noarch 0:0.32-2
awips2-rcm.x86_64 0:17.2.1-26 awips2-tools.x86_64 0:1.8.5-17.2.1.22.el7
awips2-yajsw.noarch 0:11.11-17.2.1.17.el7
Dependency Installed:
SDL.x86_64 0:1.2.15-14.el7 atk.x86_64 0:2.22.0-3.el7 avahi-libs.x86_64 0:0.6.31-17.el7
awips2-qpid-lib.i386 0:0.32-4.el7 boost.x86_64 0:1.53.0-27.el7 boost-atomic.x86_64 0:1.53.0-27.el7
boost-chrono.x86_64 0:1.53.0-27.el7 boost-context.x86_64 0:1.53.0-27.el7 boost-filesystem.x86_64 0:1.53.0-27.el7
boost-graph.x86_64 0:1.53.0-27.el7 boost-iostreams.x86_64 0:1.53.0-27.el7 boost-locale.x86_64 0:1.53.0-27.el7
boost-math.x86_64 0:1.53.0-27.el7 boost-program-options.x86_64 0:1.53.0-27.el7 boost-python.x86_64 0:1.53.0-27.el7
boost-random.x86_64 0:1.53.0-27.el7 boost-regex.x86_64 0:1.53.0-27.el7 boost-serialization.x86_64 0:1.53.0-27.el7
boost-signals.x86_64 0:1.53.0-27.el7 boost-test.x86_64 0:1.53.0-27.el7 boost-timer.x86_64 0:1.53.0-27.el7
boost-wave.x86_64 0:1.53.0-27.el7 cairo.x86_64 0:1.14.8-2.el7 compat-libf2c-34.x86_64 0:3.4.6-32.el7
cups-libs.x86_64 1:1.6.3-29.el7 cyrus-sasl.x86_64 0:2.1.26-21.el7 fontconfig.x86_64 0:2.10.95-11.el7
fontpackages-filesystem.noarch 0:1.44-8.el7 glibc.i686 0:2.17-196.el7_4.2 graphite2.x86_64 0:1.3.10-1.el7_3
gtk-update-icon-cache.x86_64 0:3.22.10-5.el7_4 gtk2.x86_64 0:2.24.31-1.el7 harfbuzz.x86_64 0:1.3.2-1.el7
hicolor-icon-theme.noarch 0:0.12-7.el7 libICE.x86_64 0:1.0.9-9.el7 libSM.x86_64 0:1.2.2-2.el7
libXcomposite.x86_64 0:0.4.4-4.1.el7 libXcursor.x86_64 0:1.1.14-8.el7 libXdamage.x86_64 0:1.1.4-4.1.el7
libXext.x86_64 0:1.3.3-3.el7 libXfixes.x86_64 0:5.0.3-1.el7 libXft.x86_64 0:2.3.2-2.el7
libXi.x86_64 0:1.7.9-1.el7 libXinerama.x86_64 0:1.1.3-2.1.el7 libXrandr.x86_64 0:1.5.1-2.el7
libXrender.x86_64 0:0.9.10-1.el7 libXxf86vm.x86_64 0:1.1.4-1.el7 libthai.x86_64 0:0.1.14-9.el7
libxshmfence.x86_64 0:1.2-1.el7 lyx-fonts.noarch 0:2.2.3-1.el7 mailcap.noarch 0:2.1.41-2.el7
mesa-libEGL.x86_64 0:17.0.1-6.20170307.el7 mesa-libGL.x86_64 0:17.0.1-6.20170307.el7 mesa-libgbm.x86_64 0:17.0.1-6.20170307.el7
mesa-libglapi.x86_64 0:17.0.1-6.20170307.el7 nss-softokn-freebl.i686 0:3.28.3-8.el7_4 pango.x86_64 0:1.40.4-1.el7
pixman.x86_64 0:0.34.0-1.el7 tcl.x86_64 1:8.5.13-8.el7 tk.x86_64 1:8.5.13-6.el7
wxBase.x86_64 0:2.8.12-20.el7 wxGTK.x86_64 0:2.8.12-20.el7 zlib.i686 0:1.2.7-17.el7
Complete!
Edit /awips2/edex/bin/setup.env and change export AW_SITE_IDENTIFIER=AFC and export DATA_ARCHIVE_ROOT=/awips2/data_store/
mkdir /awips2/data_store && chown awips:fxalpha /awips2/data_store && chmod 775 /awips2/data_store
- install localizations for AFX
sudo su - awips
cd edex_processing_scripts/bin
./edex.sh start
- NO LDM or /data_store/download dir yet
finished config check
Starting edex_postgres (via systemctl): [ OK ]
Starting httpd-pypies (via systemctl): [ OK ]
postgre started
Starting qpidd (via systemctl): [ OK ]
qpidd started
Starting EDEX Camel (ingest):
Starting EDEX Camel (ingestDat):
Starting EDEX Camel (ingestGrib):
Starting EDEX Camel (request):
EDEX Camel (ingest) is running (wrapper PID 7925)
EDEX Camel (ingest) is running (java PID 8139)
EDEX Camel (ingestDat) is running (wrapper PID 7924)
EDEX Camel (ingestDat) is running (java PID 8345)
EDEX Camel (ingestGrib) is running (wrapper PID 7927)
EDEX Camel (ingestGrib) is running (java PID 8221)
EDEX Camel (request) is running (wrapper PID 8098)
EDEX Camel (request) is running (java PID 8180)
edex_camel started
done?
[awips@edex-test bin]$ ./edex.sh status
[edex status]
postgres :: running :: pid 7313
pypies :: running :: pid 7394
qpid :: running :: pid 7439
EDEXingest :: running :: pid 7268 7877 8139
EDEXgrib :: running :: pid 7278 7876 8221
EDEXrequest :: running :: pid 7286 8082 8180
done?
Checking logs for errors:
grep ERROR /awips2/edex/logs/edex-ingest-20180330.log
ERROR 2018-03-30 20:19:04,233 4521 [pool-3-thread-1] SiteAwareRegistry: EDEX - No site config directory found for AFC
ERROR 2018-03-30 20:21:14,976 4536 [EDEXMain] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/geo_data/util/run_create_bas_bound": error=2, No such file or directory
ERROR 2018-03-30 20:21:14,977 4537 [EDEXMain] SetupSrv: EDEX - OHD Setup did not successfully complete: null
ERROR 2018-03-30 20:21:17,091 4549 [EDEXMain] MetarToShefFilter: Failed to process filter configuration.
ERROR 2018-03-30 20:21:17,121 4551 [EDEXMain] SynopticToShefFilter: Failed to process filter configuration.
ERROR 2018-03-30 20:21:36,532 4641 [EDEXMain] Ingest: Plugin geomag does not have a valid distribution patterns file in localization.
ERROR 2018-03-30 20:21:37,526 4678 [EDEXMain] Ingest: Plugin ghcd does not have a valid distribution patterns file in localization.
ERROR 2018-03-30 20:24:32,183 6869 [gaffScheduledWork-1] GAFF: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: org.springframework.transaction.CannotCreateTransactionException, CaughtExceptionMessage: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection, StackTrace: org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection
ERROR 2018-03-30 20:25:00,076 6891 [mpeFieldGenScheduledWork-1] MainMethod: EDEX - Cannot run program "ksh": error=2, No such file or directory
ERROR 2018-03-30 20:25:00,078 6892 [mpeFieldGenScheduledWork-1] mpeFieldGen: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 20:26:00,476 6906 [DefaultQuartzScheduler_Worker-9] RPGEnvironmentalDataManager: EDEX - Error connecting to RadarServer
ERROR 2018-03-30 20:27:00,126 6925 [alarmWhfsScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_alarm_whfs": error=2, No such file or directory
ERROR 2018-03-30 20:27:00,128 6926 [alarmWhfsScheduledWork-1] alarmWhfs: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 20:27:09,401 6932 [alarmWhfsScheduledWork-1] RunReportAlarmSrv: EDEX - Could not generate Alertalarm report.
ERROR 2018-03-30 20:30:00,102 6978 [mpeLightSrvScheduledWork-1] Logger: Failed delivery for (MessageId: ID:1a9356ad-af54-37cb-aab1-259d4d193056 on ExchangeId: ID-edex-test-gina-alaska-edu-41951-1522441126373-36-3). Exhausted after delivery attempt: 1 caught: com.raytheon.uf.edex.core.EdexException: Couldn't get BinLightning records from metadata database. Failed SQL: select datauri from binlightning where reftime > (now()- interval '30 minutes ')and source = :source
Caused by: org.postgresql.util.PSQLException: ERROR: relation "binlightning" does not exist
Caused by: org.postgresql.util.PSQLException: ERROR: relation "binlightning" does not exist
ERROR 2018-03-30 20:32:00,073 7009 [pointDataScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_pdc_pp": error=2, No such file or directory
ERROR 2018-03-30 20:32:00,074 7010 [pointDataScheduledWork-1] pointData: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 20:36:31,004 7066 [gaffScheduledWork-1] GAFF: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: org.springframework.transaction.CannotCreateTransactionException, CaughtExceptionMessage: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection, StackTrace: org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection
ERROR 2018-03-30 20:37:00,114 7086 [alarmWhfsScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_alarm_whfs": error=2, No such file or directory
ERROR 2018-03-30 20:37:00,116 7087 [alarmWhfsScheduledWork-1] alarmWhfs: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 20:37:09,378 7093 [alarmWhfsScheduledWork-1] RunReportAlarmSrv: EDEX - Could not generate Alertalarm report.
ERROR 2018-03-30 20:47:00,103 7235 [pointDataScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_pdc_pp": error=2, No such file or directory
ERROR 2018-03-30 20:47:00,103 7236 [alarmWhfsScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_alarm_whfs": error=2, No such file or directory
ERROR 2018-03-30 20:47:00,104 7237 [pointDataScheduledWork-1] pointData: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 20:47:00,104 7238 [alarmWhfsScheduledWork-1] alarmWhfs: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 20:47:09,359 7244 [alarmWhfsScheduledWork-1] RunReportAlarmSrv: EDEX - Could not generate Alertalarm report.
ERROR 2018-03-30 20:48:24,319 7262 [gaffScheduledWork-1] GAFF: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: org.springframework.transaction.CannotCreateTransactionException, CaughtExceptionMessage: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection, StackTrace: org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection
ERROR 2018-03-30 20:52:17,616 7325 [Camel (clusteredDatabaseMonitor) thread #35 - timer://databaseMonitor] DatabaseMonitorRegistry: Error occurred running database monitor: damBloatMonitor
ERROR 2018-03-30 20:52:41,803 7333 [Camel (clusteredDatabaseMonitor) thread #35 - timer://databaseMonitor] DatabaseMonitorRegistry: Error occurred running database monitor: ihfsBloatMonitor
ERROR 2018-03-30 20:57:00,014 7395 [alarmWhfsScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_alarm_whfs": error=2, No such file or directory
ERROR 2018-03-30 20:57:00,015 7396 [alarmWhfsScheduledWork-1] alarmWhfs: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 20:57:09,327 7405 [alarmWhfsScheduledWork-1] RunReportAlarmSrv: EDEX - Could not generate Alertalarm report.
ERROR 2018-03-30 21:00:00,018 7445 [DefaultQuartzScheduler_Worker-9] RPGEnvironmentalDataManager: EDEX - Error connecting to RadarServer
ERROR 2018-03-30 21:00:00,109 7454 [mpeLightSrvScheduledWork-1] Logger: Failed delivery for (MessageId: ID:8d75c72e-39a8-38a3-96f9-ce446abbe5ba on ExchangeId: ID-edex-test-gina-alaska-edu-41951-1522441126373-36-6). Exhausted after delivery attempt: 1 caught: com.raytheon.uf.edex.core.EdexException: Couldn't get BinLightning records from metadata database. Failed SQL: select datauri from binlightning where reftime > (now()- interval '30 minutes ')and source = :source
Caused by: org.postgresql.util.PSQLException: ERROR: relation "binlightning" does not exist
Caused by: org.postgresql.util.PSQLException: ERROR: relation "binlightning" does not exist
ERROR 2018-03-30 21:00:30,995 7458 [gaffScheduledWork-1] GAFF: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: org.springframework.transaction.CannotCreateTransactionException, CaughtExceptionMessage: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection, StackTrace: org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection
ERROR 2018-03-30 21:01:00,063 7471 [logFilePurgerScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/purge_files": error=2, No such file or directory
ERROR 2018-03-30 21:01:00,063 7472 [logFilePurgerScheduledWork-1] logFilePurger: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 21:02:00,066 7494 [pointDataScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_pdc_pp": error=2, No such file or directory
ERROR 2018-03-30 21:02:00,068 7495 [pointDataScheduledWork-1] pointData: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 21:07:00,070 7567 [alarmWhfsScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_alarm_whfs": error=2, No such file or directory
ERROR 2018-03-30 21:07:00,071 7568 [alarmWhfsScheduledWork-1] alarmWhfs: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 21:07:09,374 7572 [alarmWhfsScheduledWork-1] RunReportAlarmSrv: EDEX - Could not generate Alertalarm report.
ERROR 2018-03-30 21:12:09,318 7604 [gaffScheduledWork-1] GAFF: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: org.springframework.transaction.CannotCreateTransactionException, CaughtExceptionMessage: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection, StackTrace: org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection
ERROR 2018-03-30 21:17:00,118 7645 [pointDataScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_pdc_pp": error=2, No such file or directory
ERROR 2018-03-30 21:17:00,119 7646 [pointDataScheduledWork-1] pointData: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 21:17:00,124 7647 [alarmWhfsScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_alarm_whfs": error=2, No such file or directory
ERROR 2018-03-30 21:17:00,125 7648 [alarmWhfsScheduledWork-1] alarmWhfs: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 21:17:09,337 7652 [alarmWhfsScheduledWork-1] RunReportAlarmSrv: EDEX - Could not generate Alertalarm report.
ERROR 2018-03-30 21:24:30,947 7772 [gaffScheduledWork-1] GAFF: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: org.springframework.transaction.CannotCreateTransactionException, CaughtExceptionMessage: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection, StackTrace: org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection
ERROR 2018-03-30 21:25:00,060 7796 [mpeFieldGenScheduledWork-1] MainMethod: EDEX - Cannot run program "ksh": error=2, No such file or directory
ERROR 2018-03-30 21:25:00,064 7798 [mpeFieldGenScheduledWork-1] mpeFieldGen: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 21:26:00,029 7811 [DefaultQuartzScheduler_Worker-7] RPGEnvironmentalDataManager: EDEX - Error connecting to RadarServer
ERROR 2018-03-30 21:27:00,063 7834 [alarmWhfsScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_alarm_whfs": error=2, No such file or directory
ERROR 2018-03-30 21:27:00,064 7835 [alarmWhfsScheduledWork-1] alarmWhfs: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 21:27:09,326 7841 [alarmWhfsScheduledWork-1] RunReportAlarmSrv: EDEX - Could not generate Alertalarm report.
ERROR 2018-03-30 21:30:00,062 7889 [mpeLightSrvScheduledWork-1] Logger: Failed delivery for (MessageId: ID:ff25a464-62a6-3c8a-a6fd-bda1535e0a10 on ExchangeId: ID-edex-test-gina-alaska-edu-41951-1522441126373-36-9). Exhausted after delivery attempt: 1 caught: com.raytheon.uf.edex.core.EdexException: Couldn't get BinLightning records from metadata database. Failed SQL: select datauri from binlightning where reftime > (now()- interval '30 minutes ')and source = :source
Caused by: org.postgresql.util.PSQLException: ERROR: relation "binlightning" does not exist
Caused by: org.postgresql.util.PSQLException: ERROR: relation "binlightning" does not exist
ERROR 2018-03-30 21:32:00,063 7921 [pointDataScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_pdc_pp": error=2, No such file or directory
ERROR 2018-03-30 21:32:00,064 7922 [pointDataScheduledWork-1] pointData: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
With the fact that a new (with localizations) awips (without data) is erroring. Lets try not using AFC.
sudo su - awips
cd ~/edex_processing_scripts/bin
./edex.sh stop
- change from AFC to OAX for
AW_SITE_IDENTIFIER=OAXin setup.env - start edex again
- still seeing the error:
EDEX - No site config directory found for OAX
Full grep'ed errors from log:
[awips@edex-test logs]$ tail -f edex-ingest-20180330.log | grep ERROR
ERROR 2018-03-30 21:46:32,102 4581 [pool-3-thread-1] SiteAwareRegistry: EDEX - No site config directory found for OAX
ERROR 2018-03-30 21:48:42,063 4592 [EDEXMain] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/geo_data/util/run_create_bas_bound": error=2, No such file or directory
ERROR 2018-03-30 21:48:42,064 4593 [EDEXMain] SetupSrv: EDEX - OHD Setup did not successfully complete: null
ERROR 2018-03-30 21:48:44,043 4605 [EDEXMain] ObsToSHEFOptions: File /awips2/edex/data/utility/common_static/site/OAX/shef does not exist
ERROR 2018-03-30 21:48:44,066 4606 [EDEXMain] MetarToShefFilter: Failed to process filter configuration.
ERROR 2018-03-30 21:48:44,079 4608 [EDEXMain] ObsToSHEFOptions: File /awips2/edex/data/utility/common_static/site/OAX/shef does not exist
ERROR 2018-03-30 21:48:44,089 4609 [EDEXMain] SynopticToShefFilter: Failed to process filter configuration.
ERROR 2018-03-30 21:48:47,872 4675 [EDEXMain] Ingest: Plugin geomag does not have a valid distribution patterns file in localization.
ERROR 2018-03-30 21:48:49,144 4711 [EDEXMain] Ingest: Plugin ghcd does not have a valid distribution patterns file in localization.
[awips@edex-test logs]$ date
Fri Mar 30 21:53:31 UTC 2018
Next phase ... not sure ... switch it back from OAX to AFC and try ingesting data perhaps..
- Install & enabled LDM
- Started edex again and noticed this message in logs:
YAJSW: yajsw-stable-11.11
OS : Linux/3.10.0-693.21.1.el7.x86_64/amd64
JVM : Oracle Corporation/1.8.0_101//awips2/java/jre/64
JVM Parameter: 'wrapper.java.additional.security.1' COULD NOT BE RESOLVED!!!
working dir /awips2/edex/bin
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
[INFO] StandardFileSystemManager - Using "/awips2/tmp/vfs_cache" as temporary files store.
- LDM downloaded data & the processing scripts moved it over into the EDEX manual dir which picked it up and pulled into EDEX.
- a client was able to connect
- but no localizations for AFC were found
Still seeing errors in logs
ERROR 2018-03-30 22:54:00,020 5271 [DefaultQuartzScheduler_Worker-6] DistributionPatterns: EDEX - No valid distribution patterns for ghcd
ERROR 2018-03-30 22:54:00,020 5272 [DefaultQuartzScheduler_Worker-6] DistributionPatterns: EDEX - No valid distribution patterns for geomag
ERROR 2018-03-30 22:56:00,016 5325 [DefaultQuartzScheduler_Worker-2] DistributionPatterns: EDEX - No valid distribution patterns for ghcd
ERROR 2018-03-30 22:56:00,016 5326 [DefaultQuartzScheduler_Worker-2] DistributionPatterns: EDEX - No valid distribution patterns for geomag
ERROR 2018-03-30 22:57:00,014 5342 [alarmWhfsScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_alarm_whfs": error=2, No such file or directory
ERROR 2018-03-30 22:57:00,016 5343 [alarmWhfsScheduledWork-1] alarmWhfs: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 22:57:30,864 5352 [alarmWhfsScheduledWork-1] RunReportAlarmSrv: EDEX - Could not generate Alertalarm report.
ERROR 2018-03-30 23:00:00,044 5400 [DefaultQuartzScheduler_Worker-5] RPGEnvironmentalDataManager: EDEX - Error connecting to RadarServer
ERROR 2018-03-30 23:00:00,067 5401 [DefaultQuartzScheduler_Worker-2] DistributionPatterns: EDEX - No valid distribution patterns for ghcd
ERROR 2018-03-30 23:00:00,067 5402 [DefaultQuartzScheduler_Worker-2] DistributionPatterns: EDEX - No valid distribution patterns for geomag
ERROR 2018-03-30 23:00:00,121 5411 [mpeLightSrvScheduledWork-1] Logger: Failed delivery for (MessageId: ID:d4aad317-78d7-3ebb-b41c-48d40bbbe5e3 on ExchangeId: ID-edex-test-gina-alaska-edu-34201-1522447543262-36-6). Exhausted after delivery attempt: 1 caught: com.raytheon.uf.edex.core.EdexException: Couldn't get BinLightning records from metadata database. Failed SQL: select datauri from binlightning where reftime > (now()- interval '30 minutes ')and source = :source
Caused by: org.postgresql.util.PSQLException: ERROR: relation "binlightning" does not exist
Caused by: org.postgresql.util.PSQLException: ERROR: relation "binlightning" does not exist
ERROR 2018-03-30 23:00:20,168 5417 [gaffScheduledWork-1] GAFF: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: org.springframework.transaction.CannotCreateTransactionException, CaughtExceptionMessage: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection, StackTrace: org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBCException: Could not open connection
ERROR 2018-03-30 23:02:00,021 5436 [pointDataScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_pdc_pp": error=2, No such file or directory
ERROR 2018-03-30 23:02:00,023 5437 [pointDataScheduledWork-1] pointData: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 23:02:00,024 5438 [DefaultQuartzScheduler_Worker-2] DistributionPatterns: EDEX - No valid distribution patterns for ghcd
ERROR 2018-03-30 23:02:00,024 5439 [DefaultQuartzScheduler_Worker-2] DistributionPatterns: EDEX - No valid distribution patterns for geomag
ERROR 2018-03-30 23:04:00,016 5472 [DefaultQuartzScheduler_Worker-10] DistributionPatterns: EDEX - No valid distribution patterns for ghcd
ERROR 2018-03-30 23:04:00,016 5473 [DefaultQuartzScheduler_Worker-10] DistributionPatterns: EDEX - No valid distribution patterns for geomag
ERROR 2018-03-30 23:06:00,015 5487 [DefaultQuartzScheduler_Worker-3] DistributionPatterns: EDEX - No valid distribution patterns for ghcd
ERROR 2018-03-30 23:06:00,015 5488 [DefaultQuartzScheduler_Worker-3] DistributionPatterns: EDEX - No valid distribution patterns for geomag
ERROR 2018-03-30 23:07:00,078 5502 [alarmWhfsScheduledWork-1] MainMethod: EDEX - Cannot run program "/awips2/edex/data/share/hydroapps/whfs/bin/run_alarm_whfs": error=2, No such file or directory
ERROR 2018-03-30 23:07:00,079 5503 [alarmWhfsScheduledWork-1] alarmWhfs: Exchange[ExchangePattern: InOnly, BodyType: String, CaughtExceptionType: com.raytheon.uf.edex.ohd.MainMethod.ProcessNotStartedException, CaughtExceptionMessage: null, StackTrace: com.raytheon.uf.edex.ohd.MainMethod$ProcessNotStartedException
ERROR 2018-03-30 23:07:30,851 5521 [alarmWhfsScheduledWork-1] RunReportAlarmSrv: EDEX - Could not generate Alertalarm report.
- Is the core issue the AFC localization missing?
- Will ingest keep working or will it die after an hour like we saw before
- Get a review by somebody who has done AWIPS 17.2.1 successfully -- review: install steps and how we had to get netCDF and wxGTK dependancies fixed ... scan errors and see which ones are a concern and which can be ignored.