Skip to content

Instantly share code, notes, and snippets.

@kellymenzel

kellymenzel/portworx.log

Created July 1, 2020 17:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kellymenzel/b59ad7addbc615878ab0a91327c847db to your computer and use it in GitHub Desktop.
Save kellymenzel/b59ad7addbc615878ab0a91327c847db to your computer and use it in GitHub Desktop.
Download ZIP
Troubleshooting Portworx "Failed to load PX filesystem dependencies"
Raw
portworx.log
time="2020-07-01T17:04:38Z" level=info msg="Input arguments: /px-oci-mon -c px-cluster-662f3373-cc4a-456d-93e8-c5c892b9beb3 -x kubernetes -b -s /dev/sdb -j auto -kvdb_dev /dev/sdc -secret_type k8s -r 17001"
time="2020-07-01T17:04:38Z" level=info msg="Updated arguments: /px-oci-mon -c px-cluster-662f3373-cc4a-456d-93e8-c5c892b9beb3 -x kubernetes -b -s /dev/sdb -j auto -kvdb_dev /dev/sdc -secret_type k8s -r 17001"
time="2020-07-01T17:04:38Z" level=info msg="OCI-Monitor computed version v2.5.1-g1e7eecc3-dirty"
time="2020-07-01T17:04:38Z" level=info msg="Service handler initialized via as DBus{type:dbus,svc:portworx.service,id:0xc0005f50c0}"
time="2020-07-01T17:04:38Z" level=info msg="Activating REST server"
time="2020-07-01T17:04:38Z" level=info msg="Locating my container handler"
time="2020-07-01T17:04:38Z" level=info msg="> Attempt to use Docker as container handler failed" error="/var/run/docker.sock not a socket-file"
time="2020-07-01T17:04:38Z" level=info msg="> Attempt to use ContainerD as container handler failed" error="stat /run/containerd/containerd.sock: no such file or directory"
time="2020-07-01T17:04:38Z" level=info msg="> Using k8s-CRI as container handler"
time="2020-07-01T17:04:38Z" level=info msg="Detected HostNetwork setting - will track portworx status via REST"
time="2020-07-01T17:04:38Z" level=info msg="Parsed Registry/Repo docker.io/portworx from own image URN docker.io/portworx/oci-monitor:2.5.1"
time="2020-07-01T17:04:38Z" level=info msg="Preparing to download Portworx image..."
time="2020-07-01T17:04:38Z" level=info msg="REST: Changing install-state: ST_UNKNOWN -> ST_INSTALL"
time="2020-07-01T17:04:38Z" level=info msg="Detected imagePullSecrets portworx-dockercfg-7qcx7"
time="2020-07-01T17:04:38Z" level=info msg="Attempting to retrieve latest docker.io/portworx/px-enterprise:2.5.1 image (pullPolicy Always)"
time="2020-07-01T17:04:38Z" level=info msg="Retrieved kube-system/portworx-dockercfg-7qcx7 secret"
time="2020-07-01T17:04:38Z" level=warning msg="Could not retrieve Docker-registry credentials -- will still attempt to pull" error="Error parsing Docker/dockerconfigjson secret: Could not unmarshal input: %!s(<nil>)"
time="2020-07-01T17:04:38Z" level=info msg="Got error while checking remote image digest for docker.io/portworx/px-enterprise:2.5.1 (will pull)" error="Not implemented"
time="2020-07-01T17:04:38Z" level=info msg="Pulling image docker.io/portworx/px-enterprise:2.5.1"
time="2020-07-01T17:04:39Z" level=info msg="Got requested Portworx image docker.io/portworx/px-enterprise:2.5.1 with digest sha256:209d64ac399122cfc96f6b0c4c865b1b6da118bcd520ed9f8e3be801ccd1fd8e"
time="2020-07-01T17:04:39Z" level=info msg="Installed image digest sha256:209d64ac399122cfc96f6b0c4c865b1b6da118bcd520ed9f8e3be801ccd1fd8e same as remote/pulled image's (no need to reinstall)"
time="2020-07-01T17:04:39Z" level=info msg="Installing Portworx OCI service to /opt/pwx..."
time="2020-07-01T17:04:39Z" level=info msg="Prepending computed kubelet directory mount /var/lib/kubelet:/var/lib/kubelet:shared"
time="2020-07-01T17:04:39Z" level=info msg="Adding computed k8s-secrets-vol as volatile mount regex"
time="2020-07-01T17:04:39Z" level=info msg="RSYNC: Found volatile mount '/var/lib/kubelet/pods/5f63d383-e140-42d9-ab7b-8c815f2b5063/etc-hosts:/etc/hosts'"
time="2020-07-01T17:04:39Z" level=info msg="> run-local: /usr/bin/rsync -aL --delete /etc/hosts /opt/pwx/oci/mounts/etc/hosts"
time="2020-07-01T17:04:39Z" level=info msg="> Changed mount /var/lib/kubelet/pods/5f63d383-e140-42d9-ab7b-8c815f2b5063/etc-hosts:/etc/hosts to /opt/pwx/oci/mounts/etc/hosts:/etc/hosts"
time="2020-07-01T17:04:39Z" level=info msg="RSYNC: Found volatile mount '/var/lib/kubelet/pods/5f63d383-e140-42d9-ab7b-8c815f2b5063/containers/portworx/8de061a5:/tmp/px-termination-log'"
time="2020-07-01T17:04:39Z" level=info msg="> run-local: /usr/bin/rsync -aL --delete /tmp/px-termination-log /opt/pwx/oci/mounts/tmp/px-termination-log"
time="2020-07-01T17:04:39Z" level=info msg="> Changed mount /var/lib/kubelet/pods/5f63d383-e140-42d9-ab7b-8c815f2b5063/containers/portworx/8de061a5:/tmp/px-termination-log to /opt/pwx/oci/mounts/tmp/px-termination-log:/tmp/px-termination-log"
time="2020-07-01T17:04:39Z" level=info msg="RSYNC: Found volatile mount '/var/lib/kubelet/pods/5f63d383-e140-42d9-ab7b-8c815f2b5063/volumes/kubernetes.io~secret/portworx-token-hhqc9:/var/run/secrets/kubernetes.io/serviceaccount:ro'"
time="2020-07-01T17:04:39Z" level=info msg="> run-local: /usr/bin/rsync -aL --delete /var/run/secrets/kubernetes.io/serviceaccount/ /opt/pwx/oci/mounts/var/run/secrets/kubernetes.io/serviceaccount/"
time="2020-07-01T17:04:39Z" level=info msg="> Changed mount /var/lib/kubelet/pods/5f63d383-e140-42d9-ab7b-8c815f2b5063/volumes/kubernetes.io~secret/portworx-token-hhqc9:/var/run/secrets/kubernetes.io/serviceaccount:ro to /opt/pwx/oci/mounts/var/run/secrets/kubernetes.io/serviceaccount:/var/run/secrets/kubernetes.io/serviceaccount"
time="2020-07-01T17:04:39Z" level=info msg="> run-host: /opt/pwx/bin/px-runc install -c px-cluster-662f3373-cc4a-456d-93e8-c5c892b9beb3 -x kubernetes -b -s /dev/sdb -j auto -kvdb_dev /dev/sdc -secret_type k8s -r 17001 -v /var/lib/kubelet:/var/lib/kubelet:shared -v /opt/pwx/oci/mounts/etc/hosts:/etc/hosts -v /opt/pwx/oci/mounts/tmp/px-termination-log:/tmp/px-termination-log -v /var/cores:/var/cores -v /var/run/dbus:/var/run/dbus -v /opt/pwx/oci/mounts/var/run/secrets/kubernetes.io/serviceaccount:/var/run/secrets/kubernetes.io/serviceaccount -e AUTO_NODE_RECOVERY_TIMEOUT_IN_SECS=1500 -e CSI_ENDPOINT=unix:///var/lib/kubelet/plugins/pxd.portworx.com/csi.sock -e HOME=/root -e HOSTNAME=lab-worker-05 -e KUBERNETES_PORT=tcp://172.30.0.1:443 -e KUBERNETES_PORT_443_TCP=tcp://172.30.0.1:443 -e KUBERNETES_PORT_443_TCP_ADDR=172.30.0.1 -e KUBERNETES_PORT_443_TCP_PORT=443 -e KUBERNETES_PORT_443_TCP_PROTO=tcp -e KUBERNETES_SERVICE_HOST=172.30.0.1 -e KUBERNETES_SERVICE_PORT=443 -e KUBERNETES_SERVICE_PORT_HTTPS=443 -e NSS_SDB_USE_CACHE=no -e PORTWORX_API_PORT=tcp://172.30.152.24:9001 -e PORTWORX_API_PORT_9001_TCP=tcp://172.30.152.24:9001 -e PORTWORX_API_PORT_9001_TCP_ADDR=172.30.152.24 -e PORTWORX_API_PORT_9001_TCP_PORT=9001 -e PORTWORX_API_PORT_9001_TCP_PROTO=tcp -e PORTWORX_API_PORT_9020_TCP=tcp://172.30.152.24:9020 -e PORTWORX_API_PORT_9020_TCP_ADDR=172.30.152.24 -e PORTWORX_API_PORT_9020_TCP_PORT=9020 -e PORTWORX_API_PORT_9020_TCP_PROTO=tcp -e PORTWORX_API_PORT_9021_TCP=tcp://172.30.152.24:9021 -e PORTWORX_API_PORT_9021_TCP_ADDR=172.30.152.24 -e PORTWORX_API_PORT_9021_TCP_PORT=9021 -e PORTWORX_API_PORT_9021_TCP_PROTO=tcp -e PORTWORX_API_SERVICE_HOST=172.30.152.24 -e PORTWORX_API_SERVICE_PORT=9001 -e PORTWORX_API_SERVICE_PORT_PX_API=9001 -e PORTWORX_API_SERVICE_PORT_PX_REST_GATEWAY=9021 -e PORTWORX_API_SERVICE_PORT_PX_SDK=9020 -e PORTWORX_OPERATOR_METRICS_PORT=tcp://172.30.86.61:8999 -e PORTWORX_OPERATOR_METRICS_PORT_8999_TCP=tcp://172.30.86.61:8999 -e PORTWORX_OPERATOR_METRICS_PORT_8999_TCP_ADDR=172.30.86.61 -e PORTWORX_OPERATOR_METRICS_PORT_8999_TCP_PORT=8999 -e PORTWORX_OPERATOR_METRICS_PORT_8999_TCP_PROTO=tcp -e PORTWORX_OPERATOR_METRICS_SERVICE_HOST=172.30.86.61 -e PORTWORX_OPERATOR_METRICS_SERVICE_PORT=8999 -e PORTWORX_OPERATOR_METRICS_SERVICE_PORT_METRICS=8999 -e PORTWORX_SERVICE_PORT=tcp://172.30.14.140:9001 -e PORTWORX_SERVICE_PORT_9001_TCP=tcp://172.30.14.140:9001 -e PORTWORX_SERVICE_PORT_9001_TCP_ADDR=172.30.14.140 -e PORTWORX_SERVICE_PORT_9001_TCP_PORT=9001 -e PORTWORX_SERVICE_PORT_9001_TCP_PROTO=tcp -e PORTWORX_SERVICE_PORT_9019_TCP=tcp://172.30.14.140:9019 -e PORTWORX_SERVICE_PORT_9019_TCP_ADDR=172.30.14.140 -e PORTWORX_SERVICE_PORT_9019_TCP_PORT=9019 -e PORTWORX_SERVICE_PORT_9019_TCP_PROTO=tcp -e PORTWORX_SERVICE_PORT_9020_TCP=tcp://172.30.14.140:9020 -e PORTWORX_SERVICE_PORT_9020_TCP_ADDR=172.30.14.140 -e PORTWORX_SERVICE_PORT_9020_TCP_PORT=9020 -e PORTWORX_SERVICE_PORT_9020_TCP_PROTO=tcp -e PORTWORX_SERVICE_PORT_9021_TCP=tcp://172.30.14.140:9021 -e PORTWORX_SERVICE_PORT_9021_TCP_ADDR=172.30.14.140 -e PORTWORX_SERVICE_PORT_9021_TCP_PORT=9021 -e PORTWORX_SERVICE_PORT_9021_TCP_PROTO=tcp -e PORTWORX_SERVICE_SERVICE_HOST=172.30.14.140 -e PORTWORX_SERVICE_SERVICE_PORT=9001 -e PORTWORX_SERVICE_SERVICE_PORT_PX_API=9001 -e PORTWORX_SERVICE_SERVICE_PORT_PX_KVDB=9019 -e PORTWORX_SERVICE_SERVICE_PORT_PX_REST_GATEWAY=9021 -e PORTWORX_SERVICE_SERVICE_PORT_PX_SDK=9020 -e PX_NAMESPACE=kube-system -e PX_SECRETS_NAMESPACE=kube-system -e PX_TEMPLATE_VERSION=v4 -e STORK_SERVICE_PORT=tcp://172.30.222.227:8099 -e STORK_SERVICE_PORT_8099_TCP=tcp://172.30.222.227:8099 -e STORK_SERVICE_PORT_8099_TCP_ADDR=172.30.222.227 -e STORK_SERVICE_PORT_8099_TCP_PORT=8099 -e STORK_SERVICE_PORT_8099_TCP_PROTO=tcp -e STORK_SERVICE_SERVICE_HOST=172.30.222.227 -e STORK_SERVICE_SERVICE_PORT=8099 -e TERM=xterm -e container=oci -e PX_IMAGE=docker.io/portworx/px-enterprise:2.5.1 -e CONTAINER_RUNTIME=cri/cri-o -e PX_IMAGE_DIGEST=sha256:209d64ac399122cfc96f6b0c4c865b1b6da118bcd520ed9f8e3be801ccd1fd8e -e KUBELET_DIR=/var/lib/kubelet"
time="2020-07-01T17:04:39Z" level=info msg="Rootfs found at /var/opt/pwx/oci/rootfs"
time="2020-07-01T17:04:39Z" level=info msg="PX binaries found at /var/opt/pwx/bin/px-runc"
time="2020-07-01T17:04:39Z" level=info msg="Initializing as version 2.5.1.0-3e947c0 (OCI)"
time="2020-07-01T17:04:39Z" level=info msg="Enabling Sharedv4 NFS support ..."
time="2020-07-01T17:04:39Z" level=info msg="Setting up NFS service"
time="2020-07-01T17:04:39Z" level=info msg="> Initialized service controls via DBus{type:dbus,svc:nfs-server.service,id:0xc4201e7180}"
time="2020-07-01T17:04:39Z" level=info msg="> NFS nfs-server.service@RedHat already running"
time="2020-07-01T17:04:39Z" level=info msg="Fixing docker.sock mount:"
time="2020-07-01T17:04:39Z" level=info msg="> Removing mount for /var/run/docker.sock:/var/run/docker.sock:[rbind rprivate]"
time="2020-07-01T17:04:39Z" level=info msg="> Adding mount for /run:/var/host_run:[bind rprivate]"
time="2020-07-01T17:04:39Z" level=info msg="> Soft-link /var/opt/pwx/oci/rootfs/run/docker.sock -> /var/host_run/docker.sock already exists"
time="2020-07-01T17:04:39Z" level=info msg=">>>> Environment LVM_USE_HOST="
time="2020-07-01T17:04:39Z" level=info msg=">>>> px-runc importing /run/lvm directory to container"
time="2020-07-01T17:04:39Z" level=info msg=">>>> px-runc importing /run/udev directory to container"
time="2020-07-01T17:04:39Z" level=info msg="Checking mountpoints for following shared directories: [/var/lib/kubelet /var/lib/osd]"
time="2020-07-01T17:04:39Z" level=info msg="Found following mountpoints for shared dirs: map[/:{isMP=T,Opts=shared:1} /var/lib/osd:{isMP=f,Opts=shared:64,Parent=/var} /var/lib/kubelet:{isMP=f,Opts=shared:64,Parent=/var} /var:{isMP=T,Opts=shared:64,Parent=/}]"
time="2020-07-01T17:04:39Z" level=info msg="SPEC UNCHANGED [d525e84bce421fdc9ea271311a5f5a70 /var/opt/pwx/oci/config.json]"
time="2020-07-01T17:04:39Z" level=info msg="PX-RunC arguments: -b -c px-cluster-662f3373-cc4a-456d-93e8-c5c892b9beb3 -j auto -kvdb_dev /dev/sdc -r 17001 -s /dev/sdb -secret_type k8s -x kubernetes"
time="2020-07-01T17:04:39Z" level=info msg="PX-RunC mounts: /dev:/dev /etc/exports:/etc/exports /opt/pwx/oci/mounts/etc/hosts:/etc/hosts /etc/pwx:/etc/pwx /etc/resolv.conf:/etc/resolv.conf:ro /var/opt/pwx/bin:/export_bin /lib/modules:/lib/modules proc:/proc:nosuid,noexec,nodev /run/docker:/run/docker sysfs:/sys:nosuid,noexec,nodev /sys/firmware/efi/efivars:/sys/firmware/efi/efivars:ro cgroup:/sys/fs/cgroup:nosuid,noexec,nodev /opt/pwx/oci/mounts/tmp/px-termination-log:/tmp/px-termination-log /usr/src:/usr/src /var/cores:/var/cores /run:/var/host_run:bind /var/lib/kubelet:/var/lib/kubelet:shared /var/lib/nfs:/var/lib/nfs /var/lib/osd:/var/lib/osd:shared /var/log/journal:/var/log/journal:ro /var/run/dbus:/var/run/dbus /opt/pwx/oci/mounts/var/run/secrets/kubernetes.io/serviceaccount:/var/run/secrets/kubernetes.io/serviceaccount /run/lvm:/run/lvm: /run/udev:/run/udev:"
time="2020-07-01T17:04:39Z" level=info msg="PX-RunC env: AUTO_NODE_RECOVERY_TIMEOUT_IN_SECS=1500 CONTAINER_RUNTIME=cri/cri-o CSI_ENDPOINT=unix:///var/lib/kubelet/plugins/pxd.portworx.com/csi.sock GOMAXPROCS=64 GOTRACEBACK=crash HOME=/root HOSTNAME=lab-worker-05 KUBELET_DIR=/var/lib/kubelet KUBERNETES_PORT=tcp://172.30.0.1:443 KUBERNETES_PORT_443_TCP=tcp://172.30.0.1:443 KUBERNETES_PORT_443_TCP_ADDR=172.30.0.1 KUBERNETES_PORT_443_TCP_PORT=443 KUBERNETES_PORT_443_TCP_PROTO=tcp KUBERNETES_SERVICE_HOST=172.30.0.1 KUBERNETES_SERVICE_PORT=443 KUBERNETES_SERVICE_PORT_HTTPS=443 LVM_USE_HOST=0 NFS_SERVICE=nfs-server.service NSS_SDB_USE_CACHE=no PATH=/export_bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PORTWORX_API_PORT=tcp://172.30.152.24:9001 PORTWORX_API_PORT_9001_TCP=tcp://172.30.152.24:9001 PORTWORX_API_PORT_9001_TCP_ADDR=172.30.152.24 PORTWORX_API_PORT_9001_TCP_PORT=9001 PORTWORX_API_PORT_9001_TCP_PROTO=tcp PORTWORX_API_PORT_9020_TCP=tcp://172.30.152.24:9020 PORTWORX_API_PORT_9020_TCP_ADDR=172.30.152.24 PORTWORX_API_PORT_9020_TCP_PORT=9020 PORTWORX_API_PORT_9020_TCP_PROTO=tcp PORTWORX_API_PORT_9021_TCP=tcp://172.30.152.24:9021 PORTWORX_API_PORT_9021_TCP_ADDR=172.30.152.24 PORTWORX_API_PORT_9021_TCP_PORT=9021 PORTWORX_API_PORT_9021_TCP_PROTO=tcp PORTWORX_API_SERVICE_HOST=172.30.152.24 PORTWORX_API_SERVICE_PORT=9001 PORTWORX_API_SERVICE_PORT_PX_API=9001 PORTWORX_API_SERVICE_PORT_PX_REST_GATEWAY=9021 PORTWORX_API_SERVICE_PORT_PX_SDK=9020 PORTWORX_OPERATOR_METRICS_PORT=tcp://172.30.86.61:8999 PORTWORX_OPERATOR_METRICS_PORT_8999_TCP=tcp://172.30.86.61:8999 PORTWORX_OPERATOR_METRICS_PORT_8999_TCP_ADDR=172.30.86.61 PORTWORX_OPERATOR_METRICS_PORT_8999_TCP_PORT=8999 PORTWORX_OPERATOR_METRICS_PORT_8999_TCP_PROTO=tcp PORTWORX_OPERATOR_METRICS_SERVICE_HOST=172.30.86.61 PORTWORX_OPERATOR_METRICS_SERVICE_PORT=8999 PORTWORX_OPERATOR_METRICS_SERVICE_PORT_METRICS=8999 PORTWORX_SERVICE_PORT=tcp://172.30.14.140:9001 PORTWORX_SERVICE_PORT_9001_TCP=tcp://172.30.14.140:9001 PORTWORX_SERVICE_PORT_9001_TCP_ADDR=172.30.14.140 PORTWORX_SERVICE_PORT_9001_TCP_PORT=9001 PORTWORX_SERVICE_PORT_9001_TCP_PROTO=tcp PORTWORX_SERVICE_PORT_9019_TCP=tcp://172.30.14.140:9019 PORTWORX_SERVICE_PORT_9019_TCP_ADDR=172.30.14.140 PORTWORX_SERVICE_PORT_9019_TCP_PORT=9019 PORTWORX_SERVICE_PORT_9019_TCP_PROTO=tcp PORTWORX_SERVICE_PORT_9020_TCP=tcp://172.30.14.140:9020 PORTWORX_SERVICE_PORT_9020_TCP_ADDR=172.30.14.140 PORTWORX_SERVICE_PORT_9020_TCP_PORT=9020 PORTWORX_SERVICE_PORT_9020_TCP_PROTO=tcp PORTWORX_SERVICE_PORT_9021_TCP=tcp://172.30.14.140:9021 PORTWORX_SERVICE_PORT_9021_TCP_ADDR=172.30.14.140 PORTWORX_SERVICE_PORT_9021_TCP_PORT=9021 PORTWORX_SERVICE_PORT_9021_TCP_PROTO=tcp PORTWORX_SERVICE_SERVICE_HOST=172.30.14.140 PORTWORX_SERVICE_SERVICE_PORT=9001 PORTWORX_SERVICE_SERVICE_PORT_PX_API=9001 PORTWORX_SERVICE_SERVICE_PORT_PX_KVDB=9019 PORTWORX_SERVICE_SERVICE_PORT_PX_REST_GATEWAY=9021 PORTWORX_SERVICE_SERVICE_PORT_PX_SDK=9020 PX_IMAGE=docker.io/portworx/px-enterprise:2.5.1 PX_IMAGE_DIGEST=sha256:209d64ac399122cfc96f6b0c4c865b1b6da118bcd520ed9f8e3be801ccd1fd8e PX_LOGLEVEL=info PX_NAMESPACE=kube-system PX_RUNC=true PX_SECRETS_NAMESPACE=kube-system PX_SHARED=/var/lib/kubelet:shared:64;/var/lib/osd:shared:64 PX_TEMPLATE_VERSION=v4 PX_VERSION=2.5.1.0-3e947c0 STORK_SERVICE_PORT=tcp://172.30.222.227:8099 STORK_SERVICE_PORT_8099_TCP=tcp://172.30.222.227:8099 STORK_SERVICE_PORT_8099_TCP_ADDR=172.30.222.227 STORK_SERVICE_PORT_8099_TCP_PORT=8099 STORK_SERVICE_PORT_8099_TCP_PROTO=tcp STORK_SERVICE_SERVICE_HOST=172.30.222.227 STORK_SERVICE_SERVICE_PORT=8099 TERM=xterm container=oci"
time="2020-07-01T17:04:39Z" level=info msg="/etc/systemd/system/portworx-reboot.service content unchanged [1dc97b965f3c6ad99aa3a92a02b2e8b1 /etc/systemd/system/portworx-reboot.service]"
time="2020-07-01T17:04:39Z" level=info msg="/etc/systemd/system/portworx.socket content unchanged [e80e04204b7a7d113db36c53f420d635 /etc/systemd/system/portworx.socket]"
time="2020-07-01T17:04:39Z" level=info msg="/etc/systemd/system/portworx-output.service content unchanged [7340d8be39a32f3b7d296ac8275bc2e1 /etc/systemd/system/portworx-output.service]"
time="2020-07-01T17:04:39Z" level=info msg="/etc/systemd/system/portworx.service content unchanged [16960212a6f2f3c8525c5c90278e5964 /etc/systemd/system/portworx.service]"
time="2020-07-01T17:04:39Z" level=error msg="Could not enable portworx-reboot" error="exit status 1" out="Failed to connect to bus: No data available
Failed to connect to bus: No data available
"
time="2020-07-01T17:04:39Z" level=error msg="Could create systemd service" error="Could not enable portworx-reboot: exit status 1"
time="2020-07-01T17:04:39Z" level=info msg="runC spec unchanged"
time="2020-07-01T17:04:39Z" level=info msg="Portworx service restart required due to missing/invalid /etc/pwx/config.json"
time="2020-07-01T17:04:39Z" level=warning msg="Reloading + Restarting portworx service"
time="2020-07-01T17:04:39Z" level=info msg="Service handler initialized via as DBus{type:dbus,svc:portworx-reboot.service,id:0xc000746000}"
time="2020-07-01T17:04:39Z" level=info msg="Service portworx.service not yet active"
time="2020-07-01T17:04:39Z" level=info msg="Activating node-watcher"
time="2020-07-01T17:04:39Z" level=info msg="Portworx service is ACTIVE"
time="2020-07-01T17:04:39Z" level=info msg="REST: Changing install-state: ST_INSTALL -> ST_FINISH"
time="2020-07-01T17:04:39Z" level=info msg="Start tailing portworx.service logs"
time="2020-07-01T17:04:41Z" level=info msg="> Starting local log-tailer"
time="2020-07-01T17:04:41Z" level=info msg="> run-local: /px-log-tail --follow -P @ -tf -u portworx.service -u portworx-output.service -u init.scope -n 20000 -p 2916116"
time="2020-07-01T17:04:41Z" level=info msg="Install done - MAIN exiting"
time="2020-07-01T17:04:41Z" level=info msg="-- Flushing logs for PID 2916116 [20000 lines] --"
time="2020-07-01T17:04:41Z" level=info msg="-- Start tailing the logs for portworx.service, portworx-output.service, init.scope --"
@lab-worker-05 portworx[2916107]: checking local archive, please wait...
@lab-worker-05 portworx[2916107]: Found: x86_64/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: insmod: ERROR: could not insert module x86_64/4.18.0-147.8.1.el8_1.x86_64/version/8/px.ko: Required key not available
@lab-worker-05 portworx[2916107]: Flushing buffer-cache..
@lab-worker-05 portworx[2916107]: insmod: ERROR: could not insert module x86_64/4.18.0-147.8.1.el8_1.x86_64/version/8/px.ko: Required key not available
@lab-worker-05 portworx[2916107]: checking /hostusr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /lib/modules/4.18.0-147.8.1.el8_1.x86_64/build
@lab-worker-05 portworx[2916107]: checking /usr/src/linux
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /lib/modules/4.18.0-147.8.1.el8_1.x86_64/build
@lab-worker-05 portworx[2916107]: checking /usr/src/linux
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: Checking short path for KVERSION 4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/kernels/4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-headers-4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/kernels/4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-headers-4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-4.18.0-147-obj/x86_64/el7.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-4.18.0-147-obj/x86_64/el7.x86_64
@lab-worker-05 portworx[2916107]: checking /lib/modules/4.18.0-147.el7.x86_64/build
@lab-worker-05 portworx[2916107]: checking /usr/src/linux
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/kernels/4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-headers-4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-4.18.0-147-obj/x86_64/el7.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /lib/modules/4.18.0-147.8.1.el8_1.x86_64/build
@lab-worker-05 portworx[2916107]: checking /usr/src/linux
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: Reading package lists...
@lab-worker-05 portworx[2916107]: Building dependency tree...
@lab-worker-05 portworx[2916107]: Reading state information...
@lab-worker-05 portworx[2916107]: E: Unable to locate package linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: E: Couldn't find any package by glob 'linux-headers-4.18.0-147.8.1.el8_1.x86_64'
@lab-worker-05 portworx[2916107]: E: Couldn't find any package by regex 'linux-headers-4.18.0-147.8.1.el8_1.x86_64'
@lab-worker-05 portworx[2916107]: checking /hostusr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /lib/modules/4.18.0-147.8.1.el8_1.x86_64/build
@lab-worker-05 portworx[2916107]: checking /usr/src/linux
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: Checking short path for KVERSION 4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/kernels/4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-headers-4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/kernels/4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-headers-4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-4.18.0-147-obj/x86_64/el7.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-4.18.0-147-obj/x86_64/el7.x86_64
@lab-worker-05 portworx[2916107]: checking /lib/modules/4.18.0-147.el7.x86_64/build
@lab-worker-05 portworx[2916107]: checking /usr/src/linux
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/kernels/4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-headers-4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-4.18.0-147-obj/x86_64/el7.x86_64
@lab-worker-05 portworx[2916107]: Reading package lists...
@lab-worker-05 portworx[2916107]: Building dependency tree...
@lab-worker-05 portworx[2916107]: Reading state information...
@lab-worker-05 portworx[2916107]: E: Unable to locate package linux-headers-4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: E: Couldn't find any package by glob 'linux-headers-4.18.0-147.el7.x86_64'
@lab-worker-05 portworx[2916107]: E: Couldn't find any package by regex 'linux-headers-4.18.0-147.el7.x86_64'
@lab-worker-05 portworx[2916107]: checking /hostusr/src/kernels/4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-headers-4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/kernels/4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-headers-4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-4.18.0-147-obj/x86_64/el7.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-4.18.0-147-obj/x86_64/el7.x86_64
@lab-worker-05 portworx[2916107]: checking /lib/modules/4.18.0-147.el7.x86_64/build
@lab-worker-05 portworx[2916107]: checking /usr/src/linux
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/kernels/4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-headers-4.18.0-147.el7.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-4.18.0-147-obj/x86_64/el7.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /lib/modules/4.18.0-147.8.1.el8_1.x86_64/build
@lab-worker-05 portworx[2916107]: checking /usr/src/linux
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /lib/modules/4.18.0-147.8.1.el8_1.x86_64/build
@lab-worker-05 portworx[2916107]: checking /usr/src/linux
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /lib/modules/4.18.0-147.8.1.el8_1.x86_64/build
@lab-worker-05 portworx[2916107]: checking /usr/src/linux
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /lib/modules/4.18.0-147.8.1.el8_1.x86_64/build
@lab-worker-05 portworx[2916107]: checking /usr/src/linux
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /hostusr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /lib/modules/4.18.0-147.8.1.el8_1.x86_64/build
@lab-worker-05 portworx[2916107]: checking /usr/src/linux
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-headers-4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: checking /var/lib/osd/pxfs/kernel_headers/usr/src/linux-4.18.0-147.8.1-obj/x86_64/el8_1.x86_64
@lab-worker-05 portworx[2916107]: Downloading from: http://mirrors.portworx.com/build-results/pxfuse/for-installer/x86_64/4.18.0-147.8.1.el8_1.x86_64/packages ...
@lab-worker-05 portworx[2916107]: Download kernel failed, trying again...
@lab-worker-05 portworx[2916107]: Download kernel failed...
@lab-worker-05 portworx[2916107]: Failed to install/locate kernel headers for 4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: Downloading from: http://mirrors.portworx.com/build-results/pxfuse/for-installer/x86_64/4.18.0-147.8.1.el8_1.x86_64/version/8...
@lab-worker-05 portworx[2916107]: Found: http://mirrors.portworx.com/build-results/pxfuse/for-installer/x86_64/4.18.0-147.8.1.el8_1.x86_64/version/8
@lab-worker-05 portworx[2916107]: insmod: ERROR: could not insert module /var/lib/osd/pxfs/dnld/PX/px.ko: Required key not available
@lab-worker-05 portworx[2916107]: Flushing buffer-cache..
@lab-worker-05 portworx[2916107]: insmod: ERROR: could not insert module /var/lib/osd/pxfs/dnld/PX/px.ko: Required key not available
@lab-worker-05 portworx[2916107]: checking local archive, please wait...
@lab-worker-05 portworx[2916107]: Found: x86_64/4.18.0-147.8.1.el8_1.x86_64
@lab-worker-05 portworx[2916107]: insmod: ERROR: could not insert module x86_64/4.18.0-147.8.1.el8_1.x86_64/version/8/px.ko: Required key not available
@lab-worker-05 portworx[2916107]: Flushing buffer-cache..
@lab-worker-05 portworx[2916107]: insmod: ERROR: could not insert module x86_64/4.18.0-147.8.1.el8_1.x86_64/version/8/px.ko: Required key not available
@lab-worker-05 portworx[2916107]: Secure boot is enabled. PX filesystem dependency operations will fail when secure boot is enabled.
@lab-worker-05 portworx[2916107]: Failed to load PX filesystem dependencies for kernel 4.18.0-147.8.1.el8_1.x86_64
time="2020-07-01T17:05:43Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
@lab-worker-05 systemd[1]: portworx.service: Main process exited, code=exited, status=10/n/a
@lab-worker-05 systemd[1]: portworx.service: Failed with result 'exit-code'.
@lab-worker-05 systemd[1]: portworx.service: Consumed 59.846s CPU time
@lab-worker-05 systemd[1]: Stopping Portworx FIFO logging reader...
@lab-worker-05 systemd[1]: Stopped Portworx FIFO logging reader.
@lab-worker-05 systemd[1]: portworx-output.service: Consumed 17ms CPU time
@lab-worker-05 systemd[1]: Closed Portworx logging FIFO.
time="2020-07-01T17:05:53Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:06:03Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:06:13Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:06:23Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:06:33Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:06:43Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:06:53Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:07:03Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:07:13Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:07:23Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:07:33Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:07:43Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:07:53Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:08:03Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:08:13Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:08:23Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:08:33Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:08:43Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:08:53Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:09:03Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:09:13Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:09:23Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:09:33Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:09:43Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:09:53Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:10:03Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:10:13Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:10:23Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:10:33Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:10:43Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:10:53Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:11:03Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:11:13Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:11:23Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:11:33Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:11:43Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:11:53Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:12:03Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:12:13Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
@lab-worker-05 systemd[1]: Starting Network Manager Script Dispatcher Service...
@lab-worker-05 systemd[1]: Started Network Manager Script Dispatcher Service.
time="2020-07-01T17:12:23Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
@lab-worker-05 systemd[1]: NetworkManager-dispatcher.service: Consumed 38ms CPU time
time="2020-07-01T17:12:33Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:12:43Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
time="2020-07-01T17:12:53Z" level=warning msg="Could not retrieve PX node status" error="Get http://127.0.0.1:17001/v1/cluster/nodehealth: dial tcp 127.0.0.1:17001: connect: connection refused"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment