Skip to content

Instantly share code, notes, and snippets.

@IvanBiv

IvanBiv/logs gitlab1-gitlab-565857c8f9-99tdg

Created June 4, 2018 17:37
Show Gist options
  • Save IvanBiv/f2027c01b1ac676a5c78591bb7689c87 to your computer and use it in GitHub Desktop.
Save IvanBiv/f2027c01b1ac676a5c78591bb7689c87 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
logs gitlab1-gitlab-565857c8f9-99tdg
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:30:51.85931 time="2018-06-04T17:30:51Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:51.85961 time="2018-06-04T17:30:51Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:51.94303 time="2018-06-04T17:30:51Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:30:52.18327 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:30:52 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/gitlab-monitor/current <==
2018-06-04_17:30:52.81866 127.0.0.1 - - [04/Jun/2018:17:30:52 UTC] "GET /process HTTP/1.1" 200 2194
2018-06-04_17:30:52.81871 - -> /process
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:30:52.86159 time="2018-06-04T17:30:52Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:52.86179 time="2018-06-04T17:30:52Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:52.94479 time="2018-06-04T17:30:52Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:30:53.18347 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:30:53 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:30:53.86336 time="2018-06-04T17:30:53Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:53.86373 time="2018-06-04T17:30:53Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:53.94669 time="2018-06-04T17:30:53Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:30:54.18362 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:30:54 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/health_check?token=[FILTERED]" for 127.0.0.1 at 2018-06-04 17:30:54 +0000
Processing by HealthCheckController#index as HTML
Parameters: {"token"=>"[FILTERED]"}
Completed 200 OK in 68ms (Views: 0.2ms | ActiveRecord: 7.2ms)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/health_check","format":"html","controller":"HealthCheckController","action":"index","status":200,"duration":69.26,"view":0.21,"db":7.23,"time":"2018-06-04T17:30:54.335Z","params":[{"key":"token","value":"[FILTERED]"}],"remote_ip":null,"user_id":null,"username":null}
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:30:54.40534 10.233.78.147:8005 192.168.88.11:35384 - - [2018/06/04:17:30:54 +0000] "GET /health_check?token=SXBAQichEJasbtDSygrD HTTP/1.1" 200 7 "" "kube-probe/1.10" 0.080
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:30:54.86515 time="2018-06-04T17:30:54Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:54.86596 time="2018-06-04T17:30:54Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/health_check?token=[FILTERED]" for 127.0.0.1 at 2018-06-04 17:30:54 +0000
Processing by HealthCheckController#index as HTML
Parameters: {"token"=>"[FILTERED]"}
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:30:54.94835 time="2018-06-04T17:30:54Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-rails/production.log <==
Completed 200 OK in 51ms (Views: 0.2ms | ActiveRecord: 4.2ms)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/health_check","format":"html","controller":"HealthCheckController","action":"index","status":200,"duration":52.0,"view":0.22,"db":4.23,"time":"2018-06-04T17:30:54.941Z","params":[{"key":"token","value":"[FILTERED]"}],"remote_ip":null,"user_id":null,"username":null}
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:30:54.99409 10.233.78.147:8005 192.168.88.11:35388 - - [2018/06/04:17:30:54 +0000] "GET /health_check?token=SXBAQichEJasbtDSygrD HTTP/1.1" 200 7 "" "kube-probe/1.10" 0.065
2018-06-04_17:30:55.18415 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:30:55 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:30:55.86690 time="2018-06-04T17:30:55Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:55.86773 time="2018-06-04T17:30:55Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:55.95681 time="2018-06-04T17:30:55Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:30:56.18429 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:30:56 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/-/metrics" for 127.0.0.1 at 2018-06-04 17:30:56 +0000
Processing by MetricsController#index as HTML
Completed 200 OK in 1ms (Views: 0.3ms | ActiveRecord: 0.0ms)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":2.92,"view":0.33,"db":0.0,"time":"2018-06-04T17:30:56.703Z","params":[],"remote_ip":null,"user_id":null,"username":null}
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:30:56.86897 time="2018-06-04T17:30:56Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:56.86952 time="2018-06-04T17:30:56Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:56.95832 time="2018-06-04T17:30:56Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:30:57.18451 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:30:57 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:30:57.87106 time="2018-06-04T17:30:57Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:57.87144 time="2018-06-04T17:30:57Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:57.96011 time="2018-06-04T17:30:57Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:30:58.18497 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:30:58 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:30:58.87315 time="2018-06-04T17:30:58Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:58.87326 time="2018-06-04T17:30:58Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:58.96197 time="2018-06-04T17:30:58Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:30:59.18489 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:30:59 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:30:59.87490 time="2018-06-04T17:30:59Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:59.87496 time="2018-06-04T17:30:59Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:30:59.96376 time="2018-06-04T17:30:59Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:00.18517 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:00 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:00.87714 time="2018-06-04T17:31:00Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:00.87727 time="2018-06-04T17:31:00Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:00.96556 time="2018-06-04T17:31:00Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:01.18524 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:01 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/" for 127.0.0.1 at 2018-06-04 17:31:01 +0000
Processing by RootController#index as HTML
Redirected to http://gitlab.my-domain.com/users/sign_in
Filter chain halted as :redirect_unlogged_user rendered or redirected
Completed 302 Found in 7ms (ActiveRecord: 0.0ms)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":302,"duration":8.53,"view":0.0,"db":0.0,"location":"http://gitlab.my-domain.com/users/sign_in","time":"2018-06-04T17:31:01.436Z","params":[],"remote_ip":"127.0.0.1","user_id":null,"username":null}
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:01.44612 gitlab.my-domain.com 10.233.78.148:46678 - - [2018/06/04:17:31:01 +0000] "GET / HTTP/1.1" 302 107 "" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.170 Safari/537.36" 0.021
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/users/sign_in" for 127.0.0.1 at 2018-06-04 17:31:01 +0000
Processing by SessionsController#new as HTML
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:01.87922 time="2018-06-04T17:31:01Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:01.87937 time="2018-06-04T17:31:01Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:01.96741 time="2018-06-04T17:31:01Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-rails/production.log <==
Redirected to http://gitlab.my-domain.com/users/password/edit?reset_password_token=A7ypwSFFAFYGhxN1GMdv
Filter chain halted as :check_initial_setup rendered or redirected
Completed 302 Found in 514ms (ActiveRecord: 185.9ms)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/users/sign_in","format":"html","controller":"SessionsController","action":"new","status":302,"duration":517.11,"view":0.0,"db":185.92,"location":"http://gitlab.my-domain.com/users/password/edit","time":"2018-06-04T17:31:01.466Z","params":[],"remote_ip":"127.0.0.1","user_id":null,"username":null}
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:01.98691 gitlab.my-domain.com 10.233.78.148:46680 - - [2018/06/04:17:31:01 +0000] "GET /users/sign_in HTTP/1.1" 302 155 "" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.170 Safari/537.36" 0.538
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/users/password/edit?reset_password_token=[FILTERED]" for 127.0.0.1 at 2018-06-04 17:31:01 +0000
Processing by PasswordsController#edit as HTML
Parameters: {"reset_password_token"=>"[FILTERED]"}
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:02.18560 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:02 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/gitlab-rails/production.log <==
Completed 200 OK in 367ms (Views: 118.0ms | ActiveRecord: 19.8ms)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/users/password/edit","format":"html","controller":"PasswordsController","action":"edit","status":200,"duration":368.89,"view":118.01,"db":19.79,"time":"2018-06-04T17:31:02.010Z","params":[{"key":"reset_password_token","value":"[FILTERED]"}],"remote_ip":"127.0.0.1","user_id":null,"username":null}
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:02.38066 gitlab.my-domain.com 10.233.78.148:46684 - - [2018/06/04:17:31:02 +0000] "GET /users/password/edit?reset_password_token=A7ypwSFFAFYGhxN1GMdv HTTP/1.1" 200 7916 "" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.170 Safari/537.36" 0.391
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:02.94360 time="2018-06-04T17:31:02Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:02.94369 time="2018-06-04T17:31:02Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:03.04325 time="2018-06-04T17:31:03Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:03.18557 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:03 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/gitlab-monitor/current <==
2018-06-04_17:31:03.75089 127.0.0.1 - - [04/Jun/2018:17:31:03 UTC] "GET /sidekiq HTTP/1.1" 200 12964
2018-06-04_17:31:03.75093 - -> /sidekiq
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:03.94558 time="2018-06-04T17:31:03Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:03.94563 time="2018-06-04T17:31:03Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:04.04511 time="2018-06-04T17:31:04Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:04.18596 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:04 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/health_check?token=[FILTERED]" for 127.0.0.1 at 2018-06-04 17:31:04 +0000
Processing by HealthCheckController#index as HTML
Parameters: {"token"=>"[FILTERED]"}
Completed 200 OK in 57ms (Views: 0.2ms | ActiveRecord: 9.1ms)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/health_check","format":"html","controller":"HealthCheckController","action":"index","status":200,"duration":58.31,"view":0.23,"db":9.08,"time":"2018-06-04T17:31:04.337Z","params":[{"key":"token","value":"[FILTERED]"}],"remote_ip":null,"user_id":null,"username":null}
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:04.39712 10.233.78.147:8005 192.168.88.11:35438 - - [2018/06/04:17:31:04 +0000] "GET /health_check?token=SXBAQichEJasbtDSygrD HTTP/1.1" 200 7 "" "kube-probe/1.10" 0.072
==> /var/log/gitlab/gitaly/current <==
2018-06-04_17:31:04.73600 time="2018-06-04T17:31:04Z" level=info msg="dialing to target with scheme: \"\"" system=system
2018-06-04_17:31:04.73614 time="2018-06-04T17:31:04Z" level=info msg="ccResolverWrapper: sending new addresses to cc: [{/tmp/gitaly-ruby300108084/socket.0 0 <nil>}]" system=system
2018-06-04_17:31:04.73617 time="2018-06-04T17:31:04Z" level=info msg="ClientConn switching balancer to \"pick_first\"" system=system
2018-06-04_17:31:04.73632 time="2018-06-04T17:31:04Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42038ab40, CONNECTING" system=system
2018-06-04_17:31:04.73652 time="2018-06-04T17:31:04Z" level=info msg="dialing to target with scheme: \"\"" system=system
2018-06-04_17:31:04.73680 time="2018-06-04T17:31:04Z" level=info msg="ccResolverWrapper: sending new addresses to cc: [{/tmp/gitaly-ruby300108084/socket.1 0 <nil>}]" system=system
2018-06-04_17:31:04.73684 time="2018-06-04T17:31:04Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42038ab40, READY" system=system
2018-06-04_17:31:04.73685 time="2018-06-04T17:31:04Z" level=info msg="ClientConn switching balancer to \"pick_first\"" system=system
2018-06-04_17:31:04.73686 time="2018-06-04T17:31:04Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4202239b0, CONNECTING" system=system
2018-06-04_17:31:04.73698 time="2018-06-04T17:31:04Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4202239b0, READY" system=system
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/health_check?token=[FILTERED]" for 127.0.0.1 at 2018-06-04 17:31:04 +0000
Processing by HealthCheckController#index as HTML
Parameters: {"token"=>"[FILTERED]"}
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:04.94743 time="2018-06-04T17:31:04Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:04.94787 time="2018-06-04T17:31:04Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-rails/production.log <==
Completed 200 OK in 48ms (Views: 0.2ms | ActiveRecord: 4.5ms)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/health_check","format":"html","controller":"HealthCheckController","action":"index","status":200,"duration":49.32,"view":0.19,"db":4.53,"time":"2018-06-04T17:31:04.939Z","params":[{"key":"token","value":"[FILTERED]"}],"remote_ip":null,"user_id":null,"username":null}
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:04.99001 10.233.78.147:8005 192.168.88.11:35442 - - [2018/06/04:17:31:04 +0000] "GET /health_check?token=SXBAQichEJasbtDSygrD HTTP/1.1" 200 7 "" "kube-probe/1.10" 0.061
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:05.04727 time="2018-06-04T17:31:05Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:05.18740 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:05 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.002
==> /var/log/gitlab/gitlab-monitor/current <==
2018-06-04_17:31:05.59110 127.0.0.1 - - [04/Jun/2018:17:31:05 UTC] "GET /database HTTP/1.1" 200 61070
2018-06-04_17:31:05.59114 - -> /database
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:05.94931 time="2018-06-04T17:31:05Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:05.94964 time="2018-06-04T17:31:05Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:06.05468 time="2018-06-04T17:31:06Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:06.18601 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:06 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:06.95130 time="2018-06-04T17:31:06Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:06.95159 time="2018-06-04T17:31:06Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:07.05629 time="2018-06-04T17:31:07Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:07.18639 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:07 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/gitlab-monitor/current <==
2018-06-04_17:31:07.81786 127.0.0.1 - - [04/Jun/2018:17:31:07 UTC] "GET /process HTTP/1.1" 200 2194
2018-06-04_17:31:07.81793 - -> /process
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:07.95325 time="2018-06-04T17:31:07Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:07.95340 time="2018-06-04T17:31:07Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:08.05796 time="2018-06-04T17:31:08Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:08.18643 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:08 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:08.95509 time="2018-06-04T17:31:08Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:08.95548 time="2018-06-04T17:31:08Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:09.05945 time="2018-06-04T17:31:09Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:09.18697 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:09 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:09.95717 time="2018-06-04T17:31:09Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:09.95721 time="2018-06-04T17:31:09Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:10.06083 time="2018-06-04T17:31:10Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:10.18696 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:10 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:10.95920 time="2018-06-04T17:31:10Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:10.95937 time="2018-06-04T17:31:10Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:11.06261 time="2018-06-04T17:31:11Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:11.18711 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:11 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/-/metrics" for 127.0.0.1 at 2018-06-04 17:31:11 +0000
Processing by MetricsController#index as HTML
Completed 200 OK in 2ms (Views: 0.5ms | ActiveRecord: 0.0ms)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":4.36,"view":0.52,"db":0.0,"time":"2018-06-04T17:31:11.704Z","params":[],"remote_ip":null,"user_id":null,"username":null}
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:11.96125 time="2018-06-04T17:31:11Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:11.96144 time="2018-06-04T17:31:11Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:12.06446 time="2018-06-04T17:31:12Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:12.18726 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:12 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/gitlab-rails/production.log <==
Started PUT "/users/password" for 127.0.0.1 at 2018-06-04 17:31:12 +0000
Processing by PasswordsController#update as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"[FILTERED]", "user"=>{"reset_password_token"=>"[FILTERED]", "password"=>"[FILTERED]", "password_confirmation"=>"[FILTERED]"}}
Can't verify CSRF token authenticity
Completed 422 Unprocessable Entity in 2ms (ActiveRecord: 0.0ms)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"PUT","path":"/users/password","format":"html","controller":"PasswordsController","action":"update","status":422,"error":"ActionController::InvalidAuthenticityToken: ActionController::InvalidAuthenticityToken","duration":3.19,"view":0.0,"db":0.0,"time":"2018-06-04T17:31:12.583Z","params":[{"key":"utf8","value":"✓"},{"key":"_method","value":"put"},{"key":"authenticity_token","value":"[FILTERED]"},{"key":"user","value":{"reset_password_token":"[FILTERED]","password":"[FILTERED]","password_confirmation":"[FILTERED]"}}],"remote_ip":"127.0.0.1","user_id":null,"username":null}
==> /var/log/gitlab/gitlab-rails/production.log <==
ActionController::InvalidAuthenticityToken (ActionController::InvalidAuthenticityToken):
lib/gitlab/middleware/multipart.rb:95:in `call'
lib/gitlab/request_profiler/middleware.rb:14:in `call'
lib/gitlab/middleware/go.rb:17:in `call'
lib/gitlab/etag_caching/middleware.rb:11:in `call'
lib/gitlab/middleware/read_only/controller.rb:28:in `call'
lib/gitlab/middleware/read_only.rb:16:in `call'
lib/gitlab/request_context.rb:18:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:27:in `call'
lib/gitlab/middleware/release_env.rb:10:in `call'
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:12.58832 gitlab.my-domain.com 10.233.78.148:46740 - - [2018/06/04:17:31:12 +0000] "POST /users/password HTTP/1.1" 422 2912 "http://gitlab.my-domain.com/users/password/edit?reset_password_token=A7ypwSFFAFYGhxN1GMdv" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.170 Safari/537.36" 0.055
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:12.96346 time="2018-06-04T17:31:12Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:12.96359 time="2018-06-04T17:31:12Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:13.06641 time="2018-06-04T17:31:13Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:13.18736 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:13 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:13.96582 time="2018-06-04T17:31:13Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:13.96587 time="2018-06-04T17:31:13Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:14.06788 time="2018-06-04T17:31:14Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:209: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:default:default" cannot list pods at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:14.18789 gitlab1-gitlab.default:8005 10.233.105.28:38484 - - [2018/06/04:17:31:14 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 0.001
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/health_check?token=[FILTERED]" for 127.0.0.1 at 2018-06-04 17:31:14 +0000
Processing by HealthCheckController#index as HTML
Parameters: {"token"=>"[FILTERED]"}
Completed 200 OK in 52ms (Views: 0.2ms | ActiveRecord: 8.2ms)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/health_check","format":"html","controller":"HealthCheckController","action":"index","status":200,"duration":53.51,"view":0.18,"db":8.18,"time":"2018-06-04T17:31:14.335Z","params":[{"key":"token","value":"[FILTERED]"}],"remote_ip":null,"user_id":null,"username":null}
==> /var/log/gitlab/gitlab-workhorse/current <==
2018-06-04_17:31:14.39003 10.233.78.147:8005 192.168.88.11:35486 - - [2018/06/04:17:31:14 +0000] "GET /health_check?token=SXBAQichEJasbtDSygrD HTTP/1.1" 200 7 "" "kube-probe/1.10" 0.065
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/health_check?token=[FILTERED]" for 127.0.0.1 at 2018-06-04 17:31:14 +0000
Processing by HealthCheckController#index as HTML
Parameters: {"token"=>"[FILTERED]"}
==> /var/log/gitlab/prometheus/current <==
2018-06-04_17:31:14.96771 time="2018-06-04T17:31:14Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
2018-06-04_17:31:14.96791 time="2018-06-04T17:31:14Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:267: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:default:default" cannot list nodes at the cluster scope" component="kube_client_runtime" source="kubernetes.go:76"
==> /var/log/gitlab/gitlab-rails/production.log <==
Completed 200 OK in 48ms (Views: 0.2ms | ActiveRecord: 4.2ms)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/health_check","format":"html","controller":"HealthCheckController","action":"index","status":200,"duration":49.29,"view":0.21,"db":4.22,"time":"2018-06-04T17:31:14.941Z","params":[{"key":"token","value":"[FILTERED]"}],"remote_ip":null,"user_id":null,"username":null}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment