Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Increased to 1.5G:
# Default values for orchestrator resources defined in 'topology'.
orchestrator:
image: "vitess/orchestrator:latest"
resources:
limits:
memory: "512Mi"
cpu: "100m"
mysqlResources:
limits:
memory: "1.5Gi"
cpu: "100m"
Also changed dataVolumeType:
# Default values for orchestrator resources defined in 'topology'.
orchestrator:
image: "vitess/orchestrator:latest"
resources:
limits:
memory: "512Mi"
cpu: "100m"
mysqlResources:
limits:
memory: "1.5Gi"
cpu: "100m"
dataVolumeType: PersistentVolume
kubectl get pods
NAME READY STATUS RESTARTS AGE
etcd-global-45lvb 1/1 Running 0 7m
etcd-global-k5fn4 1/1 Running 0 7m
etcd-global-sgmrx 1/1 Running 0 7m
etcd-zone1-2pmxp 1/1 Running 0 7m
etcd-zone1-mt5k9 1/1 Running 0 7m
etcd-zone1-vd9jz 1/1 Running 0 7m
orchestrator-6dd7988576-br8xc 0/2 CrashLoopBackOff 7 7m
vtctld-77d8fcd4cc-tj7nh 1/1 Running 0 7m
vtgate-zone1-75f77cfbcf-grrnz 1/1 Running 0 7m
vtgate-zone1-75f77cfbcf-gz7w9 1/1 Running 0 7m
vtgate-zone1-75f77cfbcf-pqpfr 1/1 Running 0 7m
zone1-main-80-x-replica-0 0/2 Pending 0 7m
zone1-main-x-80-replica-0 0/2 Pending 0 7m
kubectl logs orchestrator-6dd7988576-br8xc -c mysql
2017-11-04 10:00:10 0 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
2017-11-04 10:00:10 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2017-11-04 10:00:10 0 [Note] mysqld (mysqld 5.6.29-76.2-56) starting as process 1 ...
kubectl logs -p orchestrator-6dd7988576-br8xc -c mysql
2017-11-04 10:00:10 0 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
2017-11-04 10:00:10 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2017-11-04 10:00:10 0 [Note] mysqld (mysqld 5.6.29-76.2-56) starting as process 1 ...
kubectl logs orchestrator-6dd7988576-br8xc -c orchestrator
+ mysqladmin -h 127.0.0.1 ping
mysqladmin: connect to server at '127.0.0.1' failed
error: 'Can't connect to MySQL server on '127.0.0.1' (111)'
Check that mysqld is running on 127.0.0.1 and that the port is 3306.
You can check this by doing 'telnet 127.0.0.1 3306'
+ sleep 1
+ mysqladmin -h 127.0.0.1 ping
mysqladmin: connect to server at '127.0.0.1' failed
error: 'Can't connect to MySQL server on '127.0.0.1' (111)'
Check that mysqld is running on 127.0.0.1 and that the port is 3306.
You can check this by doing 'telnet 127.0.0.1 3306'
+ sleep 1
kubectl describe pod orchestrator-6dd7988576-br8xc
Name: orchestrator-6dd7988576-br8xc
Namespace: default
Node: kubernetes-minion-group-s8t2/10.128.0.4
Start Time: Sat, 04 Nov 2017 05:53:45 -0400
Labels: app=vitess
component=orchestrator
pod-template-hash=2883544132
Annotations: kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicaSet","namespace":"default","name":"orchestrator-6dd7988576","uid":"082cafb2-c146-11e7-aeb6-42010a80...
Status: Running
IP: 10.64.1.19
Created By: ReplicaSet/orchestrator-6dd7988576
Controlled By: ReplicaSet/orchestrator-6dd7988576
Containers:
orchestrator:
Container ID: docker://423b5bafade65a699064b3bbe3cff053ddb67bc2bdb1a36bc52749805b019167
Image: vitess/orchestrator:latest
Image ID: docker-pullable://vitess/orchestrator@sha256:a11793361745bab4c962cdcbde74cb1e28b5838dc5fa44df1e2ea4532bfb5a45
Port: <none>
Command:
bash
-c
set -x
until mysqladmin -h 127.0.0.1 ping; do sleep 1; done
exec orchestrator http
State: Running
Started: Sat, 04 Nov 2017 06:11:47 -0400
Last State: Terminated
Reason: Error
Exit Code: 137
Started: Sat, 04 Nov 2017 06:05:57 -0400
Finished: Sat, 04 Nov 2017 06:11:47 -0400
Ready: False
Restart Count: 3
Limits:
cpu: 100m
memory: 512Mi
Requests:
cpu: 100m
memory: 512Mi
Liveness: http-get http://:3000/ delay=300s timeout=30s period=10s #success=1 #failure=3
Readiness: http-get http://:3000/ delay=0s timeout=10s period=10s #success=1 #failure=3
Environment: <none>
Mounts:
/orc/conf from config (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-lnv6r (ro)
mysql:
Container ID: docker://301d280440b7e266ed464df3a616ae10cd70573418b99f38888a27b4459d0960
Image: vitess/orchestrator:latest
Image ID: docker-pullable://vitess/orchestrator@sha256:a11793361745bab4c962cdcbde74cb1e28b5838dc5fa44df1e2ea4532bfb5a45
Port: <none>
Command:
mysqld
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Sat, 04 Nov 2017 06:10:29 -0400
Finished: Sat, 04 Nov 2017 06:10:31 -0400
Ready: False
Restart Count: 8
Limits:
cpu: 100m
memory: 1536Mi
Requests:
cpu: 100m
memory: 1536Mi
Liveness: exec [mysqladmin ping] delay=300s timeout=10s period=10s #success=1 #failure=3
Environment: <none>
Mounts:
/var/lib/mysql from data (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-lnv6r (ro)
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
Volumes:
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: orchestrator
Optional: false
data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: orchestrator-data
ReadOnly: false
default-token-lnv6r:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-lnv6r
Optional: false
QoS Class: Guaranteed
Node-Selectors: <none>
Tolerations: node.alpha.kubernetes.io/notReady:NoExecute for 300s
node.alpha.kubernetes.io/unreachable:NoExecute for 300s
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
21m 21m 4 default-scheduler Warning FailedScheduling PersistentVolumeClaim is not bound: "orchestrator-data" (repeated 4 times)
20m 20m 1 default-scheduler Normal Scheduled Successfully assigned orchestrator-6dd7988576-br8xc to kubernetes-minion-group-s8t2
20m 20m 1 kubelet, kubernetes-minion-group-s8t2 Normal SuccessfulMountVolume MountVolume.SetUp succeeded for volume "config"
20m 20m 1 kubelet, kubernetes-minion-group-s8t2 Normal SuccessfulMountVolume MountVolume.SetUp succeeded for volume "default-token-lnv6r"
20m 20m 1 kubelet, kubernetes-minion-group-s8t2 Normal SuccessfulMountVolume MountVolume.SetUp succeeded for volume "pvc-080ee779-c146-11e7-aeb6-42010a800002"
20m 20m 1 kubelet, kubernetes-minion-group-s8t2 spec.containers{orchestrator} Normal Pulling pulling image "vitess/orchestrator:latest"
20m 20m 1 kubelet, kubernetes-minion-group-s8t2 spec.containers{orchestrator} Normal Pulled Successfully pulled image "vitess/orchestrator:latest"
20m 20m 1 kubelet, kubernetes-minion-group-s8t2 spec.containers{orchestrator} Normal Created Created container
20m 20m 1 kubelet, kubernetes-minion-group-s8t2 spec.containers{orchestrator} Normal Started Started container
20m 20m 2 kubelet, kubernetes-minion-group-s8t2 spec.containers{mysql} Normal Started Started container
20m 20m 2 kubelet, kubernetes-minion-group-s8t2 spec.containers{mysql} Warning BackOff Back-off restarting failed container
20m 20m 3 kubelet, kubernetes-minion-group-s8t2 spec.containers{mysql} Normal Pulling pulling image "vitess/orchestrator:latest"
20m 20m 3 kubelet, kubernetes-minion-group-s8t2 spec.containers{mysql} Normal Pulled Successfully pulled image "vitess/orchestrator:latest"
20m 20m 3 kubelet, kubernetes-minion-group-s8t2 spec.containers{mysql} Normal Created Created container
20m 5m 68 kubelet, kubernetes-minion-group-s8t2 Warning FailedSync Error syncing pod
20m 48s 119 kubelet, kubernetes-minion-group-s8t2 spec.containers{orchestrator} Warning Unhealthy Readiness probe failed: Get http://10.64.1.19:3000/: dial tcp 10.64.1.19:3000: getsockopt: connection refused
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment