Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Deploy rabbitmq-autocluster on k8s with persistent storage (EBS)
#!/bin/bash
set -eo pipefail
export KUBE_NAMESPACE=test
export REPLICA_COUNT=3
cat <<EOF | kubectl apply -f -
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: rabbitmq
namespace: $KUBE_NAMESPACE
spec:
serviceName: rabbitmq
replicas: $REPLICA_COUNT
template:
metadata:
labels:
app: rabbitmq
spec:
terminationGracePeriodSeconds: 10
containers:
- name: rabbitmq-autocluster
image: pivotalrabbitmq/rabbitmq-autocluster
ports:
- name: http
protocol: TCP
containerPort: 15672
- name: amqp
protocol: TCP
containerPort: 5672
livenessProbe:
exec:
command: ["rabbitmqctl", "status"]
initialDelaySeconds: 30
timeoutSeconds: 5
readinessProbe:
exec:
command: ["rabbitmqctl", "status"]
initialDelaySeconds: 10
timeoutSeconds: 5
imagePullPolicy: Always
env:
- name: MY_POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: HOSTNAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: RABBITMQ_USE_LONGNAME
value: "true"
- name: RABBITMQ_NODENAME
value: "rabbit@\$(HOSTNAME).rabbitmq.\$(NAMESPACE).svc.cluster.local"
- name: AUTOCLUSTER_TYPE
value: "k8s"
- name: AUTOCLUSTER_DELAY
value: "10"
- name: K8S_ADDRESS_TYPE
value: "hostname"
- name: K8S_SERVICE_NAME
value: rabbitmq
- name: K8S_HOSTNAME_SUFFIX
value: ".rabbitmq.\$(NAMESPACE).svc.cluster.local"
- name: AUTOCLUSTER_CLEANUP
value: "false"
- name: CLEANUP_WARN_ONLY
value: "true"
volumeMounts:
- name: rabbitmq-data
mountPath: /var/lib/rabbitmq/mnesia
volumeClaimTemplates:
- metadata:
name: rabbitmq-data
annotations:
volume.beta.kubernetes.io/storage-class: gp2
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 4Gi
EOF
# Headless service for Pod DNS
cat <<EOF | kubectl apply -f -
kind: Service
apiVersion: v1
metadata:
namespace: $KUBE_NAMESPACE
name: rabbitmq
labels:
app: rabbitmq
spec:
clusterIP: None
ports:
- name: http
protocol: TCP
port: 15672
targetPort: 15672
- name: amqp
protocol: TCP
port: 5672
targetPort: 5672
selector:
app: rabbitmq
EOF
# LoadBalancer service for public access
cat <<EOF | kubectl apply -f -
kind: Service
apiVersion: v1
metadata:
namespace: $KUBE_NAMESPACE
name: rabbitmq-lb
labels:
app: rabbitmq
type: LoadBalancer
spec:
type: LoadBalancer
ports:
- name: http
protocol: TCP
port: 15672
targetPort: 15672
- name: amqp
protocol: TCP
port: 5672
targetPort: 5672
selector:
app: rabbitmq
EOF
echo "Waiting for StatefulSet to complete rolled out"
# kubectl rollout status statefulset/rabbitmq -n $KUBE_NAMESPACE # Not supported in k8s v1.6 and prior
for i in $(seq 1 120); do
if kubectl exec rabbitmq-$(($REPLICA_COUNT - 1)) -n $KUBE_NAMESPACE -- rabbitmqctl status &> /dev/null; then
break
fi
sleep 1s
done
if [[ "$i" == 120 ]]; then
echo "StatefulSet taking too long to complete, you need to manual install..."
exit 1
fi
echo "Setting up HA policy"
kubectl exec rabbitmq-0 -n $KUBE_NAMESPACE -- rabbitmqctl set_policy ha-all "" '{"ha-mode":"all","ha-sync-mode":"automatic"}'
@prcongithub

This comment has been minimized.

Copy link

prcongithub commented Jan 2, 2020

Aren't there any prerequisites to this script?

Any ideas on how to add default user?

@gmr

This comment has been minimized.

Copy link
Owner Author

gmr commented Jan 4, 2020

Prerequisites: You need to be able to run kubectl and have a k8s cluster.

This is super naive about RabbitMQ configuration and just uses the official pivotalrabbitmq/rabbitmq-autocluster docker image.

You'd need to configure RabbitMQ and use the API to add a user or use the definitions setting to load in definitions that contain the user.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.