Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Rails Kubernetes Manifests
apiVersion: v1
kind: ConfigMap
metadata:
name: example
namespace: default
data:
APPLICATION_HOST: example.com
LANG: en_US.UTF-8
PIDFILE: /tmp/server.pid
PORT: "3000"
RACK_ENV: production
RAILS_ENV: production
RAILS_LOG_TO_STDOUT: "true"
RAILS_SERVE_STATIC_FILES: "true"
apiVersion: batch/v1
kind: Job
metadata:
generateName: db-migrate-
labels:
app.kubernetes.io/name: example
namespace: default
spec:
template:
metadata:
labels:
app.kubernetes.io/name: example
spec:
containers:
- command:
- rails
- db:migrate
envFrom:
- configMapRef:
name: env
- secretRef:
name: env
image: docker.io/mycompany/myapplication:abcd123
imagePullPolicy: IfNotPresent
name: main
restartPolicy: Never
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/name: example
process: web
name: example-web
namespace: default
spec:
selector:
matchLabels:
app.kubernetes.io/name: example
process: web
template:
metadata:
labels:
app.kubernetes.io/name: example
process: web
spec:
containers:
- env:
- name: PORT
value: "3000"
envFrom:
- configMapRef:
name: example
- secretRef:
name: example
image: docker.io/mycompany/myapplication:abcd123
imagePullPolicy: IfNotPresent
name: main
ports:
- containerPort: 3000
name: http
protocol: TCP
readinessProbe:
httpGet:
httpHeaders:
- name: Host
value: example.com
path: /robots.txt
port: 3000
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 2
initContainers:
- command:
- rake
- db:abort_if_pending_migrations
envFrom:
- configMapRef:
name: example
- secretRef:
name: example
image: docker.io/mycompany/myapplication:abcd123
imagePullPolicy: IfNotPresent
name: migrations
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
labels:
app.kubernetes.io/name: example
name: example
namespace: default
spec:
rules:
- host: example.com
http:
paths:
- backend:
serviceName: example-web
servicePort: 3000
tls:
- hosts:
- example.com
secretName: example-tls
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/name: example
process: web
name: example-web
namespace: default
spec:
ports:
- name: http
port: 3000
protocol: TCP
targetPort: http
selector:
app.kubernetes.io/name: example
process: web
type: ClusterIP
@jandudulski
Copy link

jandudulski commented Nov 4, 2021

Also, I struggle to use this approach with google_cloud_proxy. initContainer runs just a single container and the only solution I found is "build your own image with proxy binary inside". Do you have a better solution for that?

And the job is not marked as completed while proxy is running inside.

@jferris
Copy link
Author

jferris commented Nov 4, 2021

Why do you actually use generateName for the job instead of just name?

You can use name or generateName, but if you use name your CI process will need to delete the previous job first or migrations won't run.

And the job is not marked as completed while proxy is running inside.

Kubernetes itself doesn't have any special knowledge of essential containers or sidecars so if you use a sidecar container in a job you'll need to manually kill it in the main container once the job is complete.

initContainer runs just a single container and the only solution I found is "build your own image with proxy binary inside". Do you have a better solution for that?

If your initContainer needs a sidecar to connect to the database, that approach might be difficult. A few ideas you could try:

  • Run the cloud proxy as a separate deployment or daemonset rather than a sidecar so that it's available before the pod starts.
  • Replace the initContainer with a readiness probe so the sidecar is running by the time you need to connect. If you do this, you may need to configure it to restart the container if the probe fails so that Rails will cache the latest database definitions.

@jandudulski
Copy link

jandudulski commented Nov 4, 2021

@jferris thanks for your reply and suggestions.

In the meantime I found why generateName makes sense - jobs are immutable so you cannot patch them to redeploy.

so if you use a sidecar container in a job you'll need to manually kill it in the main container once the job is complete.

that's not trivial on its own but this solution worked for me: https://stackoverflow.com/a/64650086

@kwstannard
Copy link

kwstannard commented Feb 1, 2022

To improve the speed of this, create a route specifically for the readinessProbe or even better, use a startupProbe (I am using the health_check gem). That route should, among other things, return 500 if migrations are pending. Then you can get rid of the initContainer. What will then happen is the app pods will start concurrently with the migration job and sit there returning 500's to k8s until the migrations finish, at which point they will return 200 and k8s will mark them as ready for traffic. This will cut deployment time in more than half compared to initContainers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment