Skip to content

Instantly share code, notes, and snippets.

@mrhillsman
Last active April 6, 2020 14:46
Show Gist options
  • Save mrhillsman/0960401867e5a6380d66b0e170de650c to your computer and use it in GitHub Desktop.
Save mrhillsman/0960401867e5a6380d66b0e170de650c to your computer and use it in GitHub Desktop.

https://aws.amazon.com/getting-started/tutorials/create-network-file-system/
skip sections 2, 3, and 5
you will need to connect to your nodes (master(s), worker(s)) to mount the filesystem
bootstrap node is used but if not you will need to create an instance within the VPC your openshift cluster lives in to access your openshift machines

https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client

---
kind: ServiceAccount
apiVersion: v1
metadata:
  name: nfs-client-provisioner
  namespace: openshift-operators
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    namespace: openshift-operators
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  namespace: openshift-operators
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  namespace: openshift-operators
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: openshift-operators
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: nfs-dynamic
  annotations:
    storageclass.kubernetes.io/is-default-class: 'true'
provisioner: storage.openshift.io/nfs
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
---
kind: Deployment
apiVersion: apps/v1
metadata:
  annotations:
    deployment.kubernetes.io/revision: '1'
  name: nfs-client-provisioner
  namespace: openshift-operators
  labels:
    app: nfs-client-provisioner
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      restartPolicy: Always
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: 'quay.io/external_storage/nfs-client-provisioner:latest'
          env:
            - name: PROVISIONER_NAME
              value: storage.openshift.io/nfs
            - name: NFS_SERVER
              value: << amazon provided or other NFS server IP or DNS >>
            - name: NFS_PATH
              value: << amazon provided or other NFS server path (server.com:/ <- path; after colon >>
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          imagePullPolicy: IfNotPresent
      serviceAccount: nfs-client-provisioner
      volumes:
        - name: nfs-client-root
          nfs:
            server: << amazon provided or other NFS server IP or DNS >>
            path: << amazon provided or other NFS server path (server.com:/ <- path; after colon >>
  strategy:
    type: Recreate
oc adm policy add-scc-to-user hostmount-anyuid system:serviceaccount:openshift-operators:nfs-client-provisioner

NOTES:

when you scale up machinesets (create additional nodes) you have to manually go into the new machine and mount the NFS
there are 6 security groups for the default openshift-install setup VPC (master, bootstrap, work, default, k8s-elb)
should add the default to each node
would be great to add this as part of the openshift cluster install via the openshift-install tool
How this might work:
Authenticate with AWS
Assign the default VPC security group to each node EFS is to be mounted in
Make sure the EFS is assigned to the appropriate VPC
Create the EFS
For each node
sudo mkdir /tmp/efs
cd /tmp
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport <>:/ efs
Create the storage class in openshift

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment