Setting up S3 bucket using Minio on microk8s

readme.md

Setting up S3 bucket using Minio on microk8s

Enable minio

microk8s enable minio

will install minio and create a tenant for you.

Create default tenant with:

  Name: microk8s
  Capacity: 20Gi
  Servers: 1
  Volumes: 1
  Storage class: microk8s-hostpath
  TLS: no
  Prometheus: no

+ /var/snap/microk8s/common/plugins/kubectl-minio tenant create microk8s --storage-class microk8s-hostpath --capacity 20Gi --servers 1 --volumes 1 --namespace minio-operator --enable-audit-logs=false --disable-tls --enable-prometheus=false
W1105 12:42:16.348227 3301155 warnings.go:70] unknown field "spec.pools[0].volumeClaimTemplate.metadata.creationTimestamp"

Tenant 'microk8s' created in 'minio-operator' Namespace

  Username: 1GMOCKUSER
  Password: MgMockPassword
  Note: Copy the credentials to a secure location. MinIO will not display these again.

APPLICATION     SERVICE NAME            NAMESPACE       SERVICE TYPE    SERVICE PORT
MinIO           minio                   minio-operator  ClusterIP       80
Console         microk8s-console        minio-operator  ClusterIP       9090

+ set +x
================================
Enabled minio addon.

You can manage minio tenants using the kubectl-minio plugin.

For more details, use

    microk8s kubectl-minio --help

You need username and password to use mc

Install minio client

cd /usr/local/bin
sudo wget https://dl.min.io/client/mc/release/linux-amd64/mc
sudo chmod +x mc

Try to create a bucket

Set alias with credentials

$ kubectl get endpoints -A

minio-operator   microk8s-console            10.1.94.29:9090                               36m
minio-operator   microk8s-hl                 10.1.94.29:9000                               36m
minio-operator   minio                       10.1.94.29:9000                               36m

wait for a few minutes for these to come up. if not, see also this kubernetes/minikube#4350 (comment)

$ mc alias set minio http://10.1.94.29:9000 1GMOCKUSER MgMockPassword
Added `minio` successfully.

Create bucket

mc mb minio/mybucket
Bucket created successfully `minio/mybucket`.

Manage storage with minio console

$ microk8s kubectl-minio proxy -n minio-operator

Starting port forward of the Console UI.

To connect open a browser and go to http://localhost:9090

Current JWT to login: eyJhbGciO...XlDfTjQ
microk8s kubectl-minio proxy -n minio-operator

copy the jwt and open http://127.0.0.1:9090/

paste the jwt to login

Then it will show the tenant called microk8s

Click the tenant and click Console button to see the consol

You can browse and manage buckets, and manange users and more

Questions and TODOs

• Where is the uploaded file in the host machine?
  • /var/snap/microk8s/common/default-storage/
• Is the bucket volume stateful set (persistent) ? yes
• How to serve html

Where is the file?

Namespace is minio-operator

kubectl describe pv -n minio-operator

will show you the persistent volume info

and it will have the source path

ls -la /var/snap/microk8s/common/default-storage/

Make it public bucket with http(s)

You can make the bucket as public in Minio Console. Edit bucket policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "*"
                ]
            },
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::mybucket/*"
            ]
        }
    ]
}

We should already have some service for minio

$ kubectl get endpoints -n minio-operator
operator           10.1.94.26:4222,10.1.94.26:4221   24h
console            10.1.94.27:9443,10.1.94.27:9090   24h
microk8s-console   10.1.94.29:9090                   24h
microk8s-hl        10.1.94.29:9000                   24h
minio              10.1.94.29:9000                   24h

We can just expose the service as a ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minio-ingress
  namespace: minio-operator
  annotations:
    cert-manager.io/cluster-issuer: zerossl
spec:
  ingressClassName: public
  rules:
    - host: minio.yourdomain.com
      http:
        paths:
          - backend:
              service:
                name: microk8s-hl
                port:
                  number: 9000
            path: /
            pathType: Prefix
  tls:
    - hosts:
      - minio.yourdomain.com
      secretName: minio-svc-tls

How to serve SPA (TBC)

• default file (index.html)
• redirect for 404