ikurni

# After 2 years deployment, suddenly Authentication and Console Operator is degraded, and after we check the operator status, it shows that
# the certificate is expired. To solve this, just delete the secret, and Operator will create new certificate. Assume we still use
# Openshift default certificate

### Delete Ingress CA Secret
oc project openshift-ingress-operator
oc get secret router-ca -oyaml > router-ca.yaml
oc delete secret router-ca
oc delete pod --all
oc get secret router-ca

ikurni

https://access.redhat.com/solutions/396753 --> Remove Locked Entities Manually

##How to Clean up Task in RHV :
#List Tasks :
/usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh 
#Kill Tasks :
/usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh -t <taskID>

ikurni

### If nodes is stuck in pulling images, or connection are too slow, sometime process can't move. Below is how to re-trigger ###

/run/bin/machine-config-daemon firstboot-complete-machineconfig

ikurni

### Openshift Summarized KB ###
https://access.redhat.com/node/5218861

### Cluster Updates Without Error but Machine Config Pools Degraded with `Marking Degraded due to: unexpected on-disk state` on 4.6 and newer
https://access.redhat.com/solutions/5598401

### How to Use 'fio' to Check Etcd Disk Performance in OCP
https://access.redhat.com/solutions/4885641

### How to defrag etcd to decrease DB size in OpenShift 4

ikurni

### Performance check command from etcd pod

# oc rsh <etcd-pod> -n openshift-etcd
$ etcdctl check perf --load="m"
$ etcdctl check perf --load='l'
$ etcdctl --write-out=table endpoint status
$ etcdctl --write-out=table endpoint health
$ etcdctl --write-out=table member list

### Collect metrics from the cluster.

ikurni

### Most probably is caused by different pull secret that is being used by serviceaccount to pull the images ###
### Edit imagePullSecret which is used by serviceaccount ###
oc edit sa <serviceaccountname>
    # Edit below section 
    apiVersion: v1
    imagePullSecrets:
    - name: jenkins-dockercfg-8nln8 #change to available pullsecrets which can be used to pull from image-registry#
# Save the config #

ikurni

### Step 1
oc delete pod <podsname> -n myproject --grace-period=0 --force

### Step 2
oc edit pod <podsname>
#Remove deletionTimestamp
  #Before: deletionTimestamp: 2019-12-31T11:40:28Z
  #After: deletionTimestamp: null
#Remove Finalizers
  #Before

ikurni

### Create SSL file for HTTPS traffic
mkdir /etc/haproxy/ssl
cat /root/wildcard.example.com.crt /root/wildcard.example.com.key >> /root/wildcard.example.com.pem
mv /root/wildcard.example.com.pem /etc/haproxy/ssl/

### Configure HAProxy.cfg to accept HTTPS, redirect HTTPS to HTTP and replace header to targeted URL
vi /etc/haproxy/haproxy.cfg

#---
#---------------------------------------------------------------------

ikurni

### Run below command to enable rook-ceph-tools in OCS Cluster

oc patch OCSInitialization ocsinit -n openshift-storage --type json --patch  '[{ "op": "replace", "path": "/spec/enableCephTools", "value": true }]'

ikurni

### In order to remove OCS Cluster manually, when resources are stucked, need to patch the resources manually as below :

[root@ocpbastion ~]# oc patch -n openshift-stroage crd/cephobjectstoreusers.ceph.rook.io --type=merge -p '{"metadata": {"finalizers":null}}'
customresourcedefinition.apiextensions.k8s.io/cephobjectstoreusers.ceph.rook.io patched
[root@ocpbastion ~]# oc patch -n openshift-stroage crd/cephblockpools.ceph.rook.io --type=merge -p '{"metadata": {"finalizers":null}}'
customresourcedefinition.apiextensions.k8s.io/cephblockpools.ceph.rook.io patched
[root@ocpbastion ~]# oc patch -n openshift-stroage crd/cephclusters.ceph.rook.io --type=merge -p '{"metadata": {"finalizers":null}}'
customresourcedefinition.apiextensions.k8s.io/cephclusters.ceph.rook.io patched
[root@ocpbastion ~]# oc patch -n openshift-stroage crd/cephfilesystems.ceph.rook.io --type=merge -p '{"metadata": {"finalizers":null}}'
customresourcedefinition.apiextensions.k8s.io/cephfilesystems.ceph.rook.io patched