Skip to content

Instantly share code, notes, and snippets.

@danehans

danehans/test.md Secret

Last active August 19, 2019 23:24
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save danehans/4d43adf773166b41470783e4ca4f6fb6 to your computer and use it in GitHub Desktop.
Save danehans/4d43adf773166b41470783e4ca4f6fb6 to your computer and use it in GitHub Desktop.
Download ZIP
ocp_proxy_trustedca_change_test
Raw
test.md
  1. Check existing proxy:
$ oc get proxy/cluster
NAME      AGE
cluster   158m
$ oc get proxy/cluster -o yaml
apiVersion: config.openshift.io/v1
kind: Proxy
metadata:
  creationTimestamp: "2019-08-19T19:34:05Z"
  generation: 5
  name: cluster
  resourceVersion: "54582"
  selfLink: /apis/config.openshift.io/v1/proxies/cluster
  uid: 4e817e84-c2b8-11e9-bfb7-020d4b98ea92
spec:
  trustedCA:
    name: ""
status: {}

Note: Nil spec and status.

Check for the existence of configmap trusted-ca-bundle:

$ oc get cm/trusted-ca-bundle  -n openshift-config-managed -o yaml
apiVersion: v1
data:
  ca-bundle.crt: |
    # ACCVRAIZ1
    -----BEGIN CERTIFICATE-----
<SNIP>
    -----END CERTIFICATE-----
kind: ConfigMap
metadata:
  creationTimestamp: "2019-08-19T22:24:03Z"
  name: trusted-ca-bundle
  namespace: openshift-config-managed
  resourceVersion: "58310"
  selfLink: /api/v1/namespaces/openshift-config-managed/configmaps/trusted-ca-bundle
  uid: 0c97a54d-c2d0-11e9-bf2d-02b46012d9c0

Note: configmap openshift-config-managed/trusted-ca-bundle contains the system bundle only. This is because a) a user bundle has not been provided and b) proxy.spec.trustedCA does not reference a configmap name containing a user bundle.

Check for the existence of configmap user-ca-bundle:

$ oc get cm/user-ca-bundle  -n openshift-config -o yaml
Error from server (NotFound): configmaps "user-ca-bundle" not found

Create configmap with ca cert that signed the proxy's identity cert.

$ oc create -f ~/tmp/proxy-user-configmap.yaml

Verify the creation of the configmap:

$ oc get cm/user-ca-bundle  -n openshift-config -o yaml
apiVersion: v1
data:
  ca-bundle.crt: |
    -----BEGIN CERTIFICATE-----
    MIIC6jCCAdKgAwIBAgIBATANBgkqhkiG9w0BAQsFADAmMSQwIgYDVQQDDBtvcGVu
    c2hpZnQtc2lnbmVyQDE1NjU2NDA4NzYwHhcNMTkwODEyMjAxNDM1WhcNMjQwODEw
    MjAxNDM2WjAmMSQwIgYDVQQDDBtvcGVuc2hpZnQtc2lnbmVyQDE1NjU2NDA4NzYw
    ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCzbYmY8T9SPUS9c4VG1DNA
    Ub5WKz/NJigFVJ0ei9+mIMF2mHFJlRjHaOs7HaOWTcQNkNBhBfKvNcK8ZKd+kBVB
    CaT9TXOmXlDpaMbnGiGQeBaGrA2S1FzxkQbZDaztN8S3lgydzAVYN7QehRKtP7Zp
    +55gdlw0qvQiepRQaq4RWCgoALY4aJzZRWc/ZTY+wiMURuusC/viVpxhnOrZ5ZkD
    FjnGY+MxB2O4KuSuI6868Sk24ZQ7d9ocRCNRbsinpZTafz9/IpxxoR06PsSNN0NI
    4cpckcTmSysLePTSr+cVvgc9Nr+TJxISC3gtn2U80l/uml1crQ7yfpq7Lf/NPa/b
    AgMBAAGjIzAhMA4GA1UdDwEB/wQEAwICpDAPBgNVHRMBAf8EBTADAQH/MA0GCSqG
    SIb3DQEBCwUAA4IBAQCTpgR0rXLZH9WFu9RBxqa7MXdAnmb7hlcKkQHRqVKgTk2N
    z2Hio6l9mHdi42gihWAUKBIwYs2Axk/jjqaI03+CyutZvdnt9N55lsa0qntHuFm5
    jstKn08+IiX6tRRhMqIK27exV0HRbzeAyMDbhjReHnq1OnW/ycyv4p5BdOtuxTox
    8yOmu4a5lKgNfmK5qpE/VsX2jEpqmjck/JaVldcGoICd2DCoMYdHpm7ROFmdTApJ
    WqtDEPIq0PUnMrlr6Ba5GCS3385BWSMvYsbzIiyKXn7hEGh/oFQR2HXix7lYyEyd
    7t6Hv8LhnjP4+HoGlxTSReJ0lXv7mEK0FKXOdkHd
    -----END CERTIFICATE-----
kind: ConfigMap
metadata:
  creationTimestamp: "2019-08-19T22:16:16Z"
  name: user-ca-bundle
  namespace: openshift-config
  resourceVersion: "56213"
  selfLink: /api/v1/namespaces/openshift-config/configmaps/user-ca-bundle
  uid: f6b850b8-c2ce-11e9-9202-0a52930d2f5c

Since proxy "cluster" does not refer to this configmap for spec.trustedCA, reconciliation for this object is skipped:

2019/08/19 15:16:17 Reconciling additional trust bundle configmap 'openshift-config/user-ca-bundle'
2019/08/19 15:16:17 configmap 'openshift-config/user-ca-bundle' name differs from trustedCA of proxy 'cluster' or trustedCA not set; reconciliation will be skipped

Verify configmap openshift-config-managed/trusted-ca-bundle still only contains the system trust bundle:

$ oc get cm/trusted-ca-bundle  -n openshift-config-managed -o yaml
apiVersion: v1
data:
  ca-bundle.crt: |
    # ACCVRAIZ1
    -----BEGIN CERTIFICATE-----
<SNIP>
    -----END CERTIFICATE-----
kind: ConfigMap
metadata:
  creationTimestamp: "2019-08-19T22:24:03Z"
  name: trusted-ca-bundle
  namespace: openshift-config-managed
  resourceVersion: "58310"
  selfLink: /api/v1/namespaces/openshift-config-managed/configmaps/trusted-ca-bundle
  uid: 0c97a54d-c2d0-11e9-bf2d-02b46012d9c0

Update proxy "cluster" with:

spec:
  httpProxy: http://admin:admin@35.196.128.173:3128
  httpsProxy: https://admin:admin@35.231.5.161:3128
  noProxy: example.com
  readinessEndpoints:
  - http://www.google.com
  - https://www.google.com
  trustedCA:
    name: user-ca-bundle

Check operator logs:

2019/08/19 15:34:45 Reconciling proxy 'cluster'
2019/08/19 15:34:46 Reconciling configmap from  trusted-ca-bundle/openshift-config-managed
2019/08/19 15:34:46 trusted-ca-bundle changed, updating 0 configMaps
2019/08/19 15:34:47 Reconciling proxy 'cluster' complete

Note: Since no operators are requesting trust bundle injection, you see updating 0 configMaps.

Verify proxy status:

$ oc get proxy/cluster -o yaml
apiVersion: config.openshift.io/v1
kind: Proxy
metadata:
  creationTimestamp: "2019-08-19T19:34:05Z"
  generation: 6
  name: cluster
  resourceVersion: "61101"
  selfLink: /apis/config.openshift.io/v1/proxies/cluster
  uid: 4e817e84-c2b8-11e9-bfb7-020d4b98ea92
spec:
  httpProxy: http://admin:admin@35.196.128.173:3128
  httpsProxy: https://admin:admin@35.231.5.161:3128
  noProxy: example.com
  readinessEndpoints:
  - http://www.google.com
  - https://www.google.com
  trustedCA:
    name: user-ca-bundle
status:
  httpProxy: http://admin:admin@35.196.128.173:3128
  httpsProxy: https://admin:admin@35.231.5.161:3128
  noProxy: 10.0.0.0/16,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,api-int.dhansen.devcluster.openshift.com,api.dhansen.devcluster.openshift.com,etcd-0.dhansen.devcluster.openshift.com,etcd-1.dhansen.devcluster.openshift.com,etcd-2.dhansen.devcluster.openshift.com,example.com,localhost

Verify configmap openshift-config-managed/trusted-ca-bundle contains the combined user/system trust bundle:

$ oc get cm/trusted-ca-bundle  -n openshift-config-managed -o yaml
apiVersion: v1
data:
  ca-bundle.crt: |
    -----BEGIN CERTIFICATE-----
    MIIC6jCCAdKgAwIBAgIBATANBgkqhkiG9w0BAQsFADAmMSQwIgYDVQQDDBtvcGVu
    c2hpZnQtc2lnbmVyQDE1NjU2NDA4NzYwHhcNMTkwODEyMjAxNDM1WhcNMjQwODEw
    MjAxNDM2WjAmMSQwIgYDVQQDDBtvcGVuc2hpZnQtc2lnbmVyQDE1NjU2NDA4NzYw
    ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCzbYmY8T9SPUS9c4VG1DNA
    Ub5WKz/NJigFVJ0ei9+mIMF2mHFJlRjHaOs7HaOWTcQNkNBhBfKvNcK8ZKd+kBVB
    CaT9TXOmXlDpaMbnGiGQeBaGrA2S1FzxkQbZDaztN8S3lgydzAVYN7QehRKtP7Zp
    +55gdlw0qvQiepRQaq4RWCgoALY4aJzZRWc/ZTY+wiMURuusC/viVpxhnOrZ5ZkD
    FjnGY+MxB2O4KuSuI6868Sk24ZQ7d9ocRCNRbsinpZTafz9/IpxxoR06PsSNN0NI
    4cpckcTmSysLePTSr+cVvgc9Nr+TJxISC3gtn2U80l/uml1crQ7yfpq7Lf/NPa/b
    AgMBAAGjIzAhMA4GA1UdDwEB/wQEAwICpDAPBgNVHRMBAf8EBTADAQH/MA0GCSqG
    SIb3DQEBCwUAA4IBAQCTpgR0rXLZH9WFu9RBxqa7MXdAnmb7hlcKkQHRqVKgTk2N
    z2Hio6l9mHdi42gihWAUKBIwYs2Axk/jjqaI03+CyutZvdnt9N55lsa0qntHuFm5
    jstKn08+IiX6tRRhMqIK27exV0HRbzeAyMDbhjReHnq1OnW/ycyv4p5BdOtuxTox
    8yOmu4a5lKgNfmK5qpE/VsX2jEpqmjck/JaVldcGoICd2DCoMYdHpm7ROFmdTApJ
    WqtDEPIq0PUnMrlr6Ba5GCS3385BWSMvYsbzIiyKXn7hEGh/oFQR2HXix7lYyEyd
    7t6Hv8LhnjP4+HoGlxTSReJ0lXv7mEK0FKXOdkHd
    -----END CERTIFICATE-----
    # ACCVRAIZ1
    -----BEGIN CERTIFICATE-----
<SNIP>
    -----END CERTIFICATE-----
kind: ConfigMap
metadata:
  creationTimestamp: "2019-08-19T22:24:03Z"
  name: trusted-ca-bundle
  namespace: openshift-config-managed
  resourceVersion: "61097"
  selfLink: /api/v1/namespaces/openshift-config-managed/configmaps/trusted-ca-bundle
  uid: 0c97a54d-c2d0-11e9-bf2d-02b46012d9c0

Check the status of CNO:

$ oc get clusteroperator/network
NAME      VERSION                         AVAILABLE   PROGRESSING   DEGRADED   SINCE
network   4.2.0-0.okd-2019-08-19-190617   True        False         False      3h1m

Everything is working as expected. Now remove spec.trustedCA from proxy "cluster".

$ oc get proxy/cluster -o yaml
apiVersion: config.openshift.io/v1
kind: Proxy
metadata:
  creationTimestamp: "2019-08-19T19:34:05Z"
  generation: 7
  name: cluster
  resourceVersion: "67260"
  selfLink: /apis/config.openshift.io/v1/proxies/cluster
  uid: 4e817e84-c2b8-11e9-bfb7-020d4b98ea92
spec:
  httpProxy: http://admin:admin@35.196.128.173:3128
  httpsProxy: https://admin:admin@35.231.5.161:3128
  noProxy: example.com
  readinessEndpoints:
  - http://www.google.com
  - https://www.google.com
  trustedCA:
    name: ""
status:
  httpProxy: http://admin:admin@35.196.128.173:3128
  httpsProxy: https://admin:admin@35.231.5.161:3128
  noProxy: 10.0.0.0/16,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,api-int.dhansen.devcluster.openshift.com,api.dhansen.devcluster.openshift.com,etcd-0.dhansen.devcluster.openshift.com,etcd-1.dhansen.devcluster.openshift.com,etcd-2.dhansen.devcluster.openshift.com,example.com,localhost

This should cause a) proxy reconciliation to throw an error b/c the https://www.google.com readinessEndpoint will fail due to the missing proxy cert b) CNO will transition to a degraded status due to the readinessEndpoint error. c) Since proxy.spec.trustedCA is empty, openshift-config-managed/trusted-ca-bundle is updated to include only the system bundle.

Verify operator logs:

2019/08/19 15:56:34 Reconciling proxy 'cluster'
2019/08/19 15:56:35 Reconciling configmap from  trusted-ca-bundle/openshift-config-managed
2019/08/19 15:56:35 trusted-ca-bundle changed, updating 0 configMaps
2019/08/19 15:56:38 Failed to validate proxy 'cluster': readinessEndpoint probe failed for endpoint 'https://www.google.com': readiness probe failed for endpoint 'https://www.google.com' using proxy 'https://admin:admin@35.231.5.161:3128': Get https://www.google.com: proxyconnect tcp: x509: certificate signed by unknown authority
2019/08/19 15:56:39 Updated ClusterOperator with conditions:
- lastTransitionTime: "2019-08-19T22:56:38Z"
  message: 'The configuration is invalid for proxy ''cluster'' (readinessEndpoint
    probe failed for endpoint ''https://www.google.com'': readiness probe failed for
    endpoint ''https://www.google.com'' using proxy ''https://admin:admin@35.231.5.161:3128'':
    Get https://www.google.com: proxyconnect tcp: x509: certificate signed by unknown
    authority). Use ''oc edit proxy.config.openshift.io cluster'' to fix.'
  reason: InvalidProxyConfig
  status: "True"
  type: Degraded
- lastTransitionTime: "2019-08-19T19:34:51Z"
  status: "True"
  type: Upgradeable
- lastTransitionTime: "2019-08-19T21:51:18Z"
  status: "False"
  type: Progressing
- lastTransitionTime: "2019-08-19T19:37:11Z"
  status: "True"
  type: Available
2019/08/19 15:56:40 Reconciling proxy 'cluster'
2019/08/19 15:56:43 Failed to validate proxy 'cluster': readinessEndpoint probe failed for endpoint 'https://www.google.com': readiness probe failed for endpoint 'https://www.google.com' using proxy 'https://admin:admin@35.231.5.161:3128': Get https://www.google.com: proxyconnect tcp: x509: certificate signed by unknown authority
<SNIP>

Verify configmap openshift-config-managed/trusted-ca-bundle still contains only the system bundle:

$ oc get cm/trusted-ca-bundle  -n openshift-config-managed -o yaml
apiVersion: v1
data:
  ca-bundle.crt: |
    # ACCVRAIZ1
    -----BEGIN CERTIFICATE-----
<SNIP>
    -----END CERTIFICATE-----
kind: ConfigMap
metadata:
  creationTimestamp: "2019-08-19T22:24:03Z"
  name: trusted-ca-bundle
  namespace: openshift-config-managed
  resourceVersion: "61097"
  selfLink: /api/v1/namespaces/openshift-config-managed/configmaps/trusted-ca-bundle
  uid: 0c97a54d-c2d0-11e9-bf2d-02b46012d9c0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment