timmyreilly/service_connection_instructions.md

## service_connection_instructions.md

      
    Raw
  

              service_connection_instructions.md
            
          
    v 1.0
Setting up a service account for azure devops and kubernetes

Service accounts in Kubernetes allow you to enforce RBAC for all Kubernetes resources in your cluster.
Service connections in Azure Devops allow you to use RBAC policies for infrastructure, including Kubernetes clusters.
Asserting RBAC on all systems that have an associated cost to operate is a great start to keeping costs under control.
Service accounts are neat, they allow processes impersonate a user and do things. Kinda like a computer/av system in a conference room. ie: you send a meeting request to an email account to include the conference in your meeting. It has the side effect of keeping people out of the room, but it needs a reference in the system so that it can be addressed accurately. Anyways... let's say we want to accurately address a computer capable of sending 10000000 pods to the cluster. That might get expensive if it can do it willy-nilly and anything with costs associated shouldn't be willy-nilly.
SO. To enforce these ideas we create service accounts for computers that can interact with the cluster.
Follow along at home:


Keep this handy + Follow the autocomplete setup

Get your bearings

k get serviceaccount -A
k get clusterrolebinding -A
k get rolebinding -A
k get role -A
k get clusterrole -A
k config view
kubectl api-resources

Start Creating

k create serviceaccount keda-agent1-sa
k create namespace keda-dev-1

Validate

k get serviceaccount -A
k describe serviceaccount
k describe namespace keda-dev-1

Now let's get more specific,

For these scenarios the serviceaccount needs to be capable of managing an "api-resources typed interface" in a specific namespace.
The reason we're doing this is so we can provide least priveledge to our devops agent.
To build the policies we bind the service account with a role using a role binding.
Here' an example chart to demonstrate this scenario.
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: demo-sa
  namespace: default
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
 name: list-pods
 namespace: default
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["list", "get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
 name: list-pods_demo-sa
 namespace: default
roleRef:
 kind: Role
 name: list-pods
 apiGroup: rbac.authorization.k8s.io
subjects:
  - kind: ServiceAccount
    name: demo-sa
    namespace: default
---
And heres a pod that will be helpful for debugging:
---
apiVersion: v1
kind: Pod
metadata:
 name: pod-demo-sa
spec:
 serviceAccountName: demo-sa
 containers:
 - name: alpine
   image: alpine:3.9
   command:
   - "sleep"
   - "10000"
Or you could do it from the command line:
kubectl api-resources
Insert api of choice, in this case its scaledobjects from the keda api.
k create role scaledobjects-admin-role -n default --verb=get --verb=update --verb=list --resource=scaledobjects.keda.sh
k create serviceaccount scaledobjectuser -n default
k create rolebinding scaleobjectsadminbinding -n default --role="scaledobjects-admin-role" --serviceaccount="default:scaledobjectuser"
k get so --as="system:serviceaccount:default:scaledobjectuser"
This one shouldn't work:
k get so --as="system:serviceaccount:default:scaledobjectuser" -n test
Alright we've got the basics!
Feel free to clean up after yourself:

k delete namespace test
k delete namespace keda-dev-1
k delete role scaledobjects-admin-role
k delete rolebinding scaledobjectsadminbinding
k delete serviceaccount scaledobjectuser -n default

Or just use charts:

k delete -f kubernetes/devops_service_account/chart.yaml


Using Service Accounts with Devops:

Now we have the basics with service accounts, lets add integration with devops.
To use a service account in Azure Devops we create a service connection of Kubernetes type and reference it in our pipelines:
A login snippet of a pipeline referencing a kubernetes service connection looks like:
    - task: Kubernetes@1
      displayName: 'kubectl login'
      inputs:
        connectionType: 'Kubernetes Service Connection'
        kubernetesServiceEndpoint: azdo-sa-dev-1-ns
        azureResourceGroup: $(azureResourceGroup)
        kubernetesCluster: $(kubernetesCluster)
        namespace: default
        command: login
To create a service account and follow along the demo you can use the chart here: kubernetes/devops_service_account/sa_chart_alpha.yaml (eg: k apply -f kubernetes/devops_service_account/sa_chart_alpha.yaml)
Now to get that to work as we expect we need to create azdo-sa-dev-1-ns service connection. Go to your Azure DevOps Organization > Project > Project Settings > Service Connections > New Service Connection.
Hint: This is one of my favorite links for getting to the root of your Azdo orgs: https://aex.dev.azure.com/me?mkt=en-US

Then select the kubernetes type and select next, we have three options here and service account is going to allow us a good balance of security and rbac compliance.
We can follow the instructions provided by wizard for accessing the server url and secret token information for the service account. See image below for what the looked like on my side.

The commands I used to get the json to paste in the secret section:
$ k get serviceaccount -n dev-1
$ k config view --minify -o jsonpath={.clusters[0].cluster.server}
$ k get service
$ k get serviceAccounts devops-sa -n dev-1 -o=jsonpath={.secrets[*].name}
$ k get secret devops-sa-token-f6wnk -n dev-1 -o json

Copy the entire contents of the last command into the authorization box and give the service connection name something specific and use the description to help users understand the privileges of the service connection/service account. Then allow access to all pipelines and click save.
Great! We have a service connection ready for our Azure Pipelines! Let's test it out!
One of the great things about using a service account in the pipeline is we can start interacting with the cluster as if we're the pipeline from our own machines! No need to push changes to DevOps every time! To start interacting with our cluster as if we're the service account we can append --as="system:serviceaccount:default:scaledobjectuser" to all our commands... or we can build a new context using the service account secret.
To build our context as our azdev-sa:
$ TOKEN=$(kubectl describe secrets "$(kubectl describe serviceaccount devops-sa -n dev-1| grep -i Tokens | awk '{print $2}')" -n dev-1 | grep token: | awk '{print $2}')

$ kubectl config set-credentials azdev-user --token=$TOKEN

$ kubectl config set-context azdev-simulation --cluster=ti-aks-1 --user=azdev-user

$ kubectl config use-context azdev-simulation 

# And verify: 
$ k config get-contexts

Now everything you do is under the same RBAC policies... let's try some stuff.
Working:
$ k get pods -n dev-1 

Not Working Error from server (Forbidden):
$ k get secrets -n dev-1

Now let's add increase the scope of permissions for our Service Account and reconcile the changes:
First, change the context back to a clusteradmin: k config get-contexts to find the original context we were working in. Should be the cluster name: k config use-context ti-aks-1. Then apply the changes:
$ k apply -f kubernetes/devops_service_account/sa_chart_alpha.yaml
$ k auth reconcile -f kubernetes/devops_service_account/sa_chart_alpha.yaml 
And switch back and see the increased scope of the service account tied to the service connection:
kubectl config use-context azdev-simulation
$ k get pods -n dev-1 
$ k get secrets -n dev-1
Still doesn't work because we're confined to the namespace dev-1
$ k get pods 

Appendix (exercises):

To validate... ClusterRole + RoleBinding

or everything but read secrets from these specific namespaces
or delete stateful objects:
TODO: Figure out the permission types
Now validate it has what it needs to act as a pipeline agent:
Get information about the available roles, and find the appropriate role:
k get serviceaccount -n test
k get clusterrole/cluster-admin -o yaml
k get rolebinding -n test -o yaml
k get role/testadmin -n test -o yaml
k auth can-i get customresourcedefinitions --as=system:serviceaccount:default:azdev-sa-87b26d
Now, go use this service account in the azure devops:
Here's an example of a chart that looks just like a script but works with kubectl, helm

script: |
    helm install 


and one that uses the AzureDevop Tasks:


Now that you have the pipeline setup you can run the pipelines with ease.
The pipelines are capable of adding functions, upgrades to the orchestrator or other new services.
kubectl get serviceAccounts default -n default -o=jsonpath={.secrets[*].name}
k get clusterrolebinding
k config get-contexts
k config use-context ti-aks-1
Troubleshooting:

Extracting a token from a serviceaccount for debugging:

TOKEN=$(kubectl describe secrets "$(kubectl describe serviceaccount def-pod-reader -n default| grep -i Tokens | awk '{print $2}')" -n default | grep token: | awk '{print $2}')

kubectl config set-credentials test-user --token=$TOKEN
kubectl config set-context udefpodreader --cluster=ti-aks-1 --user=test-user
kubectl config use-context udefpodreader
k config get-contexts
Try to do it from in unauthenticated serviceaccount

k get serviceaccount -n test

This is what you need:
k auth can-i list pods --namespace default --as="system:serviceaccount:default:demo-sa"
Authenticate with aks:


az account set --subscription SECRET-21c3-4f7e-8329-625443da4254


az aks get-credentials --resource-group az-kubernetes-1 --name ti-aks-1


k config view


Other snippets


Still need to validate if these work ^
Next steps:


record demo!
If you wanna learn more about building quotas ping me.

v 0.2
Setting up a service account for azure devops and kubernetes

Service accounts in Kubernetes allow you to enforce RBAC for all Kubernetes resources in your cluster.
Service connections in Azure Devops allow you to use RBAC policies for infrastructure, including Kubernetes clusters.
Asserting RBAC on all systems that have an associated cost to operate is a great start to keeping costs under control.
Service accounts are neat, they allow processes impersonate a user and do things. Kinda like a computer/av system in a conference room. ie: you send a meeting request to an email account to include the conference in your meeting. It has the side effect of keeping people out of the room, but it needs a reference in the system so that it can be addressed accurately. Anyways... let's say we want to accurately address a computer capable of sending 10000000 pods to the cluster. That might get expensive if it can do it willy-nilly and anything with costs associated shouldn't be willy-nilly.
SO. To enforce these ideas we create service accounts for computers that can interact with the cluster.
Follow along at home:


Keep this handy + Follow the autocomplete setup

Get your bearings

k get serviceaccount -A
k get clusterrolebinding -A
k get rolebinding -A
k get role -A
k get clusterrole -A
k config view
kubectl api-resources

Start Creating

k create serviceaccount keda-agent1-sa
k create namespace keda-dev-1

Validate

k get serviceaccount -A
k describe serviceaccount
k describe namespace keda-dev-1

Now let's get more specific,

For these scenarios the serviceaccount needs to be capable of managing an "api-resources typed interface" in a specific namespace.
The reason we're doing this is so we can provide least priveledge to our devops agent.
To build the policies we bind the service account with a role using a role binding.
Here' an example chart to demonstrate this scenario.
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: demo-sa
  namespace: default
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
 name: list-pods
 namespace: default
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["list", "get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
 name: list-pods_demo-sa
 namespace: default
roleRef:
 kind: Role
 name: list-pods
 apiGroup: rbac.authorization.k8s.io
subjects:
  - kind: ServiceAccount
    name: demo-sa
    namespace: default
---
And heres a pod that will be helpful for debugging:
---
apiVersion: v1
kind: Pod
metadata:
 name: pod-demo-sa
spec:
 serviceAccountName: demo-sa
 containers:
 - name: alpine
   image: alpine:3.9
   command:
   - "sleep"
   - "10000"
Or you could do it from the command line:
kubectl api-resources
Insert api of choice, in this case its scaledobjects from the keda api.
k create role scaledobjects-admin-role -n default --verb=get --verb=update --verb=list --resource=scaledobjects.keda.sh
k create serviceaccount scaledobjectuser -n default
k create rolebinding scaleobjectsadminbinding -n default --role="scaledobjects-admin-role" --serviceaccount="default:scaledobjectuser"
k get so --as="system:serviceaccount:default:scaledobjectuser"
This one shouldn't work:
k get so --as="system:serviceaccount:default:scaledobjectuser" -n test
Alright we've got the basics!
Feel free to clean up after yourself:

k delete namespace test
k delete namespace keda-dev-1
k delete role scaledobjects-admin-role
k delete rolebinding scaledobjectsadminbinding
k delete serviceaccount scaledobjectuser -n default

Or just use charts:

k delete -f kubernetes/service-account/Chart.yaml


Appendix (execrises):

To validate... ClusterRole + RoleBinding

or everything but read secrets from these specific namespaces
or delete stateful objects:
TODO: Figure out the permission types
Now validate it has what it needs to act as a pipeline agent:
Get information about the available roles, and find the appropriate role:
k get serviceaccount -n test
k get clusterrole/cluster-admin -o yaml
k get rolebinding -n test -o yaml
k get role/testadmin -n test -o yaml
k auth can-i get customresourcedefinitions --as=system:serviceaccount:default:azdev-sa-87b26d
Now, go use this service account in the azure devops:
Here's an example of a chart that looks just like a script but works with kubectl, helm

script: |
    helm install 


and one that uses the AzureDevop Tasks:


Now that you have the pipeline setup you can run the pipelines with ease.
The pipelines are capable of adding functions, upgrades to the orchestrator or other new services.
kubectl get serviceAccounts default -n default -o=jsonpath={.secrets[*].name}
k get clusterrolebinding
k config get-contexts
k config use-context ti-aks-1
Troubleshooting:

Extracting a token from a serviceaccount for debugging:

TOKEN=$(kubectl describe secrets "$(kubectl describe serviceaccount def-pod-reader -n default| grep -i Tokens | awk '{print $2}')" -n default | grep token: | awk '{print $2}')

kubectl config set-credentials test-user --token=$TOKEN
kubectl config set-context udefpodreader --cluster=ti-aks-1 --user=test-user
kubectl config use-context udefpodreader
k config get-contexts
Try to do it from in unauthenticated serviceaccount

k get serviceaccount -n test

This is what you need:
k auth can-i list pods --namespace default --as="system:serviceaccount:default:demo-sa"
Authenticate with aks:


az account set --subscription SECRET-21c3-4f7e-8329-625443da4254


az aks get-credentials --resource-group az-kubernetes-1 --name ti-aks-1


k config view


Other snippets


Still need to validate if these work ^
Next steps:


record demo!
If you wanna learn more about building quotas ping me.


V 0.1
Setting up a service account for azure devops and kubernetes

Service accounts in Kubernetes allow you to enforce RBAC for all Kubernetes resources in your cluster.
Service connections in azure allow you to enforce RBAC for all resources including your devops pipeline that interact with your cluster.

first create a service account: k get serviceAccount
then give it permissions to do everything but delete resources in the devtest namespace in the cluster:
Role + RoleBinding k apply -f charts/service_accounts.yaml

Could look like this:
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: demo-sa
  namespace: default
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
 name: list-pods
 namespace: default
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["list", "get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
 name: list-pods_demo-sa
 namespace: default
roleRef:
 kind: Role
 name: list-pods
 apiGroup: rbac.authorization.k8s.io
subjects:
  - kind: ServiceAccount
    name: demo-sa
    namespace: default
---
apiVersion: v1
kind: Pod
metadata:
 name: pod-demo-sa
spec:
 serviceAccountName: demo-sa
 containers:
 - name: alpine
   image: alpine:3.9
   command:
   - "sleep"
   - "10000"

To validate... ClusterRole + RoleBinding

authorize with aks:


az account set --subscription SECRET-21c3-4f7e-8329-625443da4254


az aks get-credentials --resource-group az-kubernetes-1 --name ti-aks-1


k config view


To expedite working with Kubernetes check out: https://kubernetes.io/docs/reference/kubectl/cheatsheet/#bash
or everything but read secrets from these specific namespaces
or delete stateful objects:
TODO: Figure out the permission types
Now validate it has what it needs to act as a pipeline agent:
Get information about the available roles, and find the appropriate role:
k get serviceaccount -n test
$ k get clusterrole/cluster-admin -o yaml
k get rolebinding -n test -o yaml
k get role/testadmin -n test -o yaml
k auth can-i get customresourcedefinitions --as=system:serviceaccount:default:azdev-sa-87b26d
Now, go use this service account in the azure devops:
Here's an example of a chart that looks just like a script but works with kubectl, helm

script: |
    helm install 


and one that uses the AzureDevop Tasks:


Now that you have the pipeline setup you can run the pipelines with ease.
The pipelines are capable of adding functions, upgrades to the orchestrator or other new services.
kubectl get serviceAccounts default -n default -o=jsonpath={.secrets[*].name}
kubectl get secret default-token-pdw5b -n default -o json
kubectl get serviceAccounts azpipeline -n default -o=jsonpath={.secrets[*].name}
azpipeline-token-w5dfc
kubectl get secret azpipeline-token-w5dfc -n default -o json
k get clusterrolebinding
k auth can-i get secrets --as=system:serviceaccount:default:dada
k auth can-i get secrets --as=system:serviceaccount:default:dada
k auth can-i get secrets azdev-sa-87b26d-token-s89c5 -o yaml -as test2
empty-secret
azdev-sa-87b26d
k config get-contexts
k config use-context ti-aks-1
TOKEN=$(kubectl describe secrets "$(kubectl describe serviceaccount def-pod-reader -n default| grep -i Tokens | awk '{print $2}')" -n default | grep token: | awk '{print $2}')

kubectl config set-credentials test-user --token=$TOKEN
kubectl config set-context udefpodreader --cluster=deepqna-aks --user=test-user

This is what you need:
k auth can-i list pods --namespace default --as="system:serviceaccount:default:demo-sa"