Skip to content

Instantly share code, notes, and snippets.

@ptesny
Last active July 5, 2024 08:59
Show Gist options
  • Save ptesny/b07267e5966b382f6cef994dd97b5e17 to your computer and use it in GitHub Desktop.
Save ptesny/b07267e5966b382f6cef994dd97b5e17 to your computer and use it in GitHub Desktop.
Getting on the kubernetes ladder with SAP BTP.

Assuming you are neither afraid of heights, nor of any wobbly steps, this brief is about getting you on the kubernetes ladder with SAP BTP, Kyma runtime.

To start with, one needs to a have a BTP Global Account (a commercial contract).

There is a number of ways of getting access to a BTP global account. And, as always, the choice is yours.

For this brief, I have opted for a Pay-As-You-Go SAP BTP free-tier contract.
This way, one can benefit from a number of always free services alongside any freemium and chargeable services.

Alternatively, one might opt for a BTP trial account (a trial is limited to a maximum of 90 days).

Putting it all together

SAP BTP runtime is a cloud native environment where your applications will be deployed to.

SAP BTP features CloudFoundry runtime and Kyma, a kubernetes-based managed runtime environment.

A good read on both environments:

Table of Contents
  1. entitle Kyma runtime environment plans from BTP cockpit.
    1. create a subaccount.
    2. add relevant service plans to your subaccount entitlements.
    3. provision Kyma runtime environment.
  2. Kyma Environment.
    1. default kubeconfig for the provisioned kyma cluster.
    2. service account based kubeconfig for the provisioned kyma cluster.
    3. namespaced service account based kubeconfig.
  3. Custom domains with SAP Kyma clusters
  4. Protect SAP Kyma workloads
  5. Provision SAP HANA Cloud database instance.
  6. Sharing services between CF and Kyma/Kubernetes in BTP
  7. Deploy multi-tenant applications

1. entitle Kyma runtime environment plans from BTP cockpit.

Assuming one has gained access to the BTP Cockpit and can access the global account there.

Goto Entitlements (on a global account level):

1.1 Create a subaccount.

Initially, a BTP Free-Tier global account does not have any sub-accounts created yet.
In order to able to provision environments and services one needs to create at least one sub-account.

Let's create a BTP sub-account in the us10 region (data center) as follows:

 

1.1 Add relevant service plans to your subaccount entitlements.

Goto Entitlements and add at minima the following ones:

1.2 provision Kyma runtime environment.

Next, goto Service Marketplace and select Create to enable the SAP BTP, Kyma runtime environment in the context of your BTP sub-account.

Give your service instance a readable name and proceed to the next step. One can either go with the default setting or try to have a kyma cluster in the region of choice rather, with either the set of the default modules or providing a list of zero or more modules.

Last but not least the networking range for the worker nodes can be customized.

Likewise, one can opt for using their own OIDC provider with the kyma cluster instead of the SAP-managed one.
Please refer to Configure Custom SAP IAS tenant with SAP BTP Kyma runtime environment | SAP blogs for a ready made recipe.

The choice of regions for a kyma cluster.

Please note the below choice of a kyma cluster region will impact the location of both the control plane and the worker node(s).

 

Good to know:

  • With the freemium kyma environment (one-off, and limited to merely 30 days) a cluster is limited to a single worker node. Thus, the freemium kyma clusters are only suitable as sandbox playgrounds for educational purposes.
  • If one required a permanent low-cost kyma cluster there is the azure-lite TDD (Test,Demo,Development) plan reserved for SAP Partners in the selected BTP regions.

The choice of modules:

I decided to leave the nodes networking settings unchanged and opted for a kyma default OIDC provider.
{
    "name": "rth-us10",
    "region": "eu-west-2",
    "networking": {
        "nodes": "10.250.0.0/22"
    },
    "oidc": {
        "clientID": "",
        "groupsClaim": "",
        "issuerURL": "",
        "signingAlgs": [],
        "usernameClaim": "",
        "usernamePrefix": ""
    },
    "administrators": [],
    "modules": {
        "list": [
            {
                "name": "api-gateway",
                "channel": "regular"
            },
            {
                "name": "istio",
                "channel": "regular"
            },
            {
                "name": "btp-operator",
                "channel": "regular"
            }
        ]
    }
}

Good to know:

  • it is a good practice to keep the json-formatted provisioning parameters aside for future reference, especially if anything went wrong at this stage.
  • after provisioning, you can still change the OIDC provider (back and forth) and/or amend the list of cluster administrators

Eventually, one is good to go now. The provisioning process should not take longer than half-an-hour or so.

After you had a coffee or two and if this is still spinning on the screen, just refresh the browser tab and you should be able to see your kyma runtime environment dully provisioned. Hooray!

Goto Dashboard and voila your cluster with the three modules provisioned

Let us add two more modules using the above UI, namely: press Modify then Edit and select new or deselect existing modules and press Update.

Please be patient as the update may take several minutes to complete.

2. Kyma runtime environment.

It's time to do something useful with the newly provisioned kyma environment.
The first thing first is to gain access to a kyma cluster.
A kubeconfig is a kubernetes way of getting access to a cluster. Both, as named and technical users.

2.1 default kubeconfig.

A default kubeconfig is OIDC based, named users only access. How does it work? During the provisioning we provided a list of email addresses of the designated kyma cluster administrators. Assuming, we opted for a kyma default OIDC provider, every email address must have been registered with the SAP Universal ID (account.sap.com) and SAP ID (accounts.sap.com)

Please refer to this description of the default kubeconfig for more information.

Alternatively, the below make target is a convenient way of downloading a kyma cluster kubeconfig directly from a command line leveraging both btp cli and jq.

.PHONY: download-kubeconfig
download-kubeconfig: ## download-kubeconfig
	curl -o kubeconfig.yaml $$(btp list accounts/environment-instance | jq -r '.environmentInstances[] | select(.serviceName == "kymaruntime") | .labels | fromjson | .KubeconfigURL '  )

2.2 service account based kubeconfig.

The default OIDC-based kubeconfig is not well suited for unmanned operations for instance with github actions, devops pipelines or deployment tools.

Service accounts are namespaced resources. Every single namespace has an existing service account called default.

Thus, let's re-use an existing default service account from the default namespace for the purpose of creating a sa-based kubeconfig.

a. ClusterRoleBinding

  • clusteradmin-role-binding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: default-sa-cluster-scope
subjects:
  - kind: ServiceAccount
    name: default
    namespace: default
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

b. Create a secret with a never expiring token. That's not something you would be doing in your production cluster(s)

  • default-token-sa-secret.yaml
apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
  annotations:
    kubernetes.io/service-account.name: default
  name: default-token-sa
  namespace: default
  labels:
    app.kubernetes.io/name: default-token-sa
data: {}

c. update service account manifest - this is to ensure the above secret is linked with the service account

  • update-default-token-sa-with-secret.yaml
kind: ServiceAccount
apiVersion: v1
metadata:
  name: default
  namespace: default

secrets:
  - name: default-token-sa
automountServiceAccountToken: false

d. kubectl-view_serviceaccount_kubeconfig - view and write down a sa kubeconfig:

kubectl-view_serviceaccount_kubeconfig default -n default --kubeconfig ~/.kube/kubeconfig-shoot--kyma--c-***.yaml

kubectl-view_serviceaccount_kubeconfig default -n default --kubeconfig ~/.kube/kubeconfig-shoot--kyma--c-***.yaml >  ~/.kube/kubeconfig--c-***.yaml

chmod go-r ~/.kube/kubeconfig--c-***.yaml

cluster-default-sa-deploy automation

.PHONY: cluster-default-sa-deploy
cluster-default-sa-deploy: ## cluster-default-sa-deploy
	kubectl create --kubeconfig $(KUBECONFIG) -f clusteradmin-role-binding.yaml --dry-run=client -o yaml | kubectl apply --kubeconfig $(KUBECONFIG) -f -
		kubectl create --kubeconfig $(KUBECONFIG) -f default-token-sa-secret.yaml --dry-run=client -o yaml | kubectl apply --kubeconfig $(KUBECONFIG) -f -
	kubectl create --kubeconfig $(KUBECONFIG) -f update-default-token-sa-with-secret.yaml --dry-run=client -o yaml | kubectl apply --kubeconfig $(KUBECONFIG) -f -


.PHONY: cluster-kubeconfig
cluster-kubeconfig: ## create cluster-kubeconfig with the default namespace service account
	kubectl-view_serviceaccount_kubeconfig default -n default --kubeconfig $(KUBECONFIG) > ~/.kube/kubeconfig--$(CLUSTER_SHOOT_NAME)-default.yaml
	chmod go-r  ~/.kube/kubeconfig--$(CLUSTER_SHOOT_NAME)-default.yaml

.PHONY: view-cluster-kubeconfig
view-cluster-kubeconfig: ## view-cluster-kubeconfig
	kubectl-view_serviceaccount_kubeconfig  default -n default --kubeconfig $(KUBECONFIG)

2.3 namespaced service account based kubeconfig.

3. Custom domains with SAP Kyma.

domain type description
public custom domain Public custom domains easy with SAP BTP, Kyma runtime.
kyma system domain for mtls https://blogs.sap.com/2023/12/27/developing-enterprise-grade-applications-with-mutual-tls-easy-with-sap-btp-kyma-runtime-and-btp-destinations/#kyma-system-mtls
kyma custom domain for mtls https://blogs.sap.com/2023/12/27/developing-enterprise-grade-applications-with-mutual-tls-easy-with-sap-btp-kyma-runtime-and-btp-destinations/#kyma-custom-mtls

4. Protect SAP Kyma workloads.

SAP Kyma is an internet facing cloud native runtime environment. Thus, it is paramount to understand how kyma/kuberntes native workloads can be protected from eavesdropping.

method description
with mutual TLS Developing enterprise-grade applications with mutual TLS easy with SAP BTP, Kyma runtime and BTP destinations
with JWT Protect your Kyma workloads from eavesdropping with JWT access strategy and BTP destinations.
with introspection Protect your Kyma workloads from eavesdropping with introspection and BTP destinations

5. Provision SAP HANA Cloud database instance.

Let's look up SAP HANA Cloud service plans from the service marketplace:

Next step is the subscribe to SAP HANA Cloud tools (aka SAP HANA Cloud Central cockpit).
SAP HANA Cloud graphical tools allow to create and manage SAP HANA Cloud instances.
No additional costs are charged for using the SAP HANA Cloud graphical tools.

.PHONY: bootstrap-hana-cloud-tools2
bootstrap-hana-cloud-tools2: ## bootstrap hana cloud tools
	btp list accounts/subscription | jq -r '.applications[] | select (.state == "NOT_SUBSCRIBED" and .commercialAppName == "hana-cloud-tools") | .subscribedTenantId ' > subscribedTenantId.txt
	#btp list accounts/subscription | jq -r '.applications[] | select (.state == "NOT_SUBSCRIBED" and .commercialAppName == "hana-cloud-tools") '
	btp subscribe accounts/subaccount --to-app hana-cloud-tools --plan tools
	btp list accounts/subscription | jq -r '.applications[] | select (.state == "SUBSCRIBED" and .commercialAppName == "hana-cloud-tools") | .subscribedTenantId ' > subscribedTenantId.txt
	btp assign security/role-collection "SAP HANA Cloud Administrator" --to-user "$$USERNAME1" --create-user-if-missing
	btp assign security/role-collection "SAP HANA Cloud Security Administrator" --to-user "$$USERNAME1" --create-user-if-missing

.PHONY: delete-hana-cloud-tools2
delete-hana-cloud-tools2: ## delete hana cloud tools
	btp list accounts/subscription | jq -r '.applications[] | select (.state == "SUBSCRIBED" and .commercialAppName == "hana-cloud-tools") | .subscribedTenantId ' > subscribedTenantId.txt
	btp unsubscribe accounts/subaccount --from-app hana-cloud-tools --confirm --subaccount $$(cat subscribedTenantId.txt)

alternatively, let's use a bash script in a makefile target, namely:

.PHONY: bootstrap-hana-cloud-tools
bootstrap-hana-cloud-tools: ## bootstrap hana cloud tools (aubscription)
	while read -r state ; do \
	      if [ "$$state" = "NOT_SUBSCRIBED" ] ;\
	      then \
				read -r subscribedSubaccountId && read -r category; \
				echo "state: $$state"; \
				echo "subscribedSubaccountId: $$subscribedSubaccountId "; \
				echo "category: $$category "; \
				echo "### bootstrap-hana-cloud-tools ###" ;\
				btp subscribe accounts/subaccount --to-app hana-cloud-tools --plan tools ;\
				sleep 2; \
				btp assign security/role-collection "SAP HANA Cloud Administrator" --to-user "$$USERNAME1" --create-user-if-missing ;\
				btp assign security/role-collection "SAP HANA Cloud Security Administrator" --to-user "$$USERNAME1" --create-user-if-missing ;\
				break ; \
	      fi ;\
		done <<< "$$(btp get accounts/subscription --of-app hana-cloud-tools --plan tools | jq -r '. | .state, .subscribedSubaccountId, .category ' )"


.PHONY: delete-hana-cloud-tools
delete-hana-cloud-tools: ## delete hana cloud free tier plan
	while read -r state ; do \
	      if [ "$$state" = "SUBSCRIBED" ] ;\
	      then \
	        read -r subscribedSubaccountId && read -r category; \
	        echo "state: $$state"; \
	        echo "subscribedSubaccountId: $$subscribedSubaccountId "; \
	        echo "category: $$category "; \
	        echo "### delete-hana-cloud-tools ###" ;\
	        btp unsubscribe accounts/subaccount --from-app hana-cloud-tools --confirm --subaccount $$subscribedSubaccountId ;\
	        break ; \
	      fi ;\
		done <<< "$$(btp get accounts/subscription --of-app hana-cloud-tools --plan tools | jq -r '. | .state, .subscribedSubaccountId, .category ' )"

The advantage being while read -r state ; do loop will execute only once and allows to test whether hana-cloud-tools subscription exists or not in the context of a given subaccount.

The below command will create a hana cloud service instance called hc-3tier:


.PHONY: bootstrap-hana-cloud-free
bootstrap-hana-cloud-free: ## bootstrap hana cloud free tier plan
	#$$(btp create services/instance --offering-name hana-cloud --plan-name hana-free --name hc-3tier --parameters hc-3tier.json | jq  -r '. | .command' )
	btp create services/instance --offering-name hana-cloud --plan-name hana-free --name hc-3tier --parameters hc-3tier.json | jq '. ' > hc-command.json
	echo | $$(jq  -r '. | .command' hc-command.json) | jq -r '. | .dashboard_url '
	echo | $$(jq  -r '. | .command' hc-command.json) | jq -r '. | .ready '	
	#curl -L -v  $$(echo | $$(jq  -r '. | .command' hc-command.json) | jq -r '. | .dashboard_url ')


.PHONY: delete-hana-cloud-free
delete-hana-cloud-free: ## delete hana cloud free service instance
	btp get services/instance --name hc-3tier | jq  -r '. | .ready' 
	btp delete services/instance --name hc-3tier --confirm | jq  -r '. | .command' 

	
.PHONY: wait-hana-cloud-free
wait-hana-cloud-free: ## wait hana cloud free tier plan
	until [ "$$(btp get services/instance --name hc-3tier | jq  -r '. | .ready' )"  = "true" ] ; \
	do \
	  echo "bootstrap-hana-cloud-free is running......"  ;\
	  sleep 10 ; \
	  if [ "$$(btp get services/instance --name hc-3tier | jq  -r '. | .ready' )"  = "false" ] ; \
	  then \
	    echo "hana-cloud-free not ready yet. please continue waiting..." ; \
	  fi  ; \
	done 


.PHONY: bootstrap-hana-cloud-free
bootstrap-hana-cloud-free: ## bootstrap hana cloud free tier plan
	#$$(btp create services/instance --offering-name hana-cloud --plan-name hana-free --name hc-3tier --parameters hc-3tier.json | jq  -r '. | .command' )
	btp create services/instance --offering-name hana-cloud --plan-name hana-free --name hc-3tier --parameters hc-3tier.json | jq '. ' > hc-command.json
	echo | $$(jq  -r '. | .command' hc-command.json) | jq -r '. | .dashboard_url '
	echo | $$(jq  -r '. | .command' hc-command.json) | jq -r '. | .ready '	
	#curl -L -v  $$(echo | $$(jq  -r '. | .command' hc-command.json) | jq -r '. | .dashboard_url ')
	until [ "$$(btp get services/instance --name hc-3tier | jq  -r '. | .ready' )"  = "true" ] ; \
	do \
	  echo "bootstrap-hana-cloud-free is running......"  ;\
	  sleep 10 ; \
	  if [ "$$(btp get services/instance --name hc-3tier | jq  -r '. | .ready' )"  = "false" ] ; \
	  then \
	    echo "hana-cloud-free not ready yet. please continue waiting..." ; \
	  fi  ; \
	done 


.PHONY: delete-hana-cloud-free
delete-hana-cloud-free: ## delete hana cloud free service instance
	btp delete services/instance --name hc-3tier --confirm | jq  '. | .command' > hc-command.json
	while [ "$$(btp get services/instance --name hc-3tier | jq  -r '. | .ready' )"  = "true" ] ; \
	do \
	  echo "delete-hana-cloud-free is running......"  ;\
	  $$(jq  -r '. | .command' hc-command.json) | jq -r '. | .ready ' ; \
	  if [ "$$(btp get services/instance --name hc-3tier | jq  -r '. | .ready' )"  = "true" ] ; \
	  then \
	    echo "deletion in progress. please continue waiting..." ; \
	  fi  ; \
	  sleep 1 ; \
	done 
  • view from SAP HANA Cloud Central cockpit

  • view from a BTP subaccount
image

6. Sharing services between CF and Kyma/Kubernetes in BTP.

7. Deploy multi-tenant applications

  • ArgoCDaaS
  • SAP Approuter
  • SAP Build Apps
 

Additional resources

Sharing services between CF and Kyma/Kubernetes in BTP
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment