Welcome! This is a series of quick guides to setting up dev/test/lab environment with Tanzu Kubernetes Grid and/or with VMware Telco Cloud Automation.
This isn't a replacement for the official documentation but rather is a curated, streamlined set of "how tos" from several locations based on my experiences.
This is a distillation of the official docs.
/etc/systemd/system/docker.service.d/http-proxy.conf for setting proxy environment variables
[Service]
Environment="HTTP_PROXY=http://192.168.1.5:8118"
Environment="HTTPS_PROXY=http://192.168.1.5:8118"
Environment="NO_PROXY=harbor.bigco.lab,127.0.0.1,localhost,192.168.1.0/24,172.17.0.0/16,172.18.0.0/16,cluster.local"
~/.docker/config.json
{
"proxies": {
"default": {
"httpProxy": "http://192.168.1.5:8118",
"httpsProxy": "http://192.168.1.5:8118",
"noProxy": "harbor.bigco.lab,127.0.0.1,localhost,192.168.1.0/24,172.17.0.0/16,172.18.0.0/16,cluster.local"
}
}
}
tar zxvf tanzu-cli-bundle-v1.3.0-linux-amd64.tar (or similar filename)gunzip kubectl-linux-v1.20.5-vmware.1.gz (or similar filename)mv kubectl-linux-v1.20.5-vmware.1 /usr/local/bin/kubectlchmod +x /usr/local/bin/kubectltanzu/cli folder unpacked from the Tanzu CLI and install the binary
sudo install core/v1.3.0/tanzu-core-linux_amd64 /usr/local/bin/tanzutanzu plugin cleantanzu plugin install --local cli alltanzu plugin listNAME LATEST VERSION DESCRIPTION REPOSITORY VERSION STATUS
cluster v1.3.1 Kubernetes cluster operations core v1.3.1 installed
login v1.3.1 Login to the platform core v1.3.1 installed
pinniped-auth v1.3.1 Pinniped authentication operations (usually not directly invoked) core v1.3.1 installed
kubernetes-release v1.3.1 Kubernetes release operations core v1.3.1 installed
management-cluster v1.3.1 Kubernetes management cluster operations tkg v1.3.1 installed
gunzip ytt-linux-amd64-v0.31.0+vmware.1.gz (or similar filename)sudo mv ./ytt-linux-amd64-v0.31.0+vmware.1 /usr/local/bin/yttsudo chmod +x /usr/local/bin/yttgunzip kapp-linux-amd64-v0.36.0+vmware.1.gz (or similar filename)sudo mv ./kapp-linux-amd64-v0.36.0+vmware.1 /usr/local/bin/kappsudo chmod +x /usr/local/bin/kappgunzip kbld-linux-amd64-v0.28.0+vmware.1.gz (or similar filename)sudo mv ./kbld-linux-amd64-v0.28.0+vmware.1 /usr/local/bin/kbldsudo chmod +x /usr/local/bin/kbldgunzip imgpkg-linux-amd64-v0.5.0+vmware.1.gz (or similar filename)sudo mv ./imgpkg-linux-amd64-v0.5.0+vmware.1 /usr/local/bin/imgpkgsudo chmod +x /usr/local/bin/imgpkgsudo mv yq_linux_amd64 /usr/local/bin/yqsudo chmod +x /usr/local/bin/yq/etc/dnsmasq.conf
listen-address=192.168.1.5 (replace with your jumpbox network address)listen-address=127.0.0.1 (seperate line)dhcp-range=192.168.1.50,192.168.1.180,12h (change to the range of DHCP addresses you want dnsmasq to manage)dhcp-host=00:50:56:ab:6d:db,192.168.1.57 (MAC address, IP address)sudo systemctl restart dnsmasqTanzu includes the "Essentials" version of NSX ALB which allows for Layer 4 (TCP/UDP) Kubernetes Service Type Load Balancers only. This is an active/passive similar-to-VRRP HA load balancer.
There's also the "Basic" version of Avi if you are an NSX-T customer which also allows for Ingress controllers (Layer 7 HTTP, virtual host routing, on top of a TCP Service Type Load Balalncer) implemented and managed by Avi.
Full NSX ALB Enterprise supports multi-cloud, multi-cluster controllers, BGP ECMP scale-out load balancers, multi-cluster GSLB, etc.
Step by Step Guide (This all can be automated via API too!)
admin accountrNone for SMTP configuration, then -> NextOrchestrator Integration, select VMwareSDN integration select None, then -> NextSystem IP Address Management, select DHCP. This assumes that dynamically created VMs for service engines will be assigned management IPs via a DHCP server on their subnet.Virtual Service Placement Settings leave both boxes unchecked, then -> NextDHCP, and then -> NextSupport Multiple Tenants, Select NoIPAM ProfileType Should be Avi Vantage IPAM, Leave Allocate IP in VRF unchecked.Add Usable Network, Select Default-Cloud, for Usable Network, select the Management VDS portgroupSaveDefault-Cloud, and select the IPAM profile in the drop down that we created in steps 13-16.Name and Common Name boxes. Select Self-Signed. For Subject Alternative Name, enter the IP address of the avi controller VM, then SaveExport icon so we can import this CA self-signed cert into TKG later.System Access SettingsSSL/TLS Certifcate Drop down menu to add the newly created custom certificate.Note this assumes HTTP Proxy access to the internet for image downloads; airgap setup is below
Working on a distallation of these, but for now, here are the official docs:
Assumes that most CNF deployers / overall network managers aren't Kubernetes experts so provides a GUI for knowing the status/health and instantiation of CNFs
Also dramatically simplifies the CI/CD required to manage an entire global rollout of CNFs across domains & clusters - CI/CD like Concourse is great and necessary but should be glue to orchestrate-the-orchestrators, rather than the platform that does everything itself (which never works).
Most CNFs are just Helm charts wrapped with extra files/metadata in a ZIP file called a CSAR (Cloud Service Archive). The main metadata is a TOSCA YAML file, which describes the Helm chart and any VM/OS customizations. TOSCA and CSAR are an OASIS standard that was adopted by ETSI for VNF packaging, and being repurposed for CNF & Kubernetes. You can't really do this kind of node specs/customization solely with Helm/K8s beyond some DaemonSet hackery, this standard feels more clean/standardized though likely isn't the endgame in this space. See https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/004/02.05.01_60/gs_nfv-sol004v020501p.pdf
Some VM customization examples:
Some OS configuration examples
For each existing vCenter you want to deploy Tanzu clusters on, you need a TCA CP appliance. Similar to the above:
tar zxvf harbor-offline-installer-v2.3.0.tgz in the directory you want it inopenssl genrsa -out ca.key 4096 to generate the CA keyopenssl req -x509 -new -nodes -sha512 -days 3650 -subj "/C=CA/L=Toronto/O=bigco/OU=lab/CN=harborCA" -key ca.key -out ca.crt to generate the CA cert, change the values to your preferencesopenssl genrsa -out harbor.bigo.lab.key 4096 to generate the server keyopenssl req -sha512 -new -subj "/C=CA/L=Toronto/O=bigco/OU=Lab/CN=harbor.bigo.lab" -key yourdomain.com.key -out yourdomain.com.csr to generate the server cert and CSR, edit the CN to your desired FQDN for harbor| Requirements | |
Requirements
Requirements
Requirements