Skip to content

Instantly share code, notes, and snippets.

@jpbetz
Last active May 7, 2024 20:04
Show Gist options
  • Save jpbetz/9cc09e7f24e215c63474eb2fd7b20eee to your computer and use it in GitHub Desktop.
Save jpbetz/9cc09e7f24e215c63474eb2fd7b20eee to your computer and use it in GitHub Desktop.
# config for 1 control plane node and 2 workers (necessary for conformance)
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
ipFamily: ipv4
kubeProxyMode: iptables
# don't pass through host search paths
# TODO: possibly a reasonable default in the future for kind ...
dnsSearch: []
nodes:
- role: control-plane
- role: worker
- role: worker
featureGates: {}
runtimeConfig: {}
kubeadmConfigPatches:
- |
kind: ClusterConfiguration
metadata:
name: config
apiServer:
extraArgs:
"v": "4"
controllerManager:
extraArgs:
"v": "4"
scheduler:
extraArgs:
"v": "4"
---
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
"v": "4"
---
kind: JoinConfiguration
nodeRegistration:
kubeletExtraArgs:
"v": "4"
#!/usr/bin/env bash
set -o errexit
set -o nounset
set -o pipefail
# etcd and kubectl are assumed to be in the path already
kube="${KUBE_ROOT}"
workingdir="/tmp/quick-apiserver" # working directory
certdir="${workingdir}/certs"
function help() {
echo $0 "[clean] [build] [run]"
exit 1
}
function generate_apiserver_certs() {
# service account
openssl genrsa -out service-account-key.pem 4096
openssl req -new -x509 -days 365 -key service-account-key.pem -subj "/CN=test" -sha256 -out service-account.pem
cat <<EOF > csr.conf
[ req ]
default_bits = 2048
prompt = no
default_md = sha256
req_extensions = req_ext
distinguished_name = dn
[ dn ]
C = US
ST = MD
L = Odenton
O = Google
OU = Cloud
CN = 127.0.0.1
[ req_ext ]
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster
DNS.5 = kubernetes.default.svc.cluster.local
IP.1 = 127.0.0.1
IP.2 = 127.0.0.1
[ v3_ext ]
authorityKeyIdentifier=keyid,issuer:always
basicConstraints=CA:FALSE
keyUsage=keyEncipherment,dataEncipherment
extendedKeyUsage=serverAuth,clientAuth
subjectAltName=@alt_names""
EOF
# api-server
openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -subj "/CN=test" -days 10000 -out ca.crt
openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr -config csr.conf
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key \
-CAcreateserial -out server.crt -days 10000 \
-extensions v3_ext -extfile csr.conf
}
function generate_kubeconfig() {
kubectl config set-cluster local-apiserver \
--certificate-authority="${certdir}/ca.crt" \
--embed-certs=true \
--server=https://127.0.0.1:6443 \
--kubeconfig=kubeconfig
kubectl config set-credentials admin \
--client-certificate="${certdir}/server.crt" \
--client-key="${certdir}/server.key" \
--embed-certs=true \
--kubeconfig=kubeconfig
kubectl config set-context default \
--cluster=local-apiserver \
--user=admin \
--kubeconfig=kubeconfig
kubectl config use-context default --kubeconfig=kubeconfig
}
if [ "$#" -lt 1 ]; then
help
fi
clean=false
build=false
run=false
for arg in $@; do
case "${arg}" in
clean) clean=true ;;
build) build=true ;;
run) run=true ;;
*) help ;;
esac
done
if [ "${clean}" = true ]; then
rm -rf "${workingdir}"
fi
certlog="${workingdir}/certgen.log"
kubeconfiglog="${workingdir}/kubeconfig.log"
if [ "${build}" = true ]; then
mkdir -p "${workingdir}"
echo "Building kube-apiserver."
go build -C "${kube}/cmd/kube-apiserver" -o "${workingdir}"
if [ ! -d "${certdir}" ]; then
mkdir -p "${certdir}"
pushd "${certdir}" > /dev/null
echo "Generating certs."
generate_apiserver_certs &> "${certlog}"
popd > /dev/null
echo "Generating kubeconfig."
pushd "${workingdir}" > /dev/null
generate_kubeconfig &> "${kubeconfiglog}"
popd > /dev/null
fi
fi
log="${workingdir}/etcd.log"
apiserverlog="${workingdir}/kube-apiserver.log"
if [ "${run}" = true ]; then
if [ ! "${workingdir}" ]; then
echo "${workingdir} not found. Please run with 'build'."
exit 1
fi
echo "Starting etcd."
echo " logs: ${etcdlog}"
echo ""
"${kube}/third_party/etcd/etcd" "--data-dir=${workingdir}" 2> "${etcdlog}" &
echo "Starting kube-apiserver."
echo " logs: ${apiserverlog}"
echo ""
echo "To use kubectl:"
echo " export KUBECONFIG=${workingdir}/kubeconfig"
echo ""
echo "Running. Ctrl-C to stop."
"${workingdir}/kube-apiserver" --etcd-servers http://localhost:2379 \
--service-account-key-file="${certdir}/service-account-key.pem" \
--service-account-signing-key-file="${certdir}/service-account-key.pem" \
--service-account-issuer=api \
--tls-cert-file="${certdir}/server.crt" \
--tls-private-key-file="${certdir}/server.key" \
--client-ca-file="${certdir}/ca.crt" 2> "${apiserverlog}"
fi
@jpbetz
Copy link
Author

jpbetz commented Apr 24, 2024

This works well for when only the control plane and no nodes are needed:

local-up-cp.sh:

export API_BIND_ADDR=127.0.0.1
export ADVERTISE_ADDRESS=127.0.0.1
export KUBE_ENABLE_CLUSTER_DNS=false
export KUBE_CONTROLLERS=*,-nodelifecycle,-podgc
export ENABLE_CLUSTER_DNS=false
export START_MODE=nokubelet,nokubeproxy
local-up-cluster.sh
sudo -E PATH=$PATH local-up-cp.sh
# e2e testing:                                                                                                                                                           
# sudo PATH=$PATH local-up.sh                                                                                                                                            
# cd test/e2e                                                                                                                                                            
# go test -ginkgo.focus "{regex}"

@jpbetz
Copy link
Author

jpbetz commented May 7, 2024

This is for when a test cluster is needed. It is similar to what e2e-kind does:

 kind build node-image -v 1                                                                                                                                               
 kind create cluster --image=kindest/node:latest --retain --wait=1m -v=3 --config="${HOME}/bin/kind-config.yaml"                                                          
 export KUBECONFIG=~/.kube/config                                                                                                                                         
 # e2e testing                                                                                                                                                            
 # make WHAT="vendor/github.com/onsi/ginkgo/v2/ginkgo cmd/kubectl test/e2e/e2e.test"                                                                                      
 # KUBERNETES_CONFORMANCE_TEST='y' ./hack/ginkgo-e2e.sh --ginkgo.focus "{regexp}" '--provider=skeleton'        

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment