Skip to content

Instantly share code, notes, and snippets.

@anapsix

anapsix/k8s-vault

Last active Jan 18, 2021
Embed
What would you like to do?
K8s-Vault, like AWS-Vault, but for cli tools using KUBECONFIG (~/.kube/config), such as helm, kubectl, etc..
#!/usr/bin/env bash
#
# K8s-Vault, like AWS-Vault is a helper for AWS related CLI tools
# is a helper for CLI tools using kubectl config and K8s API.
# Unlike AWS-Vault, vault here is used as a verb,
# synonymous to leap, jump, spring, etc..
# Copyright (C) 2019-2020 Anastas Dancha (aka @anapsix)
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
# Usage examples:
# k8s-vault exec <context-name> -- kubectl get nodes
# k8s-vault exec <context-name> -s
# Example of config file
: <<'EXAMPLE_CONFIG_YAML'
k8s_api_timeout: 5 # in seconds
ssh_ttl: 10 # in seconds
ssh_forwarding_port:
random: true
static_port: 32845
clusters:
prod:
enabled: true
ssh_jump_host: jumphost.prod.example.com
qa:
enabled: true
ssh_jump_host: jumphost.qa.example.com
dev:
enabled: false
ssh_jump_host: jumphost.dev.example.com
EXAMPLE_CONFIG_YAML
# Dependencies
: <<'DEPENDENCIES'
- jq
- yq (confirmed working with with v3.4.1)
- nc
- bash
- ggrep (if running on macOS)
- openssh-client
DEPENDENCIES
set -e
: ${KUBECONFIG:=${HOME}/.kube/config}
export KUBECONFIG
RANDOM_ID="$(od -x /dev/urandom | head -1 | awk '{print $2$4}')"
DEBUG=0
usage() {
cat <<EOM
Usage: $0 [--debug] [ exec | completion ] <context-name> [ -s | -- <cli tool using KUBECONFIG> ]
-h | --help | --usage displays usage
--debug enables debug mode
completion outputs bash completion
exec enables and executes K8s-Vault
This script works in two modes:
1. Single CLI command mode:
- generates KUBECONFIG from exiting one, based on context name passed
- sets up SSH Connection, Port-Forwarding random local port to K8s API
server host, selected from existing KUBECONFIG based on context name
- executes CLI command
- SSH Connection self-terminates after "ssh_ttl" is reached
2. SHELL Mode:
- generates KUBECONFIG from exiting one, based on context name passed
- sets up SSH Connection, Port-Forwarding random local port to K8s API
server host, selected from existing KUBECONFIG based on context name
- executes SHELL (using \$SHELL environmental variable), with KUBECONFIG
environment variable value set to generated temp config file
- when SHELL terminates, SSH connection is also terminated
By default, temporary KUBECONFIG is generated with
"kubectl view config view --minify --flatten", with necessary updates made
with help of YQ.
Alternatively, if GENERATE_KUBECONFIG_WITH_KUBECTL is set to anything but "1",
script will extract bits and pieces from global KUBECONFIG to create temp one.
Examples:
# Single CLI command mode
$ k8s-vault exec my-prod-context -- kubectl get nodes
# SHELL Mode
$ k8s-vault exec my-prod-context -s
(new shell is opened, with KUBECONFIG environment variable)
$ kubectl get nodes
$ exit
(SSH connection is terminated)
EOM
}
info() {
echo >&2 -e "\e[92mINFO:\e[0m $@"
}
debug(){
if [[ ${DEBUG:-0} -eq 1 ]]; then
echo >&2 -e "\e[95mDEBUG:\e[0m $@"
fi
}
error(){
local msg="$1"
local exit_code="${2:-1}"
echo >&2 -e "\e[91mERROR:\e[0m $1"
if [[ "${exit_code}" != "-" ]]; then
exit ${exit_code}
fi
}
getval() {
local x="${1%%=*}"
if [[ "$x" = "$1" ]]; then
echo "${2}"
return 2
else
echo "${1##*=}"
return 1
fi
}
output_completion() {
cat <<'EOM'
#!/usr/bin/env bash
KUBECONFIG=${KUBECONFIG:-~/.kube/config}
COMPREPLY=()
DEPS=( yq jq )
check_dep() {
if ! which $1 2>&1 >/dev/null; then
echo >&2 "ERROR: dependency missing - \"${1}\""
return 1
fi
}
for dep in ${DEPS[*]}; do
check_dep $dep
done
if [ ! -r ${KUBECONFIG} ]; then
echo >&2 "ERROR: unable to read KUBECONFIG at \"${KUBECONFIG}\""
return 1
fi
_k8svault_get_contexts()
{
local contexts
if contexts=$(yq r -j ${KUBECONFIG} contexts[*].name); then
COMPREPLY+=( $(compgen -W "${contexts[*]}" -- "${_word_last}") )
fi
}
_k8svault_completion()
{
local _word_index=$[${COMP_CWORD}-1]
local _word="${COMP_WORDS[$_word_index]}"
local _word_last="${COMP_WORDS[-1]}"
case $_word in
k8s-vault)
COMPREPLY+=( $(compgen -W "--debug exec completion" -- "${_word_last}") )
return
;;
--debug)
COMPREPLY=( $(compgen -W "exec completion" -- "${_word_last}") )
return
;;
completion)
return
;;
exec)
_k8svault_get_contexts
return
;;
-*)
return
;;
\>*)
return
;;
*)
COMPREPLY=( -s -- )
return
;;
esac
}
if [[ $(type -t compopt) = "builtin" ]]; then
complete -o default -F _k8svault_completion k8s-vault
else
complete -o default -o nospace -F _k8svault_completion k8s-vault
fi
EOM
}
if [[ ${K8S_VAULT:0} -eq 1 ]]; then
error "You are inside k8s-vault spawned shell" -
error "Nested k8s-vault sessions are not supported, exiting.."
fi
[[ "$(uname)" == "Darwin" ]] && grep="ggrep" || grep="grep"
: ${K8SVAULT_CONFIG_DIR:="${HOME}/.kube"}
: ${K8SVAULT_CONFIG:="${K8SVAULT_CONFIG_DIR}/k8s-vault.yaml"}
: ${GENERATE_KUBECONFIG_WITH_KUBECTL:=1}
[[ -d ${K8SVAULT_CONFIG_DIR} ]] || mkdir ${K8SVAULT_CONFIG_DIR}
debug "K8SVAULT_CONFIG: ${K8SVAULT_CONFIG}"
[[ -r "${K8SVAULT_CONFIG}" ]] || \
error "Unable to read config file at \"${K8SVAULT_CONFIG}\", exiting.."
## Get CLI arguments
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help|--usage)
usage
exit 0
;;
--debug)
DEBUG=1
shift 1
;;
--kubeconfig|--kubeconfig=*)
export KUBECONFIG="$(getval "$1" "$2")"
shift $?
debug "KUBECONFIG: ${KUBECONFIG}"
;;
completion)
output_completion
exit 0
;;
exec)
KUBECTL_CONTEXT=$2
shift 2
if [[ "$1" == "-s" ]]; then
SPAWN_SHELL=1
shift 1
break
else
SPAWN_SHELL=0
fi
;;
--)
shift 1
break
;;
*)
error "Unexpected option \"$1\"" -
usage
exit 1
;;
esac
done
read_config_value() {
yq r "${K8SVAULT_CONFIG}" $1
}
read_kubectl_config(){
yq r "${KUBECONFIG}" $1
}
read_kubectl_config_json(){
yq r -j "${KUBECONFIG}" $1
}
if [[ ! -r "${KUBECONFIG}" ]]; then
error "KUBECONFIG is not readable or no such file at ${KUBECONFIG}"
fi
# Checking whether passed context is valid
DETECTED_CONTEXTS="$(read_kubectl_config "contexts.*.name")"
debug "Detected Contexts:\n${DETECTED_CONTEXTS}"
if ! echo "${DETECTED_CONTEXTS}" | grep -q "^${KUBECTL_CONTEXT}$"; then
info "Following contexts are available\n${DETECTED_CONTEXTS}"
error "Context \"${KUBECTL_CONTEXT}\" is not found in \"${KUBECONFIG}\""
else
debug "Context selected: ${KUBECTL_CONTEXT}"
fi
CONTEXT_CLUSTER="$(
read_kubectl_config_json contexts | \
jq -r --arg context ${KUBECTL_CONTEXT} '.[] | select(.name==$context) | .context.cluster'
)"
debug "Context Cluster: ${CONTEXT_CLUSTER}"
CONTEXT_USER="$(
read_kubectl_config_json contexts | \
jq -r --arg context ${KUBECTL_CONTEXT} '.[] | select(.name==$context) | .context.user'
)"
debug "Context User: ${CONTEXT_USER}"
CONTEXT_SERVER_URL="$(
read_kubectl_config_json clusters | \
jq -r --arg cluster ${CONTEXT_CLUSTER} '.[] | select(.name==$cluster) | .cluster.server'
)"
debug "K8s API Server URL: ${CONTEXT_SERVER_URL}"
CONTEXT_SERVER_HOST="$(echo "${CONTEXT_SERVER_URL}" | $grep -Po "(?<=//)[^:]+")"
debug "K8s API Server Host: ${CONTEXT_SERVER_HOST}"
CONTEXT_SERVER_PORT="$(echo "${CONTEXT_SERVER_URL}" | $grep -Po "(?<=:)[0-9]+")"
debug "K8s API Server Port: ${CONTEXT_SERVER_PORT}"
SSH_JUMP_HOST="$(read_config_value clusters.${CONTEXT_CLUSTER}.ssh_jump_host)"
debug "SSH Jumphost: ${SSH_JUMP_HOST}"
SSH_RANDOM_PORT_ENABLED="$(read_config_value ssh_forwarding_port.random)"
if [[ "${SSH_RANDOM_PORT_ENABLED:-false}" == "true" ]]; then
debug "SSH Random Forwarding Port enabled"
SSH_FORWARDING_PORT=$[$[RANDOM%9000]+30000]
debug "Using random-generated port: ${SSH_FORWARDING_PORT}"
else
debug "SSH Random Forwarding Port disabled"
SSH_FORWARDING_PORT="$(read_config_value ssh_forwarding_port.static)"
debug "Using port from config: ${SSH_FORWARDING_PORT}"
fi
PROXY_ELIGIBLE_CONTEXT="$(read_config_value clusters.${KUBECTL_CONTEXT}.enabled)"
debug "Proxy Eligible: ${PROXY_ELIGIBLE_CONTEXT}"
get_context_part() {
read_kubectl_config_json contexts | \
jq --arg context ${KUBECTL_CONTEXT} '.[] | select(.name==$context) | {contexts:[.]}' | \
yq r -P -
}
get_cluster_part() {
if [[ "${PROXY_ELIGIBLE_CONTEXT}" == "true" ]]; then
cat <<EOB
clusters:
- name: ${CONTEXT_CLUSTER}
cluster:
server: https://127.0.0.1:${SSH_FORWARDING_PORT}
insecure-skip-tls-verify: true
EOB
else
read_kubectl_config_json clusters | \
jq --arg cluster ${CONTEXT_CLUSTER} '.[] | select(.name==$cluster) | {clusters:[.]}' | \
yq r -P -
fi
}
get_user_part(){
read_kubectl_config_json users | \
jq --arg user ${CONTEXT_USER} '.[] | select(.name==$user) | {users:[.]}' | \
yq r -P -
}
# generate temp KUBECONFIG
GENERATED_KUBECONFIG="${K8SVAULT_CONFIG_DIR}/k8s-vault-kubeconfig-${RANDOM_ID}.yaml"
touch ${GENERATED_KUBECONFIG}
chmod 600 ${GENERATED_KUBECONFIG}
if [[ ${GENERATE_KUBECONFIG_WITH_KUBECTL:-1} -eq 1 ]]; then
# use native kubectl method
kubectl --context=${CONTEXT_CLUSTER} config view --minify --flatten > ${GENERATED_KUBECONFIG}
yq d -i ${GENERATED_KUBECONFIG} clusters[0].cluster.certificate-authority-data
yq w -i ${GENERATED_KUBECONFIG} clusters[0].cluster.insecure-skip-tls-verify true
yq w -i ${GENERATED_KUBECONFIG} clusters[0].cluster.server https://127.0.0.1:${SSH_FORWARDING_PORT}
else
# generate from bits and pieces
cat >${GENERATED_KUBECONFIG} <<EOB
apiVersion: v1
kind: Config
current-context: ${KUBECTL_CONTEXT}
$(get_context_part)
$(get_cluster_part)
$(get_user_part)
EOB
fi
debug "Generated file: ${GENERATED_KUBECONFIG}"
# use generated KUBECONFIG
export KUBECONFIG=${GENERATED_KUBECONFIG}
K8S_API_CONNECT_TIMEOUT="$(read_config_value k8s_api_timeout)"
debug "K8s API Connection Timeout: ${K8S_API_CONNECT_TIMEOUT}"
SSH_TTL="$(read_config_value ssh_ttl)"
debug "SSH Time-To-Live: ${SSH_TTL}"
SSH_PORT_FORWARD_OPT="-L${SSH_FORWARDING_PORT}:${CONTEXT_SERVER_HOST}:${CONTEXT_SERVER_PORT}"
SSH_PID=""
function _exit {
if kill -0 ${SSH_PID:-99999999} 2>/dev/null; then
debug 'Shutting down SSH Port-Forward..'
kill $SSH_PID
fi
if [[ -r "${GENERATED_KUBECONFIG}" ]]; then
debug "Removing temporary KUBECONFIG: \"${GENERATED_KUBECONFIG}\""
rm ${GENERATED_KUBECONFIG}
fi
}
trap _exit EXIT
if [[ "${PROXY_ELIGIBLE_CONTEXT}" != "true" ]]; then
if [[ ${SPAWN_SHELL:0} -eq 1 ]]; then
error "No point in spawning shell for Context not configured for SSH Proxying"
fi
debug "Passing though"
exec $@
fi
start_ssh_session(){
ssh -N ${SSH_PORT_FORWARD_OPT} ${SSH_JUMP_HOST} &
SSH_PID="$!"
}
start_self_destruct_ssh_session(){
ssh -n ${SSH_PORT_FORWARD_OPT} ${SSH_JUMP_HOST} "sleep ${SSH_TTL}" &
SSH_PID="$!"
}
check_connection() {
local check_start_epoch="$(date +%s)"
local now_epoch
local elapsed
until (echo "" | nc 127.0.0.1 ${SSH_FORWARDING_PORT}); do
now_epoch="$(date +%s)"
elapsed=$((now_epoch-check_start_epoch))
debug "could not tcp connect to K8s API (elapsed: $elapsed).."
if [[ ${elapsed} -gt ${K8S_API_CONNECT_TIMEOUT} ]]; then
error "Could not connect to K8s API within specified timeout (${K8S_API_CONNECT_TIMEOUT}s)"
fi
sleep 0.5
done
}
if [[ "${SSH_TTL}" != "null" ]] && [[ ${SPAWN_SHELL:-0} -eq 0 ]]; then
start_self_destruct_ssh_session
else
start_ssh_session
fi
debug "SSH Proxy PID: $SSH_PID"
check_connection
if [[ ${SPAWN_SHELL:-0} -eq 1 ]]; then
KUBECONFIG="${KUBECONFIG}" \
KUBECTL_CONTEXT=${KUBECTL_CONTEXT} \
K8S_VAULT=1 \
${SHELL}
else
$@
fi
@peterjanes

This comment has been minimized.

Copy link

@peterjanes peterjanes commented Jan 15, 2021

For others who've found a number of yq tools out there, this script needs https://github.com/mikefarah/yq v3. (Took me a few minutes to go through the different possibilities.)

@anapsix

This comment has been minimized.

Copy link
Owner Author

@anapsix anapsix commented Jan 15, 2021

@peterjanes yep, you're right..
yq changes bit me few times here.. had to update the code to get what was expected from YQ
I've stated working on switching to oq, which is just awesome.. but haven't gotten around to finish that

@peterjanes

This comment has been minimized.

Copy link

@peterjanes peterjanes commented Jan 15, 2021

Thanks for putting this together, by the way... took a bit to get my k8s-vault and kubeconfig files to agree, but it's been working well since.

@anapsix

This comment has been minimized.

Copy link
Owner Author

@anapsix anapsix commented Jan 18, 2021

@peterjanes, finished a first version of Crystal implementation: k8s-vault.cr

NOTE: There is a slight change in k8s-vault.yaml config file - check out included example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment