Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Playing with kubectl output

Kubectl output options

Let's look at some basic kubectl output options.

Our intention is to list nodes (with their AWS InstanceId) and Pods (sorted by node).

We can start with:

kubectl get no

and

kubectl get po -o wide

Json and Jq

I've found the internal data structures easier to explore using the -o json output with jid and jq.

Once both jq and jid are installed (assuming OSX), we can quickly discover the data with the following command:

kubectl get no -o json | jid -q | pbcopy

This allows us to explore the json data interactively and keep our final jq query on the clipboard:

asciicast

note: jid currently implements it's own query parser to allow powerfull autocompletion, the drawback is a lack of support for all the jq constructs (i.e.: we have to specify an index for array elements during discovery).

As can be seen in the recording: once done with jid, getting rid of the index on the items array in jq, did gave us the full listing.

jq gives us a lot more power for example:

Boxing the result into it's own array and constructing a new object combining several nested attributes gives us the following query:

kubectl get no -o json | jq -r '[.items[] | {name:.metadata.name, id:.spec.externalID, unschedulable:.spec.unschedulable}]'

Here is how the above query was built up using jid and jq: asciicast

Converting the json array into a tabular output with jq can be done using @tsv as follows:

kubectl get no -o json | jq -r '.items[] | select(.spec.unschedulable!=true) | [.metadata.name,.spec.externalID] | @tsv'

Jq also allows us to sort:

kubectl get po -o json | jq -r '.items | sort_by(.spec.nodeName)[] | [.spec.nodeName,.metadata.name] | @tsv'

The input for the sort_by command must be an array, we iterate the elements after the sorting.

Custom Columns and Sorting

If all we need is a nicely formatted, sorted tabular report, kubectl has built-in support for powerfull sorting:

kubectl get po -o wide --sort-by=.spec.nodeName

Using jid to list pods sorted by node: asciicast

The usage of Custom Columns with the knowledge of the data structure gained from jid, is also much easier:

kubectl get no -o=custom-columns=NAME:.metadata.name,AWS-INSTANCE:.spec.externalID,UNSCHEDULABLE:.spec.unschedulable

Note: apart from using grep, there is no easy way to filter.

Golang Templates

If we do not wish to use jq (or have no access to jq) need filtering and powerfull output control, we may use Kubectl's built-in support for golang templates (inline or from a template file on disk):

kubectl get no -o go-template='{{range .items}}{{if .spec.unschedulable}}{{.metadata.name}} {{.spec.externalID}}{{"\n"}}{{end}}{{end}}'
or
kubectl get no -o go-template="{{range .items}}{{if .spec.unschedulable}}{{.metadata.name}} {{.spec.externalID}}:{{end}}{{end}}" | tr ":" "\n"

I could not find an easy way to print newline characters with inline golang template, so used a trick printing colons and using tr to convert colons to newlines.

JSONPath

Golang templates can be complicated and verbose - an alternative, if you are more familiar with jq-style queries, or awscli, is to use JSONPath.

kubectl get no -o jsonpath="{.items[?(@.spec.unschedulable)].metadata.name}"

Internally, this seems tightly coupled to the golang templates.

Kubectl supports a superset of JSONPath, with a special range keyword to iterate over ranges, using the same trick to add newlines:

kubectl get no -o jsonpath="{range.items[?(@.spec.unschedulable)]}{.metadata.name}:{end}" | tr ":" "\n"

More examples of using jsonpath can be found in the Kubernetes tests for the JSONPath utility

@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Feb 2, 2017

To list all services in a cluster and their nodeports:

kubectl get --all-namespaces svc -o json | jq -r '.items[] | [.metadata.name,([.spec.ports[].nodePort | tostring ] | join("|"))] | @csv'

Exercise: make it more concise using gotemplates

@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Mar 14, 2017

Quickly switch contexts and namespaces with these aliases

alias kswitch='kubectl config use-context'
alias knamespace='kubectl config set-context `kubectl config current-context` --namespace'

Get namespace of current context

export kcontext=`kubectl config current-context`
export kns=kubectl config view -o jsonpath="{.contexts[?(@.name==\"$kcontext\")].context.namespace}"

Can be added to bash prompt, if using powerline with following segment https://github.com/so0k/powerline-kubernetes

or blended into https://twitter.com/micahhausler/status/785833744863268871

NORMAL="\[\033[00m\]"
BLUE="\[\033[01;34m\]"
RED="\[\e[1;31m\]"
YELLOW="\[\e[1;33m\]"
GREEN="\[\e[1;32m\]"
PS1_WORKDIR="\w"
PS1_HOSTNAME="\h"
PS1_USER="\u"

__kube_ps1()
{
    CONTEXT=$(kubectl config current-context)
    NAMESPACE=$(kubectl config view -o jsonpath="{.contexts[?(@.name==\"${CONTEXT}\")].context.namespace}")
    if [ -z "$NAMESPACE"]; then
        NAMESPACE="default"
    fi
    if [ -n "$CONTEXT" ]; then
        case "$CONTEXT" in
          *prod*)
            echo "${RED}(⎈ ${CONTEXT} - ${NAMESPACE})"
            ;;
          *test*)
            echo "${YELLOW}(⎈ ${CONTEXT} - ${NAMESPACE})"
            ;;
          *)
            echo "${GREEN}(⎈ ${CONTEXT} - ${NAMESPACE})"
            ;;
        esac
    fi
}

export PROMPT_COMMAND='PS1="${GREEN}${PS1_USER}@${PS1_HOSTNAME}${NORMAL}:$(__kube_ps1)${BLUE}${PS1_WORKDIR}${NORMAL}\$ "'
@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Mar 14, 2017

other handy aliases:

# get all services with in a cluster and the nodePorts they use (if any)
alias ksvc="kubectl get --all-namespaces svc -o json | jq -r '.items[] | [.metadata.name,([.spec.ports[].nodePort | tostring ] | join(\"|\"))] | @csv'"
# shortcuts for frequent kubernetes commands
alias kpods="kubectl get po"
alias kinspect="kubectl describe"
function krun() { name=$1; shift; image=$1; shift; kubectl run -it --generator=run-pod/v1 --image $image $name -- $@; }
function klogs() { kubectl logs $*;}
function kexec(){ pod=$1; shift; kubectl exec -it $pod -- $@; }
@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Mar 17, 2017

sample of running a pipeline on every pod (in this case, es bound to primary interface, not 0.0.0.0):

export appname=elasticsearch
for p in `kubectl get po -l app=$appname -o name`;do echo $p; kubectl exec ${p:4} -- sh -c 'hostname -i | xargs -I {} wget -qO- {}:9200/| grep "\"name\""'; done

output

pod/1
  "name" : "Archer",
pod/2
  "name" : "Ogress",
pod/3
  "name" : "Silver Dagger",

(only works if all elasticsearch pods expose client services..)

@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Mar 21, 2017

Get status transitions of each pod (can use grep to filter down to terminated pods)

export jsonpath='{range .items[*]}{"\n"}{@.metadata.name}{range @.status.conditions[*]}{"\t"}{@.type}={@.status}{end}{end}' 
kubectl get po -ao jsonpath="$jsonpath" && echo

source

see also:

Delete failed jobs (thnks @RobinSE)

$ kubectl delete pod $(kubectl get pods --show-all -o go-template='{{range .items}}{{if eq .status.phase "Failed"}}{{.metadata.name}}{{"\n"}}{{end}}{{end}}
@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Mar 29, 2017

Run a command on every cluster node,

for example, get uptime of all nodes (depends on your ssh setup, see below)

while read ip; do echo $ip; ssh $ip uptime </dev/null; done < <(kubectl get no -o jsonpath='{range.items[*].status.addresses[?(@.type=="InternalIP")]}{.address}{"\n"}{end}')

note: ssh eats stdin by default, have to redirect it's stdin (</dev/null) to ensure the while loop can continue reading from stdin

note: ssh setup - if you are using a bastion host for your k8s nodes in subnet 10.100.*, you can use following ssh

Host <bastion-host-ip>
  User core
  IdentityFile ~/.ssh/<bastion-host-key>.pem

Host 10.100.*
  User core
  ProxyCommand ssh -A -W %h:%p <bastion-host-ip>
  IdentityFile ~/.ssh/<bastion-host-key>.pem

kubectl delete pod stays in terminating status

Is docker ps hanging on any node? Where is my container (maybe you force deleted the pod in k8s, but it seems it is still alive somewhere...:

while read ip; do echo $ip; ssh $ip "docker ps | grep my-container" < /dev/null; done < <(kubectl get no -o json | jq -r '.items[].status.addresses[] | select(.type=="InternalIP") | .address')

this time with jq as an alternative

What else is running on this mis-behaving node?

kubectl get po -o wide | grep ip-10-100-10-25
@tuannvm

This comment has been minimized.

Copy link

@tuannvm tuannvm commented Jul 13, 2017

Get k8s node's internal IPs:

kubectl get no -o json | jq -r '.items[].status.addresses[] | select(.type=="InternalIP") | .address
@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Aug 4, 2017

Get Secret:

secret_name=test
DOCKER_REGISTRY=registry.example.com
kubectl get secret ${secret_name}  -o jsonpath="{['data']['\.dockercfg']}" | base64 -D | jq .

# or just read it into 2 env vars
read DOCKER_USERNAME DOCKER_PASSWORD <<<$(kubectl get secret ${secret_name} -o jsonpath="{['data']['\.dockercfg']}" | base64 -D | jq  -r ".[\"${DOCKER_REGISTRY}\"] | [.username,.password] | @tsv")
@TerraTech

This comment has been minimized.

Copy link

@TerraTech TerraTech commented Aug 28, 2017

Original:

Kubectl supports a superset of JSONPath, with a special range keyword to iterate over ranges, using the same trick to add newlines:

kubectl get no -o jsonpath="{range.items[?(@.spec.unschedulable)]}{.metadata.name}:{end}" | tr ":" "\n"

@so0k This should work for adding newlines:

kubectl get no -o jsonpath='{range.items[?(@.spec.unschedulable)]}{.metadata.name}{"\n"}{end}'

I'm currently using this for finding PVs that have Released PVCs:

kubectl get pv -o jsonpath='{range.items[?(@.status.phase=="Released")]}{.metadata.name}{"\n"}{end}'
@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Aug 28, 2017

@ghulevishal

This comment has been minimized.

Copy link

@ghulevishal ghulevishal commented Oct 30, 2017

Get PV

kubectl get pv -o json | jq -r '.items | sort_by(.spec.capacity.storage)[]|[.metadata.name,.spec.capacity.storage]| @tsv'
@ghulevishal

This comment has been minimized.

Copy link

@ghulevishal ghulevishal commented Oct 30, 2017

Get Pod on specific node.

kubectl get po -o json | jq -r '.items | sort_by(.spec.nodeName)[]|select(.spec.nodeName=="node3")|[.metadata.name,.spec.nodeName]| @tsv'
@ghulevishal

This comment has been minimized.

Copy link

@ghulevishal ghulevishal commented Oct 30, 2017

Get sorted list of nodes with respect to Memory Capacity.

kubectl get no -o json | jq -r '.items | sort_by(.status.capacity.memory)[]|[.metadata.name,.status.capacity.memory]| @tsv'
@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Jan 10, 2018

based on kubernetes/kubernetes#55051 (comment) (and tested on k8s 1.7)

kubectl get po -a --all-namespaces -o json | jq  '.items[] | select(.status.reason!=null) | select(.status.reason | contains("Evicted")) | "kubectl delete po \(.metadata.name) --n \(.metadata.namespace)"' | xargs -n 1 bash -c

trying out fieldSelector see valid fields - to delete all non-running pods...

kubectl get po -a --all-namespaces -o json --field-selector="status.phase!=Running" | jq  '.items[]  | "kubectl delete po \(.metadata.name) --n \(.metadata.namespace)"' | xargs -n 1 bash -c
@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Apr 23, 2018

Get podcount per node, in ascending order

kubectl get po -o json --all-namespaces | jq '.items | group_by(.spec.nodeName) | map({"nodeName": .[0].spec.nodeName, "count": length}) | sort_by(.count)'

do note that for very large clusters it might be faster to use jq inputs - https://github.com/stedolan/jq/wiki/Cookbook#processing-a-large-number-of-lines-or-json-entities

@geoand

This comment has been minimized.

Copy link

@geoand geoand commented May 8, 2018

Get the display name of all the clusterserviceclass objects that are provided by the Ansible Service Broker (Openshift specific):

kubectl get clusterserviceclass -o json | jq -r '.items[].spec | select(.clusterServiceBrokerName=="ansible-service-broker") | .externalMetadata.displayName'
@kevinbringard

This comment has been minimized.

Copy link

@kevinbringard kevinbringard commented May 22, 2018

FYI, It looks like someone created a fork of jid and added jq filtering queries: https://github.com/fiatjaf/jiq

@pracucci

This comment has been minimized.

Copy link

@pracucci pracucci commented Jun 19, 2018

For each pod, list whether it containers run on a read-only root filesystem or not:

kubectl get pods --all-namespaces -o go-template --template='{{range .items}}{{.metadata.name}}{{"\n"}}{{range .spec.containers}}    read-only: {{if .securityContext.readOnlyRootFilesystem}}{{printf "\033[32m%t\033[0m" .securityContext.readOnlyRootFilesystem}} {{else}}{{printf "\033[91m%s\033[0m" "false"}}{{end}} ({{.name}}){{"\n"}}{{end}}{{"\n"}}{{end}}'
@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Jun 29, 2018

for any resource type, labels can be printed as columns using the -L flag:

kubectl get no -Lbeta.kubernetes.io/instance-type -Lfailure-domain.beta.kubernetes.io/zone
NAME                                           STATUS    ROLES     AGE       VERSION   INSTANCE-TYPE   ZONE
ip-10-51-....ap-southeast-1.compute.internal   Ready     node      39d       v1.9.3    m4.xlarge       ap-southeast-1c
ip-10-51-....ap-southeast-1.compute.internal   Ready     node      43d       v1.9.3    m4.xlarge       ap-southeast-1c
ip-10-51-....ap-southeast-1.compute.internal   Ready     node      43d       v1.9.3    m4.xlarge       ap-southeast-1a
ip-10-51-....ap-southeast-1.compute.internal   Ready     master    43d       v1.9.3    m4.large        ap-southeast-1a
ip-10-51-....ap-southeast-1.compute.internal   Ready     master    43d       v1.9.3    m4.large        ap-southeast-1b
ip-10-51-....ap-southeast-1.compute.internal   Ready     node      43d       v1.9.3    m4.xlarge       ap-southeast-1b
ip-10-51-....ap-southeast-1.compute.internal   Ready     master    43d       v1.9.3    m4.large        ap-southeast-1c
$ kubectl get po -l release=panama-dev -Lcomponent
NAME                                             READY     STATUS    RESTARTS   AGE       COMPONENT
panama-dev-amqp-68fbd99f9c-4w2ph                 1/1       Running   0          42d       amqp
panama-dev-apidoc-6bc7c87c9d-s4t7x               2/2       Running   0          31d       apidoc
panama-dev-artemis-56c786b6c8-bgn5f              1/1       Running   0          31d       artemis
panama-dev-artemis-56c786b6c8-p6hfv              1/1       Running   0          31d       artemis
panama-dev-asteria-56cfdbf769-rjclt              2/2       Running   0          31d       asteria
@jhliptak

This comment has been minimized.

Copy link

@jhliptak jhliptak commented Sep 5, 2018

go-templates can have newlines in them using the println function.

For example, to get all Users in all namespaces (assuming you have permissions):
kubectl get rolebindings --all-namepsaces -o go-template --template='{{range .items}}{{println}}{{.metadata.namespace}}={{range .subjects}}{{if eq .kind \"User\"}}{{.name}} {{end}}{{end}}{{end}}'
will print the namespaces followed by a space separated list of users.

@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Nov 16, 2018

$ jq --version
jq-1.6

$ k get secret -o json | jq '.items[] | [{"secret":.metadata.name,"data":.data | values | to_entries | map({"key":.key,"value":.value | @base64d}) | from_entries}]'

EDIT - this also works:

$ k get secret my-secret -o json | jq -r '.data | map_values(@base64d)'
@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Jan 13, 2019

With base64d function exposed to go template

template='{{ range $k, $v := .data }}{{ $v | base64decode}}{{"\n"}}{{end}}'

Source - https://twitter.com/oldmanuk/status/1065358163497676801?s=21

@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Jan 21, 2019

This may be more useful for exploration than jid- https://github.com/antonmedv/fx

@StevenACoffman

This comment has been minimized.

Copy link

@StevenACoffman StevenACoffman commented Mar 2, 2019

You may find that https://github.com/ashleyschuett/kubernetes-secret-decode allows nicer output:

k get secret -o yaml | ksd 

It will automatically decode the base64 output

@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Apr 3, 2019

Read Datadog annotations (this is mostly jq features)

$ k get po -l app=envoy -o json | jq '.items[0].metadata.annotations | with_entries(select(.key | startswith("ad.datadoghq.com/"))) | map_values(fromjson)'
{
  "ad.datadoghq.com/envoy.check_names": [
    "envoy"
  ],
  "ad.datadoghq.com/envoy.init_configs": [
    {}
  ],
  "ad.datadoghq.com/envoy.instances": [
    {
      "stats_url": "http://%%host%%:8002/stats"
    }
  ],
  "ad.datadoghq.com/envoy.logs": [
    {
      "source": "envoy",
      "service": "ingress"
    }
  ]
}

or kops channel addons configuration

$ kubectl get ns kube-system -o json | jq '.metadata.annotations | with_entries(select(.value | contains("addons"))) | map_values(fromjson | .version)'
{
  "addons.k8s.io/authentication.aws": "0.3.0",
  "addons.k8s.io/bootstrap.swatmobile.io": "0.0.9",
  "addons.k8s.io/core.addons.k8s.io": "1.4.0",
  "addons.k8s.io/coredns.addons.k8s.io": "1.2.6-kops.1",
  "addons.k8s.io/dns-controller.addons.k8s.io": "1.11.0",
  "addons.k8s.io/fluent-bit-logging.swatmobile.io": "0.0.2",
  "addons.k8s.io/heptio-auth-aws.swatmobile.io": "0.0.3",
  "addons.k8s.io/ingress.swatmobile.io": "0.0.10",
  "addons.k8s.io/limit-range.addons.k8s.io": "1.5.0",
  "addons.k8s.io/networking.cilium.io": "v1.0-kops.2",
  "addons.k8s.io/rbac.addons.k8s.io": "1.8.0",
  "addons.k8s.io/state-metrics.swatmobile.io": "0.0.2",
  "addons.k8s.io/storage-aws.addons.k8s.io": "1.7.0"
}
@kivagant-ba

This comment has been minimized.

Copy link

@kivagant-ba kivagant-ba commented Apr 19, 2019

Find all resources requests and limits per pod

kubectl get pods -o json -n kube-system | jq -r '.items[] | .metadata.name + " \n Req. RAM: " + .spec.containers[].resources.requests.memory + " \n Lim. RAM: " + .spec.containers[].resources.limits.memory + " \n Req. CPU: " + .spec.containers[].resources.requests.cpu + " \n Lim. CPU: " + .spec.containers[].resources.limits.cpu + " \n Req. Eph. DISK: " + .spec.containers[].resources.requests["ephemeral-storage"] + " \n Lim. Eph. DISK: " + .spec.containers[].resources.limits["ephemeral-storage"] + "\n"'

coredns-7bcbfc4774-7sxg2
 Req. RAM: 70Mi
 Lim. RAM: 170Mi
 Req. CPU: 100m
 Lim. CPU:
 Req. Eph. DISK: 100Mi
 Lim. Eph. DISK: 768Mi
...

Get all requests or limits from a namespace

echo "\nRAM Requests TOTAL:" && kubectl describe namespace kube-system | grep 'requests.memory' && echo "\nRAM Requests:\n" && kubectl get pods -o json -n kube-system | jq -r '.items[] | .spec.containers[].resources.requests.memory + " | " + .metadata.name'

echo "\nRAM Limits TOTAL:" && kubectl describe namespace kube-system | grep 'limits.memory' &&  echo "\nRAM Limits:\n" && kubectl get pods -o json -n kube-system | jq -r '.items[] | .spec.containers[].resources.limits.memory + " | " + .metadata.name'

echo "\nCPU Requests TOTAL:" && kubectl describe namespace kube-system | grep 'requests.cpu' &&  echo "\nCPU Requests:\n" && kubectl get pods -o json -n kube-system | jq -r '.items[] | .spec.containers[].resources.requests.cpu + " | " + .metadata.name'

echo "\nCPU Limits TOTAL:" && kubectl describe namespace kube-system | grep 'limits.cpu' &&  echo "\nCPU Limits:\n" && kubectl get pods -o json -n kube-system | jq -r '.items[] | .spec.containers[].resources.limits.cpu + " | " + .metadata.name'

echo "\nEph. DISK Requests TOTAL:" && kubectl describe namespace kube-system | grep 'requests.ephemeral-storage' && echo "\nEph. DISK Requests:\n" && kubectl get pods -o json -n kube-system | jq -r '.items[] | .spec.containers[].resources.requests["ephemeral-storage"] + " | " + .metadata.name'

echo "\nEph. DISK Limits TOTAL:" && kubectl describe namespace kube-system | grep 'limits.ephemeral-storage' && echo "\nEph. DISK Limits:\n" && kubectl get pods -o json -n kube-system | jq -r '.items[] | .spec.containers[].resources.limits["ephemeral-storage"] + " | " + .metadata.name'


RAM Requests TOTAL:
 requests.memory               3640Mi   16Gi

RAM Requests:

70Mi | coredns-7bcbfc4774-7sxg2
...

Do some operations for pods using simple grep

NS=kube-system; kubectl get pod -n $NS | grep "ContainerCreating \|Init:0/1\|ContainerCreating\|ImagePullBackOff" | awk '{print $1}'|xargs -I % sh -c "kubectl -n $NS describe pod % "
@steinrh

This comment has been minimized.

Copy link

@steinrh steinrh commented Sep 18, 2019

Hi!
I would like to ask if there is a query with jsonpath that I could get the running time from a pod in seconds?
Using this query I will get the complete date:
$ kubectl get pods -o=jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.startTime}{"\n"}{end}'
alertmanager-main-0 2019-XX-XXT16:34:01Z
What I expect is to write a query and get:
alertmanager-main-0 36567(as seconds running)

Is that possible to do?

@Guillaume-Mayer

This comment has been minimized.

Copy link

@Guillaume-Mayer Guillaume-Mayer commented Oct 29, 2019

What about piping json logs to jq?
I tried kubectl logs -f mypod | jq but it shows nothing until I ctrl+c it

@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Oct 30, 2019

@steinrh - I think all the age fields are more polished in kubectl (same with trying to get total allocated vs total available resources of nodes, only the kubectl describe has a nice result there)

@Guillaume-Mayer ... that's a tricky one, maybe jq needs to use streaming flag to read stdin the way kubectl logs expects it?

@Guillaume-Mayer

This comment has been minimized.

Copy link

@Guillaume-Mayer Guillaume-Mayer commented Oct 30, 2019

@so0k - It should be --unbuffered or --stream but none of those worked for me

@kksudo

This comment has been minimized.

Copy link

@kksudo kksudo commented Dec 13, 2019

@kivagant-ba

Find all resources requests and limits per pod

kubectl get pods -o json -n kube-system | jq -r '.items[] | .metadata.name + " \n Req. RAM: " + .spec.containers[].resources.requests.memory + " \n Lim. RAM: " + .spec.containers[].resources.limits.memory + " \n Req. CPU: " + .spec.containers[].resources.requests.cpu + " \n Lim. CPU: " + .spec.containers[].resources.limits.cpu + " \n Req. Eph. DISK: " + .spec.containers[].resources.requests["ephemeral-storage"] + " \n Lim. Eph. DISK: " + .spec.containers[].resources.limits["ephemeral-storage"] + "\n"'

This recursively will duplicate containers and pods, you can check it, just add container name.

kubectl get pods -o json -n kube-system | jq -r '.items[] | .metadata.name + " \n Container name: " + .spec.containers[].name + " \n Req. RAM: " + .spec.containers[].resources.requests.memory + " \n Lim. RAM: " + .spec.containers[].resources.limits.memory + " \n Req. CPU: " + .spec.containers[].resources.requests.cpu + " \n Lim. CPU: " + .spec.containers[].resources.limits.cpu + " \n Req. Eph. DISK: " + .spec.containers[].resources.requests["ephemeral-storage"] + " \n Lim. Eph. DISK: " + .spec.containers[].resources.limits["ephemeral-storage"] + "\n"'

Unfortunately I did not find how to remove duplicates (

@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Jan 7, 2020

if you still have 1.11 clusters, k get apiservices looks very different from 1.13+ clusters..

before:

$ k get apiservice
NAME                                   CREATED AT
v1.                                    2019-02-04T04:02:51Z
v1.apps                                2019-02-04T04:02:51Z
v1.authentication.k8s.io               2019-02-04T04:02:52Z
v1.authorization.k8s.io                2019-02-04T04:02:52Z
v1.autoscaling                         2019-02-04T04:02:52Z
v1.batch                               2019-02-04T04:02:52Z
v1.monitoring.coreos.com               2019-07-03T07:28:49Z
...

with jq -r and column -t:

$ k get apiservice -o json | jq -r '.items[] | [.metadata.name,(if .spec.service.name == null then "Local" else .spec.service.name end),.status.conditions[].type,.metadata.creationTimestamp] | @tsv' | column -t
v1.                                   Local                              Available  2019-02-04T04:02:51Z
v1.apps                               Local                              Available  2019-02-04T04:02:51Z
v1.authentication.k8s.io              Local                              Available  2019-02-04T04:02:52Z
v1.authorization.k8s.io               Local                              Available  2019-02-04T04:02:52Z
v1.autoscaling                        Local                              Available  2019-02-04T04:02:52Z
v1.batch                              Local                              Available  2019-02-04T04:02:52Z
v1.monitoring.coreos.com              Local                              Available  2019-07-03T07:28:49Z
v1.networking.k8s.io                  Local                              Available  2019-02-04T04:02:52Z
v1.rbac.authorization.k8s.io          Local                              Available  2019-02-04T04:02:52Z
v1.storage.k8s.io                     Local                              Available  2019-02-04T04:02:52Z
v1beta1.extensions                    Local                              Available  2019-02-04T04:02:52Z
v1beta1.external.metrics.k8s.io       datadog-cluster-agent-metrics-api  Available  2019-12-12T00:36:22Z
v1beta1.metrics.k8s.io                metrics-server                     Available  2019-12-20T01:17:22Z
v1beta1.policy                        Local                              Available  2019-02-04T04:02:52Z
...
@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Jan 13, 2020

Inspect a TLS secret

$ k get secret
NAME                          TYPE                                  DATA   AGE
my-certs                      kubernetes.io/tls                     3      273d

k get secret my-certs -o json | jq -r .data[\"tls.crt\"] | base64 -D | openssl x509 -in /dev/stdin -text -noout
@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Mar 5, 2020

converting a configmap to a secret, you want to base64 encode all values - using yq to convert yaml to json for jq and back to yaml:

yq r secrets.yaml data -j | jq 'map_values(@base64)' | yq r -

this one liner does not change the kind to secret nor does it add the type: Opaque fields required for secrets

EDIT: or reading a secret from the cluster

$ k get secret my-secrets -o json | jq -r '.data | map_values(@base64d)'
@boomkap

This comment has been minimized.

Copy link

@boomkap boomkap commented Mar 25, 2020

How do i output the value of a field that has a . in the name when using a go-template. For e.g. the Quota object has fields with names like requests.cpu, requests.memory.

I am trying to pull quota allocations vs used for memory, cpu & storage using a go-template.

The closest I got is using the following:

oc get quota -o go-template --template='{{range .items}}{{.status}}{{range .hard}}{{end}}{{end}}' -n <namespace>

This produced the following output. As you can see, this is not ideal for further analysis.

map[hard:map[requests.cpu:5 requests.memory:150Gi] used:map[requests.cpu:1650m requests.memory:841Mi]]

I am assuming there is a way to escape the . in the name so the template engine does not interpret it as a subtree/leaf node.

Any ideas?

@so0k

This comment has been minimized.

Copy link
Owner Author

@so0k so0k commented Apr 30, 2020

@boomkap - sorry, I didn't use quota - did you try\ as escape character?

to check image used by containers in pod:

k get po -o custom-columns=name:.metadata.name,status:.status.phase,image:'.spec.containers[0].image'
name                 status    image
frontend             Running   1234567.dkr.ecr.ap-southeast-1.amazonaws.com/frontend:0.1.0-058d8d4
...
@boomkap

This comment has been minimized.

Copy link

@boomkap boomkap commented Apr 30, 2020

@so0k - does not work with \ as an escape character. Throws the following error

template: output1: bad character U+005c

@venkatalolla

This comment has been minimized.

Copy link

@venkatalolla venkatalolla commented Jul 16, 2020

@boomkap I think this what you are looking for, a command to output the total number of Pods running, CPU request/limits and Memory requests/limits for each projects in a cluster.

oc get quota --all-namespaces -o=custom-columns=Project:.metadata.namespace,TotalPods:.status.used.pods,TotalCPURequest:.status.used.requests'\.'cpu,TotalCPULimits:.status.used.limits'\.'cpu,TotalMemoryRequest:.status.used.requests'\.'memory,TotalMemoryLimit:.status.used.limits'\.'memory

🙂

@vic501

This comment has been minimized.

Copy link

@vic501 vic501 commented Aug 16, 2020

kubectl get no -o json | jq -r '.items | sort_by(.status.capacity.memory)[]|[.metadata.name,.status.capacity.memory]| @TSV'

how to display in MB or GB, currently is displaying in kb. ?

Thanks.

@tuxknight

This comment has been minimized.

Copy link

@tuxknight tuxknight commented Sep 13, 2020

save ConfigMap into files, per file each key,using key as filename.

cm_name=config
ns=default
files=$(kubectl get cm $cm_name -n $ns -o go-template='{{range $k,$v := .data}}{{$k}}{{"\n"}}{{end}}')
for f in $(echo $files);do
  kubectl get cm $cm_name -n $ns -o go-template='{{range $k,$v := .data}}{{ if eq $k "'$f'"}}{{$v}}{{end}}{{end}}' > $f 2>/dev/null
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.