prasanthkothuri

Check details of certificate openssl x509 -in /etc/ssl/certs/ca-certificates.crt -text -noout Show certificates in the chain openssl s_client -showcerts -connect host_name:port

prasanthkothuri

import prefect
from datetime import datetime, timezone
client = prefect.Client()

flows = client.graphql(
    {
        'query': {
            'flow': {
                'id'
            }

prasanthkothuri

SELinux provides a user, a role, a type, and a level (ls -Z). This information is used to make access control decisions. On DAC systems, access is controlled based on Linux user and group IDs. SELinux policy rules are checked after DAC rules. SELinux policy rules are not used if DAC rules deny access first.

For container to mount the host volume if SELinux is enabled on the host

1. Kubernetes

• Enable PodSecurityPolicy in Admission Controller link
• Create PodSecurityPolicy with SeLinux rule link

prasanthkothuri

Number of tables in each namespace

# table count per namespace (environment)
echo "NAMESPACE,TABLE_COUNT"
for namespace in $(echo "list_namespace" | hbase shell -n | grep nxcals)
do
table_count=$(echo "list_namespace_tables '$namespace'" | hbase shell -n | wc -l)
echo "$namespace,$table_count"
done

prasanthkothuri

Link to the master - https://kubernetes.io/docs/reference/kubectl/cheatsheet/

list services

kubectl get services --all-namespaces

kubernetes dashboard

kubectl describe services/kubernetes-dashboard -n kube-system

get user token

prasanthkothuri

Reading json into Spark Dataframe

method 1 (efficient, specify the schema on construction the dataframe)

from pyspark.sql.types import *
schema = StructType([StructField('aggregated', StringType(), True),
                     StructField('body', StringType(), True),
                     StructField('entity', StringType(), True),
                     StructField('metric_id', StringType(), True),

prasanthkothuri

find all large files in the root filesystem

find / -xdev -type f -size +100M
OR
find / -xdev -type f -size +100M -exec ls -la {} \; | sort -nk 5
OR
find / -xdev -type f -size +100M -exec du -sh {} ';' | sort -rh | head -n50

install netstat on centos 7

prasanthkothuri

squash the last X commits into a single commit

combine

git reset --soft HEAD~X  
git commit -m "combined commit message"

push

git push origin +name-of-branch

prasanthkothuri

# set acl
hdfs dfs -setfacl -R -m user:{user_name}:rwx /path/to/the/directory
# find the quota
hdfs dfs -count -v -q -h /path/to/the/directory
# Set the space quota
hdfs dfsadmin -setSpaceQuota 500g /path/to/the/directory
# Set file / dir (inodes) quota
hdfs dfsadmin -setQuota 500000 /path/to/the/directory

prasanthkothuri

# read the csv into R dataframe
#url <- "/Users/prasanth/Desktop/amzn.csv"
url <- "https://www.dropbox.com/s/ffq7d5dnb47jzig/amzn.csv?dl=1"
amzn <- read.csv(url,header = TRUE, sep=",")

# Order the dataframe and inspect data
amzn <- amzn[order(amzn$year),] 
head(amzn)

# Simple line graph with ggplot2