Let's look at some basic kubectl output options.
Our intention is to list nodes (with their AWS InstanceId) and Pods (sorted by node).
We can start with:
kubectl get no
#!/bin/bash | |
ES_URL='https://localhost:9200' | |
curl_get='curl -s -X GET --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key' | |
date | |
$curl_get $ES_URL/_cat/health?v | |
$curl_get $ES_URL/_cat/allocation?v\&h=node,host,ip,shards,disk.indices,disk.used,disk.avail,disk.total,disk.percent | |
# See https://www.elastic.co/guide/en/elasticsearch/reference/2.4/cat-nodes.html for header meanings | |
$curl_get $ES_URL/_cat/nodes?v\&h=name,host,r,m,hc,hp,hm,rc,rp,rm,fdc,fdp,fdm,load,uptime |
1.Producer | |
1.request.required.acks=[0,1,all/-1] 0 no acknowledgement but ver fast, 1 acknowledged after leader commits, all acknowledged after replicated | |
2.use Async producer - use callback for the acknowledgement, using property producer.type=1 | |
3.Batching data - send multiple messages together. | |
batch.num.messages | |
queue.buffer.max.ms | |
4.Compression for Large files - gzip, snappy supported | |
very large files can be stored in shared location and just the file path can be logged by the kafka producer. | |
# Code for Part 1 of the video series | |
# Special thanks to Siraj Raval for his excellent work in the community, inspired me to do this | |
# Special thanks to Andrej Karpathy and his blog-post "http://karpathy.github.io/neuralnets/" | |
# Special thanks to iamtrask and his blog-post "http://iamtrask.github.io/2015/07/12/basic-python-network/" | |
import math | |
import random |
sudo su | |
# Java | |
yum -y install java-1.8.0-openjdk-devel | |
# Build Esentials (minimal) | |
yum -y install gcc gcc-c++ kernel-devel make automake autoconf swig git unzip libtool binutils | |
# Extra Packages for Enterprise Linux (EPEL) (for pip, zeromq3) | |
yum -y install epel-release |
A curated list of AWS resources to prepare for the AWS Certifications
A curated list of awesome AWS resources you need to prepare for the all 5 AWS Certifications. This gist will include: open source repos, blogs & blogposts, ebooks, PDF, whitepapers, video courses, free lecture, slides, sample test and many other resources.
Grouped by topic
sudo yum -y install epel-release | |
sudo yum -y install gcc gcc-c++ python-pip python-devel atlas atlas-devel gcc-gfortran openssl-devel libffi-devel | |
# use pip or pip3 as you prefer for python or python3 | |
pip install --upgrade virtualenv | |
virtualenv --system-site-packages ~/venvs/tensorflow | |
source ~/venvs/tensorflow/bin/activate | |
pip install --upgrade numpy scipy wheel cryptography #optional | |
pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.10.0rc0-cp35-cp35m-linux_x86_64.whl | |
# or below if you want gpu, support, but cuda and cudnn are required, see docs for more install instructions | |
pip install --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.10.0rc0-cp35-cp35m-linux_x86_64.whl |