I hereby claim:
- I am escvector on github.
- I am escvector (https://keybase.io/escvector) on keybase.
- I have a public key ASAxw1_R3ejH9LSafgPX2zyOBipytFVysqdYhs5qX6T3hgo
To claim this, I am signing this object:
development: | |
adapter: mysql2 | |
encoding: utf8 | |
database: my_database | |
username: root | |
password: |
http://answers.oreilly.com/topic/460-how-to-benchmark-a-hadoop-cluster/ | |
http://www.michael-noll.com/blog/2011/04/09/benchmarking-and-stress-testing-an-hadoop-cluster-with-terasort-testdfsio-nnbench-mrbench/ | |
## MR pi | |
sudo -u hdfs hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-0.20-mapreduce/hadoop-examples.jar pi 10 100 | |
## terasort | |
sudo -u hdfs hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-0.20-mapreduce/hadoop-examples.jar teragen 1000 /user/cloudera/terasort-input | |
sudo -u hdfs hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-0.20-mapreduce/hadoop-examples.jar teragen 100000000 /user/cloudera/terasort-input |
I hereby claim:
To claim this, I am signing this object:
#!/bin/sh | |
source ./control.sh | |
function no_hugepages() { | |
echo never > /sys/kernel/mm/transparent_hugepage/enabled | |
echo never > /sys/kernel/mm/transparent_hugepage/defrag | |
} | |
function verify_capacity() { |
## Query syntax | |
## http://jmespath.org/examples.html#filters-and-multiselect-lists | |
#list all tenancy ACTIVE compartments [name,ocid] | |
oci iam compartment list --all --output table --compartment-id-in-subtree true --query "data [?\"lifecycle-state\" =='ACTIVE'].{Name:name,OCID:id}" | |
#list all tenancy DELETED compartments [name,ocid] | |
oci iam compartment list --all --output table --compartment-id-in-subtree true --query "data [?\"lifecycle-state\" =='DELETED'].{Name:name,OCID:id}" | |
#image list |
# https://docs.aws.amazon.com/cli/latest/reference/ec2/import-key-pair.html | |
# aws ec2 delete-key-pair --key-name "cldr" | |
ssh-keygen -t rsa -C "cldr" -b 4096 -f cldr -q -N "" | |
mv cldr cldr.pem | |
aws ec2 import-key-pair --key-name "cldr" --public-key-material file://cldr.pub | |
rem # https://docs.aws.amazon.com/cli/latest/reference/ec2/import-key-pair.html | |
rm cldr.pem | |
rm cldr.pub |
# Source https://stackoverflow.com/questions/2311510/getting-a-machines-external-ip-address-with-python | |
#### script #### | |
#Returns ip4 | |
curl 'https://api6.ipify.org?format=json' | |
curl 'https://api.ipify.org?format=json' | |
#### Python #### |
ratelimits = api.rate_limit_status() | |
for y in ratelimits.get("resources").items(): | |
for q in y: | |
if isinstance(q,dict): | |
for r1,r2 in q.items(): | |
for k1,v1 in r2.items(): | |
print(f"ratelimits['resources']['{y[0]}']['{r1}']['{k1}']") | |
ratelimits['resources']['lists']['/lists/list']['limit'] | |
ratelimits['resources']['lists']['/lists/list']['remaining'] |
REM this is for direct install onto a machine without virtenv | |
REM uses combo cygwin integrated and old dos commands | |
pip list --outdated --format=columns |tail -n +3|cut -d " " -f1 > update.txt | |
for /f %i in (update.txt) do pip install %i --upgrade |
Count files in directory | |
find . -type f | wc -l | |
Diretory Size in nice format | |
du -h | |
Use all CPU to Zip | |
https://www.gnu.org/software/parallel/ | |
parallel ::: 'gzip -9' ::: * |