pat-humphreys

###Use Elastic Curator to cleanup ElasticSearch index backups in S3 ElasticSearch has a great index backup/restore API, which makes the whole process really simple. A single RESTful call creates an incremental backup of the indices and stores it to S3, simple!! Problem is they don’t give you an easy way to manage your backups, so you can end up with massive amounts of data in S3, we had over 8TB which we didn’t need. Enter Curator which allows you to manage indices and snapshots easily.

Install curator either on an ElasticSearch server or another server with access to it Remove "==3.5.1" if you want the latest version

pip install elasticsearch-curator==3.5.1

pat-humphreys

#Install Spark+Scala+Sbt on Ubuntu
scalaVersion=2.11.8
sparkVersion=2.0.0
sbtVersion=0.13.12
hadoopVersion=2.7
sudo apt-get remove scala-library scala
sudo wget www.scala-lang.org/files/archive/scala-$scalaVersion.deb
sudo dpkg -i $scalaVersion.deb
sudo apt-get update
sudo apt-get install scala

pat-humphreys

#If elasticsearch has unassigned replicas that it isnt assigning (normally after a cluster restart/recovery). 
#Realocation and merging might need to be re-enabled
#Might need to be done on all nodes
curl -XPUT 'localhost:9200/_settings' -d '{"index.routing.allocation.disable_allocation": false}' 

#Lists all keys for an index
curl localhost:9200/index-name/index-type/_search/?size=1000&pretty=1

#Turns off/on index replicas
curl -XPUT localhost:9200/index-name/_settings -d '{"number_of_replicas": 0}' 

pat-humphreys

##########################################
# To run: 
# curl -sSL https://gist.githubusercontent.com/pat-humphreys/36b61a38637249d70965cf92886d3a13/raw/de81a8da8b17902b809f8a4cdaca7d4f8ae2710b/Install-Docker-on-Linux-Mint.sh | bash -x
##########################################

# Check that HTTPS transport is available to APT
if [ ! -e /usr/lib/apt/methods/https ]; then
	sudo apt-get update
	sudo apt-get install -y apt-transport-https
fi