Skip to content

Instantly share code, notes, and snippets.

@mtsuszycki
mtsuszycki / gist:52ebd4b14e02d8bea7a47a155182c88a
Last active June 9, 2023 15:29
Linux LVS layer 4 load balancer general notes, pointers and recommendations
=============
GENERAL INFO
Linux Virtual Server (LVS) is a high availability Layer4 load balancer that transparently for the users presents cluster
of real servers as one virtual server.
Internal cluster architecture is hidden and only IP of virtual server (VIP) is available to the outside world.
In general LVS is used to load balance TCP and UDP protocols and doesn't care what machines and operating systems are used
as real servers.
Properly configured LVS system is able to handle several virtual servers with dozens of real servers in a cluster,
@mtsuszycki
mtsuszycki / lang_comparison.md
Last active December 21, 2016 12:52
ruby, python, perl, php comparision side by side
  • | perl | php | python | ruby ------------ |------------ | ------------- | -------------| ------------- block | {} | {} | : | {} statement separator | ; or nl | ; | ; or nl | ; or nl comment | # | // # | # | # comment1 | =pod =cut | /* abc */ | '''abc''' | =begin =end string slice | - | - | mystring[0:4] | -
@mtsuszycki
mtsuszycki / health_es_indexes.sh
Last active October 18, 2016 14:19
script to check ElasticSearch indexes' size and compare to their average #elasticsearch #admin #health #indexes
#!/bin/bash
# script to check ElasticSearch indexes' size and compare to their average.
# send an email if current index size is lower or grater than average +20%
# averages are in the config file, for example:
#
#declare -A index
#index[logcdn]=25000000
#index[logjboss]=950000
#index[logmonitor]=12000
@mtsuszycki
mtsuszycki / es_remove_selected_rows.sh
Created August 23, 2016 09:26
How to remove selected rows from elasticsearch index by query term filter #elasticsearch #admin #query
#!/bin/bash
# example of removing by query from elasticsearch
# redirect the output to a file and then use curl to bulk load it to ES:
# es_remove_selected_rows.sh > out.json
#
# curl --max-time 120000 -XPOST "$es_host:9200/_bulk" --data-binary @out.json
#
# change conditions in the query/term below
@mtsuszycki
mtsuszycki / hadoop_health_check.sh
Created June 19, 2016 21:16
Basic functions to do hadoop (hdfs) health check #bigdata #hadoop #bash #hdfs
#!/bin/bash
function emsg() { email_body="$email_body\n${1}"; }
function email()
{
local subject=$1 rcpt=$2
echo -e "$email_body\n\n\n" | mail -s "$subject" -r noreply@somedomain.com "$rcpt"
}
@mtsuszycki
mtsuszycki / virsh_qemu.txt
Created June 10, 2016 14:05
basic virsh commands, extending partition in guest OS etc.
# Basic command to manage VMs
# list all defined domains (VMs) and their status (running, shut off etc.)
virsh list --all
# stop VM (named _myvm_) gracefully
virsh shutdown _myvm_
# start VM
@mtsuszycki
mtsuszycki / html_grep.rb
Last active June 10, 2016 14:05
Find and print DOM element from html, ruby, mechanize
#!/usr/bin/ruby
# examples
# ./htmlgrep.rb "input#gbv" "http://www.google.co.uk"
# ./htmlgrep.rb "div#gbv" "http://www.google.co.uk"
# htmlgrep.rb "[@class='gbm']" "http://www.google.co.uk"
# ./htmlgrep.rb "[@class='gbmc']/ol/li" "http://www.google.co.uk"
# ~/htmlgrep.rb '.postblockcat_whitesquare/a' HERE\ -\ Nokia\ Conversations.html | grep -Eo '<a href[^>]+' | sed 's/title=/,/
# g; s/<a href=//g;' > HERE\ -\ Nokia\ Conversations.csv
@mtsuszycki
mtsuszycki / ssh_cluster.sh
Last active June 10, 2016 13:52
execute remote command via ssh passwordlessly (no ssh keys) on many hosts
# execute remote command via ssh passwordlessly (no ssh keys) on many hosts
export SSHPASS=my_password # sshpass will use it
printf "host1\host2\n" | xargs -i sshpass -e ssh your_username@{} 'ls -la'
# or
cat hostlist.txt | xargs -i sshpass -e ssh your_username@{} 'ls -la'
# mass copy ssh public key onto many hosts
cat hostlist.txt | xargs sshpass -e ssh-copy-id -i ~/.ssh/id_rsa.pub
@mtsuszycki
mtsuszycki / amazon_get_price.rb
Created June 10, 2016 13:10
Get product price from amazon website, ruby, mechanize
#!/usr/bin/ruby
require 'rubygems'
require 'mechanize'
# to avoid 'ran out of buffer space on element' error
module WWW
require 'hpricot'
class Mechanize
Hpricot.buffer_size = 262144 # added by naofumi