Skip to content

Instantly share code, notes, and snippets.

@0xAB41
0xAB41 / simple_mass_downloader.py
Created November 23, 2012 12:55
A simple mass file downloader in python
import urllib
def start_download(url, base, limit, ctype, ext,file_save_dir):
"""Download 'limit' number of files in base+offset.ext form(ex: 2400.jpg to 2452.jpg) from a url
Args:
url: url including the terminal slash
base: base number of the filename
limit: number of files to fetch
ctype: content type of the file
ext: extension of the file
@0xAB41
0xAB41 / ubuntu and Linux mint post installation
Last active October 13, 2018 09:01
Things to do after installing Ubuntu or Linux Mint
#Install Sublime Text 2
sudo add-apt-repository ppa:webupd8team/sublime-text-2
sudo apt-get update
sudo apt-get install sublime-text
#Install Oracle Java
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get install oracle-java7-installer
sudo update-alternatives --config java
@0xAB41
0xAB41 / .bash_profile
Last active December 21, 2015 06:09
Bash Profile file
HISTFILESIZE=2500
HISTIGNORE="ls:ltrh:ls -l:ls -ltrh:pwd:clear:c:ll"
export JAVA_HOME=$(/usr/libexec/java_home)
export DYLD_LIBRARY_PATH="/Developer/NVIDIA/CUDA-5.5/lib:$DYLD_LIBRARY_PATH"
export PATH="/usr/local/bin:$PATH"
export PATH="$PATH:/usr/local/smlnj/bin"
export PATH="/usr/local/mysql/bin:$PATH"
export PATH="/Developer/NVIDIA/CUDA-5.5/bin:$PATH"
@0xAB41
0xAB41 / memcache_delete_keys.sh
Created September 28, 2013 17:19
Simple & dirty script to list/delete memcached all the keys matching a certain pattern. Works well if you have few thousand keys (precisely <1MB keys on each slab) else execute the script multiple times to make sure that all keys of specified pattern are deleted.
for host in falcon{1..4}
do
echo "==== $host ====";
#Number of slabs memcached uses on my system is 40. Change it appropriately
for i in {1..36}
do
echo "retrieving for slab "$i;
(sleep 1; echo "stats cachedump $i 1000"; sleep 2;) | telnet $host 11211 >> keysmemcached_$host;
done;
echo "done for all slabs..Now processing";
@0xAB41
0xAB41 / calclutate_memcached_hitratio.py
Created October 4, 2013 15:37
quick script to calculate hit ratio of memcached servers
#!/usr/bin/env python
#run the following bash one liner to gather statistics first followed by python script to calculate hit ratio & average hit ratio
#for i in mem-node{1..30}; do (echo "stats"; sleep 2 ) | telnet $i 11211 | grep get_ ; done > stats; python avg.py stats
import sys
from __future__ import division
x = [int(i.strip().split()[2]) for i in open(sys.argv[1]).readlines()]
hitratios=[x[2*i]/(x[2*i]+x[2*i+1]) for i in range(len(x)/2)]
for hr in hitratios:
print hr
@0xAB41
0xAB41 / the_hindu_article_fetch.py
Created February 5, 2016 08:20
Fetches articles from Hindu
class DatesInYear(object):
def __init__(self, year):
from calendar import monthrange
self.days = []
for m in xrange(1,13):
number_of_days = monthrange(year, m)[1]
for d in xrange(1,number_of_days+1):
self.days.append((year, m, d))
@0xAB41
0xAB41 / setuphadoop.sh
Last active March 9, 2016 08:19
setting up hadoop
#Install java 8
add-apt-repository ppa:webupd8team/java
apt-get update && sudo apt-get install oracle-java8-installer
#create hadoop group and a user
addgroup hadoop
adduser --ingroup hadoop hduser
#switch to hduser account
su hduser
@0xAB41
0xAB41 / gist:28c7714c92226ea600235eeac14d7a0a
Created May 20, 2016 06:09
Pelican fails with ImportError: No module named html_parser on el capitan
# Install python/python3 using brew
brew install python
brew install python3
#in pelican/blog/path
virtualenv -p python3 venv_pelican
source venv_pelican/bin/activate
pip3 install six==1.9.0
pip3 install pelican markdown typogrify
@0xAB41
0xAB41 / osx_bootstrap.sh
Last active August 20, 2022 23:45
Brew install script to bootstrap new OSX machine
#!/bin/sh
echo "Checking whether brew is installed..."
if test ! $(which brew); then
echo "Installing homebrew..."
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
fi
brew tap d12frosted/emacs-plus
brew tap caskroom/cask
@0xAB41
0xAB41 / Makefile
Last active May 20, 2020 18:14
Using python as shell in make filels
SHELL = python
.PHONY: greet
greet:
print(" ".join(["Hello", "World!"]))