Skip to content

Instantly share code, notes, and snippets.

View uolter's full-sized avatar

Walter Traspadini uolter

View GitHub Profile
#! /bin/sh
### BEGIN INIT INFO
# Provides: elasticsearch
# Required-Start: $all
# Required-Stop: $all
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Starts elasticsearch
# Description: Starts elasticsearch using start-stop-daemon
### END INIT INFO
#!/bin/bash
# Elastic Serarch Start and Stop Script
ES_HOME="/opt/elsearch/elasticsearch"
ES_USER="esearch"
PID=$(ps ax | grep elasticsearch | grep $ES_HOME | grep -v grep | awk '{print $1}')
#echo $PID

Simple Website Crawler

The following gist is an extract of the article Building a simple crawler. It allows crawling from a URL and for a given number of bounce.

Basic Usage

from crawler import Crawler
crawler = Crawler()
crawler.crawl('http://techcrunch.com/')

displays the urls

@uolter
uolter / venv_setup.sh
Created September 22, 2014 13:37
venv_and_pip_centos.sh
curl https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py > get-pip.py;
python get-pip.py;
rm -f get-pip.py;
# change directory here. Go in your project home dir.
# cd /opt/uuid_resolver/;
pip install virtualenv;
virtualenv venv;
# activate the virtualenv
source venv/bin/activate
# change here your requirements.txt location
@uolter
uolter / pip_update_all
Created December 2, 2014 09:42
pip update all packages
pip freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs pip install -U
@uolter
uolter / quicksort.py
Created December 29, 2014 10:27
Simple Quck Sort example with python
#!/usr/bin/env
# -*- coding: utf-8 -*-
import unittest
""" Quicksort implementation """
def quicksort(arr):
""" Quicksort a list
@uolter
uolter / webgo.go
Created March 13, 2015 13:00
Dump http post with json
package main
import (
"encoding/json"
"io/ioutil"
"log"
"net/http"
)
type test_struct struct {
@uolter
uolter / init_flaskapp.sh
Created November 9, 2012 13:53
flask project dir structure
#!/bin/bash
APP_NAME="${1}"
if [ "$APP_NAME" = "" ]; then
echo "you must provide an application name!! "
echo "usage: <this script> <app name>"
exit
fi
@uolter
uolter / flaskapp_init.py
Last active October 12, 2015 15:18
flask application basic setup (python flaskapp_init.py -a <myapp> )
#!/usr/local/bin/python
from optparse import OptionParser
import os
PATH_SEPARATOR = os.sep
html_layout = "<!DOCTYPE html>\n" \
"<html>\n" \
@uolter
uolter / jsonp.py
Created November 26, 2012 09:16
jsonp decorator for python
'''
Created on Nov 23, 2012
@author: uolter
'''
from functools import wraps
from flask import request, current_app