Skip to content

Instantly share code, notes, and snippets.

Avatar

vagelim vagelim

View GitHub Profile
View keybase.md

Keybase proof

I hereby claim:

  • I am vagelim on github.
  • I am vageli (https://keybase.io/vageli) on keybase.
  • I have a public key whose fingerprint is A666 5608 1D75 8C56 AE63 7317 17C4 68D6 A639 3F33

To claim this, I am signing this object:

@vagelim
vagelim / find_nth.py
Last active Nov 30, 2015
Find the nth occurrence of a substring in a given string
View find_nth.py
def find_nth(haystack, needle, n):
"""Take a string (haystack) and find start the nth occurrence of the substring (needle)"""
start = haystack.find(needle)
while start >= 0 and n > 1:
start = haystack.find(needle, start+len(needle))
n -= 1
return start
@vagelim
vagelim / stress.sh
Created Dec 1, 2015
CPU stress test
View stress.sh
#!/bin/bash
# Usage: lc [number_of_cpus_to_load [number_of_seconds] ]
lc() {
(
pids=""
cpus=${1:-1}
seconds=${2:-60}
echo loading $cpus CPUs for $seconds seconds
trap 'for p in $pids; do kill $p; done' 0
for ((i=0;i<cpus;i++)); do while : ; do : ; done & pids="$pids $!"; done
@vagelim
vagelim / amazon_scrape.py
Last active Dec 2, 2015
scrape Amazon multiple price page for lowest price
View amazon_scrape.py
#Configuration variables
BASE_URL = "http://www.amazon.com/gp/offer-listing/"
XPATH_SELECTOR = '//body//*[@class="a-size-large a-color-price olpOfferPrice a-text-bold"]'
USER_AGENT = {'User-agent': 'Mozilla/5.0'}
ITEM = '1429218274'
View create-stack-user
#!/usr/bin/env bash
# **create-stack-user.sh**
# Create a user account suitable for running DevStack
# - create a group named $STACK_USER if it does not exist
# - create a user named $STACK_USER if it does not exist
#
# - home is $DEST
#
@vagelim
vagelim / findSubstring.py
Created Feb 19, 2016
Find all substrings
View findSubstring.py
def locations_of_substring(string, substring):
"""Return a list of locations of a substring."""
substring_length = len(substring)
def recurse(locations_found, start):
location = string.find(substring, start)
if location != -1:
return recurse(locations_found + [location], location+substring_length)
else:
return locations_found
@vagelim
vagelim / burrow.cfg
Created Feb 25, 2016
Example Burrow configuration file for local deployment (bare minimum configuration)
View burrow.cfg
[general]
logdir=/home/kafka/burrow/log
[zookeeper]
hostname=localhost
[kafka "local"]
broker=localhost
zookeeper=localhost
zookeeper-path=/kafka-cluster
@vagelim
vagelim / copytoredshift.py
Last active Feb 25, 2016
Luigi job to copy S3 data to Redshift
View copytoredshift.py
import luigi
from luigi import configuration
import datetime
from luigi.contrib import redshift
from mortar.luigi import mortartask
from dd_tasks import DatadogPigscriptTask
from luigi.s3 import S3Target, S3PathTask
from mortar.luigi.s3transfer import S3ToLocalTask
import dd_utils
#from redshift_utils import CopyToRedshiftTask
@vagelim
vagelim / jmx list_everything
Created Mar 4, 2016
output from jmx list_everything
View jmx list_everything
vageli@kafka:~
=> sudo dd-agent jmx list_everything
2016-03-04 13:35:40,325 | WARNING | dd.collector | jmxfetch(jmxfetch.py:379) | {'host': 'localhost', 'port': 9999} doesn't have a 'conf' section. Only basic JVM metrics will be collected. See http://docs.datadoghq.com/integrations/java/ for more information
2016-03-04 13:35:40,326 | INFO | dd.collector | jmxfetch(jmxfetch.py:234) | Starting jmxfetch:
2016-03-04 13:35:40,327 | INFO | dd.collector | jmxfetch(jmxfetch.py:282) | Running java -Xms50m -Xmx200m -classpath /opt/datadog-agent/agent/checks/libs/jmxfetch-0.9.0-jar-with-dependencies.jar org.datadog.jmxfetch.App --check jmx.yaml kafka.yaml --check_period 15000 --conf_directory /etc/dd-agent/conf.d --log_level INFO --log_location /var/log/datadog/jmxfetch.log --reporter console --status_location /opt/datadog-agent/run/jmx_status.yaml list_everything
#####################################
Instance: localhost:9999
#####################################
@vagelim
vagelim / test_analysis.txt
Created Mar 17, 2016
Test analysis of #music
View test_analysis.txt
==SERVICES==
Spotify: 175
YouTube: 21
SoundCloud: 4
==========
====USERS====
raycolletti: 121
matt: 107
john: 66
scottanderson: 62