Skip to content

Instantly share code, notes, and snippets.

View vagelim's full-sized avatar

vagelim vagelim

View GitHub Profile

Keybase proof

I hereby claim:

  • I am vagelim on github.
  • I am vageli (https://keybase.io/vageli) on keybase.
  • I have a public key whose fingerprint is A666 5608 1D75 8C56 AE63 7317 17C4 68D6 A639 3F33

To claim this, I am signing this object:

@vagelim
vagelim / find_nth.py
Last active November 30, 2015 18:56
Find the nth occurrence of a substring in a given string
def find_nth(haystack, needle, n):
"""Take a string (haystack) and find start the nth occurrence of the substring (needle)"""
start = haystack.find(needle)
while start >= 0 and n > 1:
start = haystack.find(needle, start+len(needle))
n -= 1
return start
@vagelim
vagelim / stress.sh
Created December 1, 2015 19:39
CPU stress test
#!/bin/bash
# Usage: lc [number_of_cpus_to_load [number_of_seconds] ]
lc() {
(
pids=""
cpus=${1:-1}
seconds=${2:-60}
echo loading $cpus CPUs for $seconds seconds
trap 'for p in $pids; do kill $p; done' 0
for ((i=0;i<cpus;i++)); do while : ; do : ; done & pids="$pids $!"; done
@vagelim
vagelim / amazon_scrape.py
Last active December 2, 2015 19:51
scrape Amazon multiple price page for lowest price
#Configuration variables
BASE_URL = "http://www.amazon.com/gp/offer-listing/"
XPATH_SELECTOR = '//body//*[@class="a-size-large a-color-price olpOfferPrice a-text-bold"]'
USER_AGENT = {'User-agent': 'Mozilla/5.0'}
ITEM = '1429218274'
#!/usr/bin/env bash
# **create-stack-user.sh**
# Create a user account suitable for running DevStack
# - create a group named $STACK_USER if it does not exist
# - create a user named $STACK_USER if it does not exist
#
# - home is $DEST
#
@vagelim
vagelim / findSubstring.py
Created February 19, 2016 20:54
Find all substrings
def locations_of_substring(string, substring):
"""Return a list of locations of a substring."""
substring_length = len(substring)
def recurse(locations_found, start):
location = string.find(substring, start)
if location != -1:
return recurse(locations_found + [location], location+substring_length)
else:
return locations_found
@vagelim
vagelim / burrow.cfg
Created February 25, 2016 17:01
Example Burrow configuration file for local deployment (bare minimum configuration)
[general]
logdir=/home/kafka/burrow/log
[zookeeper]
hostname=localhost
[kafka "local"]
broker=localhost
zookeeper=localhost
zookeeper-path=/kafka-cluster
@vagelim
vagelim / copytoredshift.py
Last active February 25, 2016 17:09
Luigi job to copy S3 data to Redshift
import luigi
from luigi import configuration
import datetime
from luigi.contrib import redshift
from mortar.luigi import mortartask
from dd_tasks import DatadogPigscriptTask
from luigi.s3 import S3Target, S3PathTask
from mortar.luigi.s3transfer import S3ToLocalTask
import dd_utils
#from redshift_utils import CopyToRedshiftTask
@vagelim
vagelim / jmx list_everything
Created March 4, 2016 18:40
output from jmx list_everything
vageli@kafka:~
=> sudo dd-agent jmx list_everything
2016-03-04 13:35:40,325 | WARNING | dd.collector | jmxfetch(jmxfetch.py:379) | {'host': 'localhost', 'port': 9999} doesn't have a 'conf' section. Only basic JVM metrics will be collected. See http://docs.datadoghq.com/integrations/java/ for more information
2016-03-04 13:35:40,326 | INFO | dd.collector | jmxfetch(jmxfetch.py:234) | Starting jmxfetch:
2016-03-04 13:35:40,327 | INFO | dd.collector | jmxfetch(jmxfetch.py:282) | Running java -Xms50m -Xmx200m -classpath /opt/datadog-agent/agent/checks/libs/jmxfetch-0.9.0-jar-with-dependencies.jar org.datadog.jmxfetch.App --check jmx.yaml kafka.yaml --check_period 15000 --conf_directory /etc/dd-agent/conf.d --log_level INFO --log_location /var/log/datadog/jmxfetch.log --reporter console --status_location /opt/datadog-agent/run/jmx_status.yaml list_everything
#####################################
Instance: localhost:9999
#####################################
@vagelim
vagelim / test_analysis.txt
Created March 17, 2016 16:29
Test analysis of #music
==SERVICES==
Spotify: 175
YouTube: 21
SoundCloud: 4
==========
====USERS====
raycolletti: 121
matt: 107
john: 66
scottanderson: 62