Skip to content

Instantly share code, notes, and snippets.

View stav's full-sized avatar
💭
c0d1ng

Steven Almeroth stav

💭
c0d1ng
View GitHub Profile
@stav
stav / clean
Created May 15, 2016 21:55
Remove Python bytecodes and deployment cache
#!/bin/bash
if [ -n "$1" ]; then
TARGET="$1"
else
TARGET="."
fi
# Remove Python compiled bytecode files
command="find $TARGET -name '*.pyc' -type f -delete -print 2>/dev/null";
@stav
stav / usbreset.c
Created May 15, 2016 21:46
Send a USB port reset to a USB device
/*
usbreset -- send a USB port reset to a USB device
http://askubuntu.com/questions/645#answer-661
cc usbreset.c -o usbreset
lsusb |grep DUB
Bus 001 Device 021: ID 2001:1a02 D-Link Corp. DUB-E100 Fast Ethernet Adapter(rev.C1) [ASIX AX88772]
@stav
stav / RouteSpider.py
Created August 14, 2015 19:56
Scrapy route spider pseudo code
class MySpider(RouteSpider):
name = "example.com"
def start_routes(self, response):
for city in response.css('#location .dropdown-bg li'):
yield Route(
url=city.xpath('a/@href'),
callback=self.parse_city,
@stav
stav / ScrapyLog.YAML-tmLanguage
Created August 13, 2015 19:19
Syntax Definition file for Scrapy log files
# [PackageDev] target_format: plist, ext: tmLanguage
---
name: Scrapy Log
scopeName: source.slog
fileTypes: [slog]
uuid: 030ae93f-5fcf-4bbd-8caa-81fcdcf77ec3
patterns:
- comment: date-time [logger] level
@stav
stav / usbreset.py
Last active August 29, 2015 14:23
send a USB port reset to a USB device
#!/usr/bin/env python
# usbreset -- send a USB port reset to a USB device
#
# http://askubuntu.com/questions/645#answer-661
#
# $ sudo usbreset.py DUB
# Looking for device: DUB
# Executing command: `lsusb | grep DUB`
# Subprocess: b'Bus 001 Device 006: ID 2001:1a02 D-Link Corp. DUB-E100 Fast...'
# Found device 001 on bus 006 for "D-Link Corp. DUB-E100 Fast Ethernet Adapt..."
@stav
stav / sitemap_generator_pipeline.py
Last active December 9, 2016 16:05
Scrapy sitemap generator pipeline
""" pipelines.py """
import collections
import scrapy
import scrapy.contrib.exporter
import myproject
class SitemapPipeline(object):
"""
Sitemap builder
@stav
stav / RouteSpider.py
Last active August 29, 2015 14:07
Scrapy spider with stream-lined routing and Item Loader processing
"""
Routed Crawler
"""
class Route(dict):
"""Spider route request"""
pass
class Router(scrapy.Spider):
"""Spider routing and loader handling"""
@stav
stav / rmpyc
Created July 10, 2014 19:34
rm Python bytecode files (.pyc)
#!/bin/bash
if [ -n "$1" ]; then
TARGET="$1"
else
TARGET="."
fi
command="find $TARGET -name '*.pyc' 2>/dev/null"
@stav
stav / ytclipr.py
Last active August 29, 2015 14:02
Python YouTube downloader: :see https://github.com/stav/clipy
# Python YouTube downloader
# 1. start script
# 2. copy youtube url into clipboard
# 3. press ctrl-d to start downloading
import sys
import pygtk
pygtk.require('2.0')
import gtk
@stav
stav / gist:5337954
Last active December 15, 2015 23:09
Scrapinghub API Job Log Sorter
import sys
import json
import argparse
from os.path import exists
from pprint import pprint
from urllib import urlencode, urlretrieve
from urllib2 import urlopen
from urlparse import urlsplit, parse_qs