Skip to content

Instantly share code, notes, and snippets.

View stav's full-sized avatar
💭
c0d1ng

Steven Almeroth stav

💭
c0d1ng
View GitHub Profile
@stav
stav / ytclipr.py
Last active August 29, 2015 14:02
Python YouTube downloader: :see https://github.com/stav/clipy
# Python YouTube downloader
# 1. start script
# 2. copy youtube url into clipboard
# 3. press ctrl-d to start downloading
import sys
import pygtk
pygtk.require('2.0')
import gtk
@stav
stav / rmpyc
Created July 10, 2014 19:34
rm Python bytecode files (.pyc)
#!/bin/bash
if [ -n "$1" ]; then
TARGET="$1"
else
TARGET="."
fi
command="find $TARGET -name '*.pyc' 2>/dev/null"
@stav
stav / RouteSpider.py
Last active August 29, 2015 14:07
Scrapy spider with stream-lined routing and Item Loader processing
"""
Routed Crawler
"""
class Route(dict):
"""Spider route request"""
pass
class Router(scrapy.Spider):
"""Spider routing and loader handling"""
@stav
stav / usbreset.py
Last active August 29, 2015 14:23
send a USB port reset to a USB device
#!/usr/bin/env python
# usbreset -- send a USB port reset to a USB device
#
# http://askubuntu.com/questions/645#answer-661
#
# $ sudo usbreset.py DUB
# Looking for device: DUB
# Executing command: `lsusb | grep DUB`
# Subprocess: b'Bus 001 Device 006: ID 2001:1a02 D-Link Corp. DUB-E100 Fast...'
# Found device 001 on bus 006 for "D-Link Corp. DUB-E100 Fast Ethernet Adapt..."
@stav
stav / ScrapyLog.YAML-tmLanguage
Created August 13, 2015 19:19
Syntax Definition file for Scrapy log files
# [PackageDev] target_format: plist, ext: tmLanguage
---
name: Scrapy Log
scopeName: source.slog
fileTypes: [slog]
uuid: 030ae93f-5fcf-4bbd-8caa-81fcdcf77ec3
patterns:
- comment: date-time [logger] level
@stav
stav / RouteSpider.py
Created August 14, 2015 19:56
Scrapy route spider pseudo code
class MySpider(RouteSpider):
name = "example.com"
def start_routes(self, response):
for city in response.css('#location .dropdown-bg li'):
yield Route(
url=city.xpath('a/@href'),
callback=self.parse_city,
@stav
stav / gist:3520611
Created August 29, 2012 23:54
Google Places API Search
#!/usr/bin/python
# Google Places Search
#
# Use the Google Places API to text search for the supplied keywords and output
# the first result to standard out.
import sys
import json
import argparse
@stav
stav / gist:4191165
Created December 2, 2012 21:33
Generic PHP debug printer
<?php
/**
* generic debug printer
*
* Because I didn't like having to pass two arguments to a debug printer, namely the
* evaluated and un-evaluated expressions, like: $baker->bread and "baker.bread", i.e.,
* I only wanted to pass the un-evaluated string and let the print routine do the
* evaluating. This script does that proceduraly, i.e., not in a function, so expression
* scope is not changed.
*
@stav
stav / gist:5137869
Last active December 14, 2015 19:39
Scrapy blocking spider that renders JavaScript with PyQt4
from PyQt4.QtCore import QUrl
from PyQt4.QtGui import QApplication
from PyQt4.QtWebKit import QWebPage
from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor
from scrapy.spider import BaseSpider
from scrapy.http import HtmlResponse
class Render(QWebPage):
def __init__(self, url):
@stav
stav / gist:5152476
Last active December 14, 2015 21:39
Crawler project running Scrapy from a script
# main.py:
from project.spiders.log_test import TestSpider as EstiloMASpider
from scrapy.xlib.pydispatch import dispatcher
from scrapy.crawler import Crawler
from twisted.internet import reactor
from scrapy.utils.project import get_project_settings
from scrapy import log, signals