Skip to content

Instantly share code, notes, and snippets.

@nyov
nyov / requests_from_generator.py
Created February 28, 2014 06:38
scrapy ranged start_requests from generator
class MySpider(Spider):
# [...]
# start requests from generator
def start_requests(self):
url = 'http://some.page.tld/%s/category'
for page in xrange(1, 247):
link = url % page
yield Request(url=link)
@IPrism
IPrism / gist:1010289
Created June 6, 2011 13:46 — forked from matiaskorhonen/gist:420880
startup script for unicorn with rvm global installation
#!/bin/bash
### BEGIN INIT INFO
# Provides: APPLICATION
# Required-Start: $all
# Required-Stop: $network $local_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Start the APPLICATION unicorns at boot
# Description: Enable APPLICATION at boot time.
### END INIT INFO
@zh
zh / redis-em-patch
Created September 12, 2011 10:07 — forked from gregorymostizky/redis-em-patch
Patch redis and em-redis to work together
require 'em-redis'
require 'redis'
require 'redis/distributed'
require "fiber_pool"
class Redis
class Distributed
def initialize(urls, options = {})
@tag = options.delete(:tag) || /^\{(.+?)\}/
gonz@bamboo:~ > git clone git://github.com/maccman/juggernaut.git
Cloning into juggernaut...
remote: Counting objects: 558, done.
remote: Compressing objects: 100% (258/258), done.
remote: Total 558 (delta 309), reused 481 (delta 255)
Receiving objects: 100% (558/558), 576.74 KiB | 135 KiB/s, done.
Resolving deltas: 100% (309/309), done.
gonz@bamboo:~ > cd juggernaut
@dblock
dblock / mongoid_criteria.rb
Created May 24, 2012 19:38
Mongoid::Criteria each_by iterator
module Mongoid
class Criteria
def each_by(by, &block)
idx = 0
total = 0
set_limit = options[:limit]
while ((results = ordered_clone.limit(by).skip(idx)) && results.any?)
results.each do |result|
return self if set_limit and set_limit >= total
@csanz
csanz / whattobuy.md
Created November 22, 2012 16:30
Drone Specs

#Drone Specs

Space Specs

  • Flat Area
  • Great wifi
  • Hopefully High ceilings without any crazy wires (eg. 120ft high ceilings in Berlin)

Spare parts

@yankov
yankov / gist:4230948
Last active October 13, 2015 17:28
Creating new Scala project with sbt
mkdir project1
cd project1
sbt
set name := "project1"
set version := "1.0"
set scalaVersion := "2.9.3"
session save
exit
@tovbinm
tovbinm / ch5_streams.scala
Last active December 26, 2015 13:59
Reading Functional Programing In Scala
// Excercies from chapter 5: Strictness and laziness
def foldRight[A,B](z: => B)(s: Stream[A])(f: (A, => B) => B): B =
s match {
case hd #:: tail => f(hd, foldRight(z)(tail)(f))
case _ => z
}
def exists[A](s: Stream[A])(p: A => Boolean): Boolean =
foldRight(false)(s)((x,a) => if (p(x)) true else a)
@tovbinm
tovbinm / waves.js
Created January 26, 2014 05:24
Breaking waves sound (Open (http://myjavatools.com/js.html) in Chrome paste the code and hit run)
var AMP = 0.5; // amplitude
var len = 15; // seconds
e = new webkitAudioContext();
var source = e.createBufferSource();
var SR = e.sampleRate;
source.buffer = e.createBuffer(1, len * SR, SR);
var dat = source.buffer.getChannelData(0);
for (var i = 0; i < len * SR; i++) {
dat[i] = AMP * (Math.random() * 2 - 1);
}
@pprett
pprett / grid_search.py
Created October 31, 2012 19:42
Parallel grid search for sklearn Gradient Boosting
"""Parallel grid search for sklearn's GradientBoosting.
This script uses IPython.parallel to run cross-validated
grid search on an IPython cluster. Each cell on the parameter grid
will be evaluated ``K`` times - results are stored in MongoDB.
The procedure tunes the number of trees ``n_estimators`` by averaging
the staged scores of the GBRT model averaged over all K folds.
You need an IPython ipcluster to connect to - for local use simply