Skip to content

Instantly share code, notes, and snippets.

@3dd13
3dd13 / batch_concurrent_requests_by_typhoeus.rb
Created April 22, 2010 09:00
example of using Typhoeus
def main
# urls is a hash containing url to retrieve statuses of items
get_csv(urls, items) { |response, item|
item_results = CSV.parse_line(response.body)
item.setStatus(item_results[0])
}
end
def process_response(urls, items)
@3dd13
3dd13 / ftp_client.py
Created May 19, 2010 16:43
VXRL - Defcon 2010 preparation
import ftplib
from ftplib import FTP
buffer = '\x41' * 400
try:
ftp = FTP('127.0.0.1')
ftp.login(user, password)
ftp.transfercmd("command" + buffer)
print 'xxx'
@3dd13
3dd13 / download_open_rice_contact.rb
Created October 19, 2010 08:36
OpenRice.com is a famous restaurant online directory in Hong Kong. Here is a ruby script to grep all the contact information of all shops. at this moment of time, there are around 26K records on the website. this program took ~ 15mins to finish. and
require 'rubygems'
require 'mechanize'
require 'fastercsv'
def get_all_option_values(page, attr_name)
page.search("[name=#{attr_name}]").search("option").map do |opt|
option_value = opt.attribute("value").value
[option_value, opt.text.strip.gsub(/\302\240\302\240/, '')] if option_value && option_value.length > 0
end.compact
end
@3dd13
3dd13 / css_xpath_selector_benchmarking.rb
Created November 5, 2010 04:00
just curious about CSS and XPath selector performance. did some benchmarking
require 'rubygems'
require 'mechanize'
require 'benchmark'
agent = Mechanize.new
# however, this fetch already spend around 2 seconds. one http request response.
page = agent.get('http://www.ufood.com.hk/search/search.action?name=&distIds=4,10,11,3,5,8,6,7,2,1,9,12,13,14,15,18,16,17,19,20,21,22,23,24,25,26,33,34,27,35,32,30,29,28,31&ftIds=')
Benchmark.bm do |b|
# search by tag name and class 10.470000 0.040000 10.510000 ( 11.149709)
@3dd13
3dd13 / 1_controller_test.rb
Created January 7, 2011 07:04
sharing on functional test refactoring with code block. cleaner and more readable.
test "not allow to update state after account reconciliation" do
# get fixture
account = accounts(:fresh)
# submit and expect no errors
post :waive_fee, :id => account.id
assert_nil flash[:error]
post :add_fee, :id => account.id
assert_nil flash[:error]
@3dd13
3dd13 / generate_ssh_keys.rb
Created January 20, 2011 16:59
chef recipe to generate ssh key for a user
define :generate_ssh_keys, :user_account => nil do
username = params[:user_account]
raise ":user_account should be provided." if username.nil?
Chef::Log.debug("generate ssh skys for #{username}.")
execute "generate ssh skys for #{username}." do
user username
creates "/home/#{username}/.ssh/id_rsa.pub"
@3dd13
3dd13 / stock_daily_price.rb
Created January 21, 2011 02:39
Download stock price data from yahoo and insert them into database
require 'rubygems'
require 'active_record'
require 'yaml'
dbconfig = YAML::load(File.new(File.join(File.dirname(__FILE__), "../../config/database.yml")))
ActiveRecord::Base.establish_connection(dbconfig["development"])
class StockDailyPrice < ActiveRecord::Base
def initialize(stock_values, stock_no = "")
super()
@3dd13
3dd13 / log_rotation.rb
Created January 21, 2011 02:46
Log Rotation methods provided by Ruby Core http://ruby-doc.org/ruby-1.9/classes/Logger.html
def logger
@logger ||= Logger.new(File.new("data_loader.log"),3,5*1024*1024)
end
@3dd13
3dd13 / clear_log.rb
Created January 27, 2011 06:34
Adding log purging job using chef
cron "housekeep application log files" do
user "housekeeper"
# mailto "your@example.com"
# 00:01 every night
hour "0"
minute "1"
# clean up logs older than 30 days
command "/find ~/log/application-*.log.tar.gz -mtime +30 -exec rm {} \;"
@3dd13
3dd13 / row_is_cut_in_the_middle_solution.rb
Created February 18, 2011 07:49
overflow / page break examples of Prawn Pdf generation.
# gem "prawn", "0.8.4"
# Sometime there is no enough space for the last table row when it reaches the end of page
Prawn::Document.generate("text_group_overflow_question.pdf") do |pdf|
add_page_break_if_overflow(pdf) do |pdf|
# generating table here
# ...
end
end