Skip to content

Instantly share code, notes, and snippets.

@seki seki/crawler.rb
Created Feb 19, 2019

Embed
What would you like to do?
require 'thread'
class Crawler
def initialize(root='.', pattern='**/*.rb', period=5)
@root = File.expand_path(root)
@pattern = pattern
@period = period
@last = {}
@queue = SizedQueue.new(2)
end
def pop
@queue.pop
end
def updated?(fname)
mtime = File.mtime(fname)
mtime != @last[fname]
ensure
@last[fname] = mtime
end
def do_crawl
Dir.chdir(@root)
Dir.glob(@pattern).each do |fname|
@queue.push(fname) if updated?(fname)
end
end
def stop
@running = false
end
def start
@running = true
Thread.new do
while @running
do_crawl
sleep(@period)
end
end
end
end
c = Crawler.new('.', '**/*.rb')
c.start
10.times do
p c.pop
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.