-
-
Save jamesbowles/45a75de8696c34893b8cf560b9a875c3 to your computer and use it in GitHub Desktop.
Error Finder - Simple Ruby site crawler, finds non-200 response codes
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env ruby | |
# Simple non-200 finder, for finding errors and redirects | |
require 'rubygems' | |
require 'anemone' | |
url = ARGV[0] | |
if url.nil? | |
puts "Usage: error_finder.rb http://example.com" | |
exit | |
end | |
Anemone.crawl(url) do |anemone| | |
anemone.on_every_page do |page| | |
if !page.code.nil? and page.code > 200 | |
puts "[#{page.code}] #{page.url} - Referrer: #{page.referer}" | |
end | |
end | |
end |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment