Skip to content

Instantly share code, notes, and snippets.

@hiroyuki-sato

hiroyuki-sato/apache_log_ruby.rb

Last active September 3, 2015 08:56
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save hiroyuki-sato/23b1861934efb27fc7b6 to your computer and use it in GitHub Desktop.
Save hiroyuki-sato/23b1861934efb27fc7b6 to your computer and use it in GitHub Desktop.
Download ZIP
Apache Log Parser Plugin
Raw
apache_log_ruby.rb
require 'apache_log/parser'
module Embulk
module Parser
class Apache_log_rubyParserPlugin < ParserPlugin
Plugin.register_parser("apache_log_ruby", self)
STRING_FIELDS = %w[ remote_host identity_check user status size ].map{|x| x.to_sym }
REQUEST_FIELDS = %w[ method path protocol ].map{ |x| x.to_sym }
COMBINED_ADD_FIELDS = %w[ referer user_agent ].map{ |x| x.to_sym }
LOG_FORMATS = %w[ common combined ]
def self.transaction(config, &control)
# configuration code:
log_format = config.param("log_format", :string).downcase
task = {
"log_format" => log_format
}
raise "Unknown log format" unless LOG_FORMATS.include?(log_format)
columns = [
Column.new(0, "remote_host", :string),
Column.new(1, "identity_check", :string),
Column.new(2, "user", :string),
Column.new(3, "status", :string),
Column.new(4, "size", :string),
Column.new(5, "datetime", :timestamp),
Column.new(6, "method", :string),
Column.new(7, "path", :string),
Column.new(8, "protocol", :string),
]
if( log_format == 'combined' )
columns += [
Column.new(9, "referer", :string),
Column.new(10,"user_agent", :string)
]
end
yield(task, columns)
end
def init
# initialization code:
@log_format = @task["log_format"]
@parser = ApacheLog::Parser.new(@log_format)
end
def run(file_input)
decoder_task = @task.load_config(Java::LineDecoder::DecoderTask)
decoder = Java::LineDecoder.new(file_input.instance_eval { @java_file_input }, decoder_task)
while decoder.nextFile
puts "parsing a file started"
while line = decoder.poll
record = []
begin
#puts "line: #{line}"
log = @parser.parse(line)
STRING_FIELDS.each { |x| record << log[x] }
record << log[:datetime].to_time
REQUEST_FIELDS.each { |x| record << log[:request][x] }
if( @log_format == 'combined' )
COMBINED_ADD_FIELDS.each { |x| record << log[x] }
end
@page_builder.add(record)
rescue
puts "*** ERROR ***"
puts $!.to_s
puts $!.backtrace
# TODO
end
end
puts "parsing a file ended"
end
@page_builder.finish
end
end
end
end
Raw
config.yml
exec: {}
in:
type: file
path_prefix: "/home/embulk/httpd/test_"
parser:
type: apache_log_ruby
log_format: combined
out: {type: stdout}
Raw
preview.txt
[embulk@bear02 ~]$ java -jar embulk.jar preview apache.yml
2015-02-21 16:07:48,371 +0900: Embulk v0.4.5
2015-02-21 16:07:50.173 +0900 [INFO] (preview): Listing local files at directory '/home/embulk/httpd' filtering filename by prefix 'test_'
2015-02-21 16:07:50.175 +0900 [INFO] (preview): Loading files [/home/embulk/httpd/test_001.dat]
parsing a file started
parsing a file ended
+--------------------+-----------------------+-------------+---------------+-------------+-------------------------+---------------+---------------------------------+-----------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| remote_host:string | identity_check:string | user:string | status:string | size:string | datetime:timestamp | method:string | path:string | protocol:string | referer:string | user_agent:string |
+--------------------+-----------------------+-------------+---------------+-------------+-------------------------+---------------+---------------------------------+-----------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 52.45.63.172 | - | - | 200 | 62 | 2015-02-18 15:39:22 UTC | GET | /category/games | HTTP/1.1 | - | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0.1) Gecko/20100101 Firefox/9.0.1 |
| 172.135.109.67 | - | - | 200 | 136 | 2015-02-18 15:39:22 UTC | GET | /category/networking | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.46 Safari/535.11 |
| 104.27.31.209 | - | - | 200 | 81 | 2015-02-18 15:39:22 UTC | GET | /category/software | HTTP/1.1 | /category/music | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 108.162.39.125 | - | - | 200 | 136 | 2015-02-18 15:39:22 UTC | GET | /category/music | HTTP/1.1 | - | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0) |
| 200.18.154.223 | - | - | 200 | 104 | 2015-02-18 15:39:22 UTC | GET | /category/games | HTTP/1.1 | /item/music/4449 | Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 28.204.76.179 | - | - | 200 | 43 | 2015-02-18 15:39:22 UTC | GET | /item/books/2667 | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7 |
| 104.204.137.111 | - | - | 200 | 50 | 2015-02-18 15:39:22 UTC | GET | /item/books/4604 | HTTP/1.1 | /item/garden/2362 | Mozilla/5.0 (Windows NT 6.0; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 220.48.187.80 | - | - | 200 | 104 | 2015-02-18 15:39:22 UTC | GET | /category/sports | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.0; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 192.93.51.174 | - | - | 200 | 99 | 2015-02-18 15:39:22 UTC | GET | /category/software | HTTP/1.1 | /category/electronics | Mozilla/5.0 (Windows NT 6.0; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 188.144.89.139 | - | - | 200 | 133 | 2015-02-18 15:39:22 UTC | GET | /category/software | HTTP/1.1 | /item/books/1528 | Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 28.105.165.132 | - | - | 200 | 119 | 2015-02-18 15:39:22 UTC | POST | /seembulk/?c=Electronics+Games | HTTP/1.1 | /category/books | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 20.165.180.44 | - | - | 200 | 42 | 2015-02-18 15:39:22 UTC | GET | /category/books?from=10 | HTTP/1.1 | /category/cameras | Mozilla/5.0 (Windows NT 6.0; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 220.96.48.25 | - | - | 200 | 40 | 2015-02-18 15:39:22 UTC | GET | /item/electronics/1101 | HTTP/1.1 | /seembulk/?c=Electronics | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 96.39.121.184 | - | - | 200 | 63 | 2015-02-18 15:39:22 UTC | GET | /category/office | HTTP/1.1 | /item/games/1415 | Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 32.186.200.141 | - | - | 200 | 120 | 2015-02-18 15:39:22 UTC | GET | /category/games | HTTP/1.1 | - | Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; WOW64; Trident/4.0; GTB6; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30618; .NET4.0C) |
| 36.45.51.151 | - | - | 200 | 66 | 2015-02-18 15:39:22 UTC | GET | /item/books/2520 | HTTP/1.1 | - | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 176.153.109.97 | - | - | 200 | 134 | 2015-02-18 15:39:22 UTC | GET | /item/software/3738 | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 52.228.196.181 | - | - | 200 | 118 | 2015-02-18 15:39:22 UTC | GET | /category/finance | HTTP/1.1 | - | Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
| 76.189.201.119 | - | - | 200 | 72 | 2015-02-18 15:39:22 UTC | POST | /seembulk/?c=Office | HTTP/1.1 | /category/software?from=20 | Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 20.153.132.76 | - | - | 200 | 45 | 2015-02-18 15:39:22 UTC | GET | /category/electronics | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.0; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 24.207.52.200 | - | - | 200 | 66 | 2015-02-18 15:39:22 UTC | GET | /item/games/4922 | HTTP/1.1 | /category/toys | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0.1) Gecko/20100101 Firefox/9.0.1 |
| 204.192.41.209 | - | - | 200 | 102 | 2015-02-18 15:39:22 UTC | GET | /category/office?from=10 | HTTP/1.1 | /category/electronics | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 28.180.78.24 | - | - | 200 | 88 | 2015-02-18 15:39:22 UTC | GET | /category/games | HTTP/1.1 | /category/office | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0) |
| 64.186.165.28 | - | - | 200 | 135 | 2015-02-18 15:39:22 UTC | GET | /category/books | HTTP/1.1 | /category/software?from=0 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0.1) Gecko/20100101 Firefox/9.0.1 |
| 164.141.30.168 | - | - | 200 | 91 | 2015-02-18 15:39:22 UTC | POST | /seembulk/?c=Computers | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 116.63.25.180 | - | - | 200 | 106 | 2015-02-18 15:39:22 UTC | POST | /seembulk/?c=Software | HTTP/1.1 | - | Mozilla/5.0 (iPad; CPU OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A405 Safari/7534.48.3 |
| 92.45.108.100 | - | - | 200 | 136 | 2015-02-18 15:39:22 UTC | GET | /item/electronics/1545 | HTTP/1.1 | http://www.google.com/seembulk?ie=UTF-8&q=google&sclient=psy-ab&q=Electronics+Software&oq=Electronics+Software&aq=f&aqi=g-vL1&aql=&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.,cf.osb&biw=2941&bih=232 | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 80.153.20.67 | - | - | 200 | 93 | 2015-02-18 15:39:22 UTC | GET | /item/sports/4013 | HTTP/1.1 | /seembulk/?c=Sports | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0.1) Gecko/20100101 Firefox/9.0.1 |
| 80.102.150.100 | - | - | 200 | 61 | 2015-02-18 15:39:22 UTC | GET | /category/software | HTTP/1.1 | /item/networking/2323 | Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A405 Safari/7534.48.3 |
| 164.150.80.43 | - | - | 200 | 54 | 2015-02-18 15:39:22 UTC | GET | /category/sports | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7 |
| 20.78.167.195 | - | - | 200 | 125 | 2015-02-18 15:39:22 UTC | POST | /seembulk/?c=Software | HTTP/1.1 | /category/electronics | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 44.111.94.132 | - | - | 200 | 120 | 2015-02-18 15:39:22 UTC | GET | /item/books/1504 | HTTP/1.1 | /item/software/2314 | Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A405 Safari/7534.48.3 |
| 124.102.94.150 | - | - | 200 | 128 | 2015-02-18 15:39:22 UTC | GET | /category/office | HTTP/1.1 | /category/sports | Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 224.162.86.224 | - | - | 200 | 66 | 2015-02-18 15:39:22 UTC | GET | /item/software/2127 | HTTP/1.1 | http://www.google.com/seembulk?ie=UTF-8&q=google&sclient=psy-ab&q=Software&oq=Software&aq=f&aqi=g-vL1&aql=&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.,cf.osb&biw=1543&bih=9 | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 96.138.38.144 | - | - | 200 | 59 | 2015-02-18 15:39:22 UTC | GET | /category/electronics | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.0; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 188.189.150.34 | - | - | 200 | 121 | 2015-02-18 15:39:22 UTC | GET | /item/electronics/3053 | HTTP/1.1 | http://www.google.com/seembulk?ie=UTF-8&q=google&sclient=psy-ab&q=Electronics+Books&oq=Electronics+Books&aq=f&aqi=g-vL1&aql=&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.,cf.osb&biw=2888&bih=145 | Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; YTB730; GTB7.2; EasyBits GO v1.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C) |
| 136.150.148.228 | - | - | 200 | 51 | 2015-02-18 15:39:22 UTC | GET | /category/giftcards | HTTP/1.1 | /category/games | Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; WOW64; Trident/4.0; GTB6; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30618; .NET4.0C) |
| 88.126.203.78 | - | - | 200 | 114 | 2015-02-18 15:39:22 UTC | GET | /item/office/2188 | HTTP/1.1 | /category/games | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 188.39.213.128 | - | - | 200 | 93 | 2015-02-18 15:39:22 UTC | GET | /category/games?from=10 | HTTP/1.1 | - | Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; YTB730; GTB7.2; EasyBits GO v1.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C) |
| 140.105.139.182 | - | - | 200 | 136 | 2015-02-18 15:39:22 UTC | GET | /item/computers/4530 | HTTP/1.1 | http://www.google.com/seembulk?ie=UTF-8&q=google&sclient=psy-ab&q=Computers+Cameras&oq=Computers+Cameras&aq=f&aqi=g-vL1&aql=&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.,cf.osb&biw=3595&bih=443 | Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; YTB730; GTB7.2; EasyBits GO v1.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C) |
| 152.171.21.176 | - | - | 200 | 65 | 2015-02-18 15:39:22 UTC | GET | /category/health | HTTP/1.1 | /item/electronics/2187 | Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.46 Safari/535.11 |
| 204.210.112.49 | - | - | 200 | 83 | 2015-02-18 15:39:22 UTC | POST | /seembulk/?c=Garden+Office | HTTP/1.1 | /category/sports | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0.1) Gecko/20100101 Firefox/9.0.1 |
| 92.180.171.138 | - | - | 200 | 86 | 2015-02-18 15:39:22 UTC | GET | /category/books | HTTP/1.1 | /seembulk/?c=Music+Electronics | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7 |
| 160.63.71.188 | - | - | 200 | 81 | 2015-02-18 15:39:22 UTC | GET | /category/office | HTTP/1.1 | /category/books | Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; YTB730; GTB7.2; EasyBits GO v1.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C) |
| 68.45.95.160 | - | - | 200 | 108 | 2015-02-18 15:39:22 UTC | GET | /item/networking/3800 | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.0; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 88.192.53.178 | - | - | 200 | 57 | 2015-02-18 15:39:22 UTC | GET | /category/books?from=20 | HTTP/1.1 | /category/books | Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 28.66.100.65 | - | - | 200 | 52 | 2015-02-18 15:39:22 UTC | GET | /item/electronics/1869 | HTTP/1.1 | /category/toys | Mozilla/5.0 (Windows NT 6.0; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 104.42.157.31 | - | - | 200 | 89 | 2015-02-18 15:39:22 UTC | GET | /category/electronics | HTTP/1.1 | /item/games/3443 | Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 164.132.54.32 | - | - | 200 | 109 | 2015-02-18 15:39:22 UTC | GET | /category/games | HTTP/1.1 | /category/games | Mozilla/5.0 (Windows NT 6.0; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 196.192.82.207 | - | - | 200 | 85 | 2015-02-18 15:39:22 UTC | GET | /category/software | HTTP/1.1 | - | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 32.159.48.26 | - | - | 200 | 68 | 2015-02-18 15:39:22 UTC | GET | /category/electronics | HTTP/1.1 | - | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0) |
| 160.168.181.59 | - | - | 200 | 117 | 2015-02-18 15:39:22 UTC | GET | /item/toys/3764 | HTTP/1.1 | /seembulk/?c=Toys+Electronics | Mozilla/5.0 (Windows NT 6.0; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 192.30.113.180 | - | - | 200 | 130 | 2015-02-18 15:39:22 UTC | GET | /item/jewelry/2704 | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.46 Safari/535.11 |
| 204.57.196.30 | - | - | 200 | 104 | 2015-02-18 15:39:22 UTC | GET | /category/toys | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 124.27.84.163 | - | - | 200 | 83 | 2015-02-18 15:39:22 UTC | GET | /category/software | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7 |
| 68.198.27.41 | - | - | 200 | 51 | 2015-02-18 15:39:22 UTC | GET | /category/software?from=10 | HTTP/1.1 | /item/office/324 | Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; WOW64; Trident/4.0; GTB6; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30618; .NET4.0C) |
| 136.201.134.207 | - | - | 200 | 62 | 2015-02-18 15:39:22 UTC | GET | /category/toys | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7 |
| 20.129.221.97 | - | - | 200 | 66 | 2015-02-18 15:39:22 UTC | GET | /item/finance/2638 | HTTP/1.1 | /category/office | Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; YTB730; GTB7.2; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; .NET4.0E; Media Center PC 6.0) |
| 100.87.81.183 | - | - | 200 | 129 | 2015-02-18 15:39:22 UTC | GET | /item/cameras/4587 | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 64.186.165.28 | - | - | 200 | 137 | 2015-02-18 15:39:22 UTC | GET | /item/software/2275 | HTTP/1.1 | /item/electronics/4524 | Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 100.171.113.188 | - | - | 200 | 82 | 2015-02-18 15:39:22 UTC | GET | /category/software | HTTP/1.1 | - | Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; BTRS122159; GTB7.2; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; BRI/2) |
| 52.96.148.130 | - | - | 200 | 49 | 2015-02-18 15:39:22 UTC | GET | /category/finance | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 172.192.77.177 | - | - | 200 | 49 | 2015-02-18 15:39:22 UTC | GET | /category/music | HTTP/1.1 | /category/games | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0.1) Gecko/20100101 Firefox/9.0.1 |
| 64.132.101.26 | - | - | 200 | 139 | 2015-02-18 15:39:22 UTC | GET | /category/electronics | HTTP/1.1 | /item/office/2111 | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0) |
| 40.141.160.146 | - | - | 200 | 66 | 2015-02-18 15:39:22 UTC | GET | /item/software/2127 | HTTP/1.1 | http://www.google.com/seembulk?ie=UTF-8&q=google&sclient=psy-ab&q=Software&oq=Software&aq=f&aqi=g-vL1&aql=&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.,cf.osb&biw=1543&bih=9 | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 48.204.211.31 | - | - | 200 | 111 | 2015-02-18 15:39:22 UTC | POST | /seembulk/?c=Jewelry | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7 |
| 36.201.37.188 | - | - | 200 | 100 | 2015-02-18 15:39:22 UTC | GET | /category/books | HTTP/1.1 | - | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 224.39.187.81 | - | - | 200 | 78 | 2015-02-18 15:39:22 UTC | GET | /category/electronics | HTTP/1.1 | /category/health | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 116.66.24.175 | - | - | 200 | 129 | 2015-02-18 15:39:22 UTC | GET | /item/cameras/4587 | HTTP/1.1 | /seembulk/?c=Cameras+Electronics | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 104.90.221.170 | - | - | 200 | 67 | 2015-02-18 15:39:22 UTC | GET | /item/jewelry/2894 | HTTP/1.1 | /category/jewelry | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0) |
| 32.36.124.60 | - | - | 200 | 130 | 2015-02-18 15:39:22 UTC | GET | /category/electronics | HTTP/1.1 | /category/software | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 216.153.25.126 | - | - | 200 | 104 | 2015-02-18 15:39:22 UTC | GET | /category/music | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7 |
| 200.117.160.140 | - | - | 200 | 116 | 2015-02-18 15:39:22 UTC | GET | /category/sports | HTTP/1.1 | - | Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
| 108.201.162.172 | - | - | 200 | 95 | 2015-02-18 15:39:22 UTC | GET | /category/electronics | HTTP/1.1 | /seembulk/?c=Games+Garden | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 24.126.180.51 | - | - | 200 | 121 | 2015-02-18 15:39:22 UTC | GET | /category/software | HTTP/1.1 | /category/games?from=10 | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 224.171.162.61 | - | - | 200 | 73 | 2015-02-18 15:39:22 UTC | GET | /category/games | HTTP/1.1 | - | Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
| 172.165.166.193 | - | - | 200 | 50 | 2015-02-18 15:39:22 UTC | GET | /item/networking/623 | HTTP/1.1 | http://www.google.com/seembulk?ie=UTF-8&q=google&sclient=psy-ab&q=Networking+Cameras&oq=Networking+Cameras&aq=f&aqi=g-vL1&aql=&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.,cf.osb&biw=3975&bih=209 | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7 |
| 108.93.97.92 | - | - | 200 | 46 | 2015-02-18 15:39:22 UTC | GET | /item/jewelry/447 | HTTP/1.1 | /item/toys/2914 | Mozilla/5.0 (iPad; CPU OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A405 Safari/7534.48.3 |
| 120.138.99.20 | - | - | 200 | 59 | 2015-02-18 15:39:22 UTC | POST | /seembulk/?c=Electronics+Office | HTTP/1.1 | /category/electronics | Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A405 Safari/7534.48.3 |
| 156.132.142.115 | - | - | 200 | 95 | 2015-02-18 15:39:22 UTC | GET | /item/electronics/4601 | HTTP/1.1 | /item/software/263 | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 76.165.45.186 | - | - | 200 | 81 | 2015-02-18 15:39:22 UTC | GET | /category/books | HTTP/1.1 | - | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 92.75.161.183 | - | - | 200 | 134 | 2015-02-18 15:39:22 UTC | GET | /category/software | HTTP/1.1 | - | Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; BTRS122159; GTB7.2; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; BRI/2) |
| 212.48.194.74 | - | - | 200 | 43 | 2015-02-18 15:39:22 UTC | GET | /item/electronics/2081 | HTTP/1.1 | /category/books | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 188.111.193.191 | - | - | 200 | 124 | 2015-02-18 15:39:22 UTC | GET | /category/office | HTTP/1.1 | - | Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; YTB720; GTB7.2; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729) |
| 20.78.167.195 | - | - | 200 | 50 | 2015-02-18 15:39:22 UTC | GET | /item/electronics/2404 | HTTP/1.1 | - | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0.1) Gecko/20100101 Firefox/9.0.1 |
| 124.102.39.97 | - | - | 200 | 91 | 2015-02-18 15:39:22 UTC | POST | /seembulk/?c=Computers | HTTP/1.1 | - | Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
| 44.120.101.83 | - | - | 200 | 76 | 2015-02-18 15:39:22 UTC | GET | /category/garden | HTTP/1.1 | /category/software | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0.1) Gecko/20100101 Firefox/9.0.1 |
| 188.171.99.76 | - | - | 200 | 73 | 2015-02-18 15:39:22 UTC | GET | /item/books/36 | HTTP/1.1 | /seembulk/?c=Books+Garden | Mozilla/5.0 (Windows NT 6.0; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 112.99.122.107 | - | - | 200 | 67 | 2015-02-18 15:39:22 UTC | GET | /item/office/324 | HTTP/1.1 | /category/office | Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 224.54.126.115 | - | - | 200 | 44 | 2015-02-18 15:39:22 UTC | POST | /seembulk/?c=Office | HTTP/1.1 | /item/electronics/3031 | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 208.63.146.66 | - | - | 200 | 118 | 2015-02-18 15:39:22 UTC | GET | /item/electronics/1306 | HTTP/1.1 | http://www.google.com/seembulk?ie=UTF-8&q=google&sclient=psy-ab&q=Electronics&oq=Electronics&aq=f&aqi=g-vL1&aql=&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.,cf.osb&biw=2038&bih=395 | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 136.99.70.40 | - | - | 200 | 93 | 2015-02-18 15:39:22 UTC | GET | /category/sports | HTTP/1.1 | /category/office | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7 |
| 176.114.202.151 | - | - | 200 | 89 | 2015-02-18 15:39:22 UTC | GET | /category/books | HTTP/1.1 | /item/software/2413 | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 148.129.210.43 | - | - | 200 | 68 | 2015-02-18 15:39:22 UTC | GET | /item/electronics/3313 | HTTP/1.1 | /category/electronics | Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
| 152.39.123.172 | - | - | 200 | 44 | 2015-02-18 15:39:22 UTC | GET | /item/computers/172 | HTTP/1.1 | - | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) |
| 184.180.168.141 | - | - | 200 | 41 | 2015-02-18 15:39:22 UTC | GET | /item/office/792 | HTTP/1.1 | http://www.google.com/seembulk?ie=UTF-8&q=google&sclient=psy-ab&q=Office&oq=Office&aq=f&aqi=g-vL1&aql=&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.,cf.osb&biw=1799&bih=473 | Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0) |
| 212.219.132.226 | - | - | 200 | 62 | 2015-02-18 15:39:22 UTC | GET | /category/books | HTTP/1.1 | /category/electronics | Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 188.144.87.34 | - | - | 200 | 112 | 2015-02-18 15:39:22 UTC | GET | /item/software/333 | HTTP/1.1 | - | Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 144.150.114.175 | - | - | 200 | 116 | 2015-02-18 15:39:22 UTC | GET | /category/sports | HTTP/1.1 | /category/finance | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 |
| 120.78.30.223 | - | - | 200 | 122 | 2015-02-18 15:39:22 UTC | POST | /seembulk/?c=Software+Electronics | HTTP/1.1 | /category/office | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7 |
| 204.45.199.171 | - | - | 200 | 130 | 2015-02-18 15:39:22 UTC | GET | /item/software/1746 | HTTP/1.1 | /category/software | Mozilla/5.0 (Windows NT 6.0; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 |
+--------------------+-----------------------+-------------+---------------+-------------+-------------------------+---------------+---------------------------------+-----------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[embulk@bear02 ~]$
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment