Skip to content

Instantly share code, notes, and snippets.

@xiangzhuyuan
Created February 11, 2019 13:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save xiangzhuyuan/32e146fbb667429c037774904bbff035 to your computer and use it in GitHub Desktop.
Save xiangzhuyuan/32e146fbb667429c037774904bbff035 to your computer and use it in GitHub Desktop.
Get latest campaign without afli
require 'net/http'
require 'uri'
require 'cgi'
require 'byebug'
require 'nokogiri'
require 'yaml'
uri = URI.parse("https://blog.cbnanashi.net/2018/10/5575/comment-page-5")
request = Net::HTTP::Get.new(uri)
request["Authority"] = "blog.cbnanashi.net"
request["Pragma"] = "akamai-x-get-client-ip, akamai-x-cache-on, akamai-x-cache-remote-on, akamai-x-check-cacheable, akamai-x-get-cache-key, akamai-x-get-extracted-values, akamai-x-get-nonces, akamai-x-get-ssl-client-session-id, akamai-x-get-true-cache-key, akamai-x-serial-no, akamai-x-feo-trace, akamai-x-get-request-id"
request["Cache-Control"] = "no-cache"
request["Upgrade-Insecure-Requests"] = "1"
request["User-Agent"] = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36"
request["Dnt"] = "1"
request["Accept"] = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"
request["Accept-Language"] = "zh-CN,zh;q=0.9,ja;q=0.8,en-US;q=0.7,en;q=0.6"
request["Cookie"] = "__cfduid=d88a255810960aee56494788f6e0a8a9c1547037779; _ga=GA1.2.744862532.1547037781; _gid=GA1.2.1157126859.1549809370; _gat=1"
req_options = {
use_ssl: uri.scheme == "https",
}
campaign = {}
response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
http.request(request)
end
begin
doc = Nokogiri::HTML(response.body)
doc.css("ol.commets-list>li>div>p:first").each do |comment|
url = CGI.parse(CGI.unescape(comment.children.first.attributes['href'].value))['p'][0]
title = comment.text
puts "#{title}--#{url}"
campaign[title]=url
end
rescue => e
puts e.message
end
File.write('latest.yml', campaign.to_yaml)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment