Skip to content

Instantly share code, notes, and snippets.

@araraloren
Last active November 25, 2017 13:18
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save araraloren/b71482265faa2c103340d99ff5cd9b66 to your computer and use it in GitHub Desktop.
Save araraloren/b71482265faa2c103340d99ff5cd9b66 to your computer and use it in GitHub Desktop.
CRO POST
has Cro::HTTP::Client $!c;
has Str $.root is required;
has Supplier $!supplier;
has $!xml-cache;
has $!result-count;
has $!index;
has $.debug;
has $!taped;
submethod TWEAK() {
$!c .= new(
:follow,
:cookie-jar,
# 这是默认的请求头
# 每次请求的时候都会带上
headers => [
'User-Agent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0',
'Accept' => 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language' => 'zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2',
'Accept-Encoding' => 'gzip, deflate',
'Content-Type' => 'application/x-www-form-urlencoded',
'DNT' => 1,
'Connection' => 'keep-alive',
'Cache-Control' => 'max-age=0',
'Upgrade-Insecure-Requests' => 1
]
);
$!supplier .= new;
}
method reset() {
$!index = 0;
$!result-count = Int;
unless $!taped {
$!supplier.Supply.tap;
$!taped = True;
}
}
# 第一次搜索,我们需要自己构造 form
# 默认提供一个关键字即可
method construct-first-body(*%body) {
state %default-body = %{
tag => 'search',
subtag => 'simsearch',
gcbook => 'yes',
viewtype => '',
flword => '',
corename => '',
searchType => 'oneSearch',
title => '',
searchFieldType => 'text',
aliasname => '全部馆藏',
# q => '中国农民战争史'
searchModel => 'front',
su => '',
field => 'title'
};
my %ret = %default-body;
for %body.kv -> $key, $value {
%ret{$key} = $value;
}
return %ret;
}
method check-status($status) {
fail "Get page failed!" if $status != 200;
}
# 获取页面
method GET(Str $path) {
note "GET PAGE => ", $path if $!debug;
my $resp = await $!c.get("{$!root}{$path}");
given $resp {
self.check-status(.status);
return $resp.body-text().result;
}
}
# 请求页面
method POST(Str $path, *%args) {
note "POST PAGE => {$path} " if $!debug;
note "--------------" if $!debug;
note %args if $!debug;
note "--------------" if $!debug;
my $resp = await $!c.post("{$!root}{$path}", |%args);
"POST OK".note if $!debug;
note $resp if $!debug;
given $resp {
self.check-status(.status);
my $bt = $resp.body-text;
while $bt.status !~~ Kept {
note "Planned ..." if $!debug;
sleep 0.2;
}
return $bt.result;
}
}
@araraloren
Copy link
Author

[TRACE(anon 1)] RequestSerializerExtension EMIT HTTP Request
POST /opac/search HTTP/1.1
Host: 202.115.193.41
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded
DNT: 1
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
[TRACE(anon 1)] Cro::HTTP::RequestSerializer QUIT Cannot flush handle: Failed to flush filehandle: Input/output error
in block at /home/sakuya/.perl6/sources/2031428AEA6969E6B02B1CCDBB08747A9FC09D56 (Cro) line 378
in block at /home/sakuya/.perl6/sources/022CB3E48DE31921D5811E074413C91E34F262EC (Cro::HTTP::Internal) line 45

[TRACE(anon 1)] Cro::TCP::Connector QUIT Cannot flush handle: Failed to flush filehandle: Input/output error
in block at /home/sakuya/.perl6/sources/2031428AEA6969E6B02B1CCDBB08747A9FC09D56 (Cro) line 386
in block at /home/sakuya/.perl6/sources/022CB3E48DE31921D5811E074413C91E34F262EC (Cro::HTTP::Internal) line 45

[TRACE(anon 1)] Cro::HTTP::ResponseParser QUIT Cannot flush handle: Failed to flush filehandle: Input/output error
in block at /home/sakuya/.perl6/sources/2031428AEA6969E6B02B1CCDBB08747A9FC09D56 (Cro) line 386
in block at /home/sakuya/.perl6/sources/022CB3E48DE31921D5811E074413C91E34F262EC (Cro::HTTP::Internal) line 45

[TRACE(anon 1)] ResponseParserExtension QUIT Cannot flush handle: Failed to flush filehandle: Input/output error
in block at /home/sakuya/.perl6/sources/2031428AEA6969E6B02B1CCDBB08747A9FC09D56 (Cro) line 386
in block at /home/sakuya/.perl6/sources/022CB3E48DE31921D5811E074413C91E34F262EC (Cro::HTTP::Internal) line 45

Usage:
./get-book-number.p6 [-h|--help] [-host=] [--port=] [-d|--debug] [-i|--interval=] [-o|--output=] [-q|--quite] *@Args

-h|--help 打印帮助信息.

-host= 设置服务器IP.[202.115.193.41]

--port= 设置服务器端口.[8080]

-d|--debug 打开调试接口.

-i|--interval= 访问间隔.[5]

-o|--output= 输出到指定文件.[result.txt]

-q|--quite 安静模式.

Cannot flush handle: Failed to flush filehandle: Input/output error
An operation first awaited:
in method POST at ./get-book-number.p6 line 203
in method search at ./get-book-number.p6 line 263
in block at ./get-book-number.p6 line 90
in sub main at ./get-book-number.p6 line 82
in method CALL-ME at /home/sakuya/.perl6/sources/1CDE47E6E1939600AFDA04BFA1900F87FE50D0D4 (Getopt::Advance::NonOption) line 22
in sub process-main at /home/sakuya/.perl6/sources/9DB55C88BEF1A62488C6EAFA3F9D1FA0773B73E4 (Getopt::Advance::Parser) line 399
in sub ga-parser at /home/sakuya/.perl6/sources/9DB55C88BEF1A62488C6EAFA3F9D1FA0773B73E4 (Getopt::Advance::Parser) line 282
in sub getopt at /home/sakuya/.perl6/sources/D5AA93820CB489850BF207F3A2293771D4C64C38 (Getopt::Advance) line 73
in sub getopt at /home/sakuya/.perl6/sources/D5AA93820CB489850BF207F3A2293771D4C64C38 (Getopt::Advance) line 18
in block at ./get-book-number.p6 line 105

Died with the exception:
Cannot flush handle: Failed to flush filehandle: Input/output error
in block at /home/sakuya/.perl6/sources/D5AA93820CB489850BF207F3A2293771D4C64C38 (Getopt::Advance) line 109
in any at /home/sakuya/.perl6/precomp/272B80FC0A239CB25AC22453FE7B3B3434A80F34.1509139376.33975/D5/D5AA93820CB489850BF207F3A2293771D4C64C38 line 1
in sub main at ./get-book-number.p6 line 82
in method CALL-ME at /home/sakuya/.perl6/sources/1CDE47E6E1939600AFDA04BFA1900F87FE50D0D4 (Getopt::Advance::NonOption) line 22
in sub process-main at /home/sakuya/.perl6/sources/9DB55C88BEF1A62488C6EAFA3F9D1FA0773B73E4 (Getopt::Advance::Parser) line 399
in sub ga-parser at /home/sakuya/.perl6/sources/9DB55C88BEF1A62488C6EAFA3F9D1FA0773B73E4 (Getopt::Advance::Parser) line 282
in sub getopt at /home/sakuya/.perl6/sources/D5AA93820CB489850BF207F3A2293771D4C64C38 (Getopt::Advance) line 73
in sub getopt at /home/sakuya/.perl6/sources/D5AA93820CB489850BF207F3A2293771D4C64C38 (Getopt::Advance) line 18
in block at ./get-book-number.p6 line 105

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment