Skip to content

Instantly share code, notes, and snippets.

@antimon2

antimon2/RequestsWithChannel.ipynb

Created December 13, 2019 15:06
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save antimon2/860ff8ba3d7175f03cde340ba7d924f7 to your computer and use it in GitHub Desktop.
Save antimon2/860ff8ba3d7175f03cde340ba7d924f7 to your computer and use it in GitHub Desktop.
Download ZIP
RequestsWithChannel.ipynb
Display the source blob
Display the rendered blob
Raw
RequestsWithChannel.ipynb
{
"cells": [
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:03:47.422Z",
"end_time": "2019-12-13T22:03:51.854000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "versioninfo()",
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"text": "Julia Version 1.3.0\nCommit 46ce4d7933 (2019-11-26 06:09 UTC)\nPlatform Info:\n OS: Linux (x86_64-pc-linux-gnu)\n CPU: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz\n WORD_SIZE: 64\n LIBM: libopenlibm\n LLVM: libLLVM-6.0.1 (ORCJIT, skylake)\n",
"name": "stdout"
}
]
},
{
"metadata": {},
"cell_type": "markdown",
"source": "## HTTP.jl"
},
{
"metadata": {},
"cell_type": "markdown",
"source": "↓インストール（最初に1回だけ実施）"
},
{
"metadata": {
"trusted": true
},
"cell_type": "code",
"source": "]add HTTP",
"execution_count": null,
"outputs": []
},
{
"metadata": {},
"cell_type": "markdown",
"source": "### GET Request Sample"
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:03:52.899Z",
"end_time": "2019-12-13T22:03:53.047000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "using HTTP",
"execution_count": 2,
"outputs": []
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:03:54.892Z",
"end_time": "2019-12-13T22:04:00.418000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "HTTP.request(\"GET\", \"https://www.google.com/search?q=julialang&oe=UTF-8\")",
"execution_count": 3,
"outputs": [
{
"output_type": "execute_result",
"execution_count": 3,
"data": {
"text/plain": "HTTP.Messages.Response:\n\"\"\"\nHTTP/1.1 200 OK\r\nContent-Type: text/html; charset=UTF-8\r\nDate: Fri, 13 Dec 2019 13:03:58 GMT\r\nExpires: -1\r\nCache-Control: private, max-age=0\r\nP3P: CP=\"This is not a P3P policy! See g.co/p3phelp for more info.\"\r\nServer: gws\r\nX-XSS-Protection: 0\r\nX-Frame-Options: SAMEORIGIN\r\nSet-Cookie: 1P_JAR=2019-12-13-13; expires=Sun, 12-Jan-2020 13:03:58 GMT; path=/; domain=.google.com\r\nSet-Cookie: CGIC=; expires=Wed, 10-Jun-2020 13:03:58 GMT; path=/complete/search; domain=.google.com; HttpOnly\r\nSet-Cookie: CGIC=; expires=Wed, 10-Jun-2020 13:03:58 GMT; path=/search; domain=.google.com; HttpOnly\r\nSet-Cookie: NID=193=hJBgnbkcfmJ1cGOh6zSa2MsVHJJv6OPklqpVXTuEdpGD1OTkAOC3hf8sRsMMYNYD8W9WMMKMcCcvCkeFMFzaN6_v4qlQKNBnW8x870D-0I-U63hPy227Fc5Pta2-pKCnXAOTzInnioznCmt5HPmukXA__RFMPyzoAP53u09fS20; expires=Sat, 13-Jun-2020 13:03:58 GMT; path=/; domain=.google.com; HttpOnly\r\nAlt-Svc: quic=\":443\"; ma=2592000; v=\"46,43\",h3-Q050=\":443\"; ma=2592000,h3-Q049=\":443\"; ma=2592000,h3-Q048=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000\r\nAccept-Ranges: none\r\nVary: Accept-Encoding\r\nTransfer-Encoding: chunked\r\n\r\n<!doctype html><html lang=\"ja\"><head><meta charset=\"UTF-8\"><meta content=\"/images/branding/googleg/1x/googleg_standard_color_128dp.png\" itemprop=\"image\"><title>julialang - Google 検索</title><script nonce=\"g4BIzr9klAm/00g+G3gmAg==\">(function(){document.documentElement.addEventListener(\"submit\",function(b){var a;if(a=b.target){var c=a.getAttribute(\"data-submitfalse\");a=\"1\"==c||\"q\"==c&&!a.elements.q.value?!0:!1}else a=!1;a&&(b.preventDefault(),b.stopPropagation())},!0);}).call(this);(function(){var a=window.performance;window.start=(new Date).getTime();a:{var b=window;if(a){var c=a.timing;if(c){var d=c.navigationStart,e=c.responseStart;if(e>d&&e<=window.start){window.start=e;b.wsrt=e-d;break a}}a.now&&(b.wsrt=Math.floor(a.now()))}}window.google=window.google||{};google.aft=function(f){f.setAttribute(\"data-iml\",+new Date)};}).call(this);(function(){window.jsarwt=function(){return!1};}).call(this);(function(){var c=[],e=0;window.ping=function(b){-1==b.indexOf(\"&zx\")&&(b+=\"&zx=\"+(new Date\n⋮\n73480-byte body\n\"\"\""
},
"metadata": {}
}
]
},
{
"metadata": {},
"cell_type": "markdown",
"source": "## Asynchronus Request Sample"
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:06.930Z",
"end_time": "2019-12-13T22:04:07.406000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "function do_search(results::Channel{Tuple{String, Float64, HTTP.Messages.Response}}, engine::String, url::String)\n res, exec_time = @timed HTTP.request(\"GET\", url)\n if isopen(results)\n put!(results, (engine, exec_time, res))\n end\nend",
"execution_count": 4,
"outputs": [
{
"output_type": "execute_result",
"execution_count": 4,
"data": {
"text/plain": "do_search (generic function with 1 method)"
},
"metadata": {}
}
]
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:08.254Z",
"end_time": "2019-12-13T22:04:08.416000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "function print_response(engine, exec_time, response)\n println(\"engine: \", engine)\n println(\"exec_time: \", exec_time)\n print(\"response: (\")\n print(\"$(response.status) $(HTTP.Messages.STATUS_MESSAGES[response.status]), \")\n print(\"$(length(response.headers)) headers, \")\n println(\"$(length(response.body)) bytes in body)\")\nend",
"execution_count": 5,
"outputs": [
{
"output_type": "execute_result",
"execution_count": 5,
"data": {
"text/plain": "print_response (generic function with 1 method)"
},
"metadata": {}
}
]
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:14.156Z",
"end_time": "2019-12-13T22:04:14.653000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "precompile(do_search, (Channel{Tuple{String, Float64, HTTP.Messages.Response}}, String, String))\nprecompile(print_response, (String, Float64, HTTP.Messages.Response))",
"execution_count": 6,
"outputs": [
{
"output_type": "execute_result",
"execution_count": 6,
"data": {
"text/plain": "true"
},
"metadata": {}
}
]
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:25.494Z",
"end_time": "2019-12-13T22:04:26.346000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "channel = Channel{Tuple{String, Float64, HTTP.Messages.Response}}(1)\n@async do_search(channel, \"Google\", \"https://www.google.com/search?q=julialang&oe=UTF-8\")\nres0 = take!(channel)\nprint_response(res0...)\nclose(channel)",
"execution_count": 7,
"outputs": [
{
"output_type": "stream",
"text": "engine: Google\nexec_time: 0.487161003\nresponse: (200 OK, 16 headers, 73480 bytes in body)\n",
"name": "stdout"
}
]
},
{
"metadata": {},
"cell_type": "markdown",
"source": "## 早いもの勝ち"
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:33.654Z",
"end_time": "2019-12-13T22:04:33.785000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "function main()\n results = Channel{Tuple{String, Float64, HTTP.Messages.Response}}(3)\n \n searches = [\n \"Google\"=>\"https://www.google.com/search?q=julialang&oe=UTF-8\", \n \"Bing\"=>\"https://www.bing.com/search?q=julialang\",\n \"DuckDuckGo\"=>\"https://duckduckgo.com/?q=julialang\"]\n for (engine, url) = searches\n @async do_search(results, engine, url)\n end\n\n res0 = take!(results)\n close(results)\n \n print_response(res0...)\nend",
"execution_count": 8,
"outputs": [
{
"output_type": "execute_result",
"execution_count": 8,
"data": {
"text/plain": "main (generic function with 1 method)"
},
"metadata": {}
}
]
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:35.163Z",
"end_time": "2019-12-13T22:04:35.236000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "precompile(main, ())",
"execution_count": 9,
"outputs": [
{
"output_type": "execute_result",
"execution_count": 9,
"data": {
"text/plain": "true"
},
"metadata": {}
}
]
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:37.199Z",
"end_time": "2019-12-13T22:04:37.644000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "main()",
"execution_count": 10,
"outputs": [
{
"output_type": "stream",
"text": "engine: Google\nexec_time: 0.435377601\nresponse: (200 OK, 16 headers, 73474 bytes in body)\n",
"name": "stdout"
}
]
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:38.682Z",
"end_time": "2019-12-13T22:04:38.939000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "main()",
"execution_count": 11,
"outputs": [
{
"output_type": "stream",
"text": "engine: DuckDuckGo\nexec_time: 0.25063164\nresponse: (200 OK, 18 headers, 19049 bytes in body)\n",
"name": "stdout"
}
]
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:39.835Z",
"end_time": "2019-12-13T22:04:40.125000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "main()",
"execution_count": 12,
"outputs": [
{
"output_type": "stream",
"text": "engine: DuckDuckGo\nexec_time: 0.287571225\nresponse: (200 OK, 18 headers, 19049 bytes in body)\n",
"name": "stdout"
}
]
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:40.790Z",
"end_time": "2019-12-13T22:04:41.079000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "main()",
"execution_count": 13,
"outputs": [
{
"output_type": "stream",
"text": "engine: DuckDuckGo\nexec_time: 0.287246897\nresponse: (200 OK, 18 headers, 19049 bytes in body)\n",
"name": "stdout"
}
]
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:41.758Z",
"end_time": "2019-12-13T22:04:42.054000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "main()",
"execution_count": 14,
"outputs": [
{
"output_type": "stream",
"text": "engine: DuckDuckGo\nexec_time: 0.289991983\nresponse: (200 OK, 18 headers, 19049 bytes in body)\n",
"name": "stdout"
}
]
},
{
"metadata": {},
"cell_type": "markdown",
"source": "## Search Race"
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:45.450Z",
"end_time": "2019-12-13T22:04:45.576000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "function main2()\n results = Channel{Tuple{String, Float64, HTTP.Messages.Response}}(3)\n \n searches = [\n \"Google\"=>\"https://www.google.com/search?q=julialang&oe=UTF-8\", \n \"Bing\"=>\"https://www.bing.com/search?q=julialang\",\n \"DuckDuckGo\"=>\"https://duckduckgo.com/?q=julialang\"]\n for (engine, url) = searches\n @async do_search(results, engine, url)\n end\n\n res1 = take!(results)\n print_response(res1...)\n println()\n \n res2 = take!(results)\n print_response(res2...)\n println()\n \n res3 = take!(results)\n print_response(res3...)\n \n close(results)\nend",
"execution_count": 15,
"outputs": [
{
"output_type": "execute_result",
"execution_count": 15,
"data": {
"text/plain": "main2 (generic function with 1 method)"
},
"metadata": {}
}
]
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:46.746Z",
"end_time": "2019-12-13T22:04:46.799000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "precompile(main2, ())",
"execution_count": 16,
"outputs": [
{
"output_type": "execute_result",
"execution_count": 16,
"data": {
"text/plain": "true"
},
"metadata": {}
}
]
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:47.835Z",
"end_time": "2019-12-13T22:04:48.349000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "main2()",
"execution_count": 17,
"outputs": [
{
"output_type": "stream",
"text": "engine: DuckDuckGo\nexec_time: 0.360540393\nresponse: (200 OK, 18 headers, 19049 bytes in body)\n\nengine: Bing\nexec_time: 0.429006315\nresponse: (200 OK, 16 headers, 158713 bytes in body)\n\nengine: Google\nexec_time: 0.508506785\nresponse: (200 OK, 16 headers, 73474 bytes in body)\n",
"name": "stdout"
}
]
},
{
"metadata": {
"ExecuteTime": {
"start_time": "2019-12-13T13:04:50.295Z",
"end_time": "2019-12-13T22:04:50.793000+09:00"
},
"trusted": true
},
"cell_type": "code",
"source": "main2()",
"execution_count": 18,
"outputs": [
{
"output_type": "stream",
"text": "engine: DuckDuckGo\nexec_time: 0.290471476\nresponse: (200 OK, 18 headers, 19049 bytes in body)\n\nengine: Bing\nexec_time: 0.374741239\nresponse: (200 OK, 16 headers, 157068 bytes in body)\n\nengine: Google\nexec_time: 0.493652595\nresponse: (200 OK, 16 headers, 73480 bytes in body)\n",
"name": "stdout"
}
]
},
{
"metadata": {
"trusted": true
},
"cell_type": "code",
"source": "",
"execution_count": null,
"outputs": []
}
],
"metadata": {
"_draft": {
"nbviewer_url": "https://gist.github.com/6944d60c126939b6704aa594466770be"
},
"gist": {
"id": "6944d60c126939b6704aa594466770be",
"data": {
"description": "RequestsWithChannel.ipynb",
"public": true
}
},
"kernelspec": {
"name": "julia-1.3",
"display_name": "Julia 1.3.0",
"language": "julia"
},
"language_info": {
"file_extension": ".jl",
"name": "julia",
"mimetype": "application/julia",
"version": "1.3.0"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment