Skip to content

Instantly share code, notes, and snippets.

@gabrielhpugliese
Last active December 13, 2018 16:50
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save gabrielhpugliese/036d06026123144427a6 to your computer and use it in GitHub Desktop.
Save gabrielhpugliese/036d06026123144427a6 to your computer and use it in GitHub Desktop.
Python 3.4 asyncio + aiohttp parallel vs sequential requests
$python parallel.py
[13:48:59:130701] Doing GET request to http://google.com
[13:48:59:134958] Doing GET request to http://facebook.com
[13:48:59:135498] Doing GET request to http://twitter.com
[13:48:59:136174] Doing GET request to http://slack.com
****** STATUSES: [{'status': 200, 'url': 'http://google.com'}, {'status': 200, 'url': 'http://twitter.com'}, {'status': 200, 'url': 'http://slack.com'}, {'status': 200, 'url': 'http://facebook.com'}]
$python sequential.py
[13:49:04:078752] Doing GET request to http://google.com
[13:49:04:232725] Doing GET request to http://facebook.com
[13:49:06:723977] Doing GET request to http://twitter.com
[13:49:07:389673] Doing GET request to http://slack.com
****** STATUSES: [{'status': 200, 'url': 'http://google.com'}, {'status': 200, 'url': 'http://facebook.com'}, {'status': 200, 'url': 'http://twitter.com'}, {'status': 200, 'url': 'http://slack.com'}]
import asyncio
import aiohttp
statuses = []
@asyncio.coroutine
def get(url):
print('[{}] Doing GET request to {}'.format(datetime.now().strftime('%H:%M:%S:%f'), url))
response = yield from aiohttp.request('GET', url)
statuses.append({'url': url, 'status': response.status})
response.close()
urls = ['http://google.com', 'http://facebook.com', 'http://twitter.com', 'http://slack.com']
tasks = [asyncio.Task(get(url)) for url in urls]
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(*tasks))
loop.close()
print('****** STATUSES:', statuses)
import asyncio
import aiohttp
statuses = []
@asyncio.coroutine
def get(url):
print('[{}] Doing GET request to {}'.format(datetime.now().strftime('%H:%M:%S:%f'), url))
response = yield from aiohttp.request('GET', url)
statuses.append({'url': url, 'status': response.status})
response.close()
@asyncio.coroutine
def main():
urls = ['http://google.com', 'http://facebook.com', 'http://twitter.com', 'http://slack.com']
for url in urls:
yield from get(url)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
print('****** STATUSES:', statuses)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment