Skip to content

Instantly share code, notes, and snippets.

@derek-adair
Last active May 4, 2016 14:31
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save derek-adair/bbb3d3087aba06d61439b0f79af49e13 to your computer and use it in GitHub Desktop.
Save derek-adair/bbb3d3087aba06d61439b0f79af49e13 to your computer and use it in GitHub Desktop.
#requires aiohttp - navigate to localhost:8080/?domain={domain_to_scrape}
from aiohttp import web
import aiohttp
import asyncio
async def handle(request):
domain = '//{}'.format(request.GET['domain'])
loop = asyncio.get_event_loop()
with aiohttp.ClientSession(loop=loop) as session:
print("fetching from {}".format(domain))
async with session.get(domain) as resp:
content = await resp.text()
return web.Response(body=content.encode('utf-8'))
app = web.Application()
app.router.add_route('GET', '/', handle)
web.run_app(app)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment