Skip to content

Instantly share code, notes, and snippets.

@pradeepbn
Created July 25, 2016 09:25
Show Gist options
  • Save pradeepbn/2e87c99b63c1c2aee96ee35b2dd80b8d to your computer and use it in GitHub Desktop.
Save pradeepbn/2e87c99b63c1c2aee96ee35b2dd80b8d to your computer and use it in GitHub Desktop.
Lua script for splash
lua_source = """
function main(splash)
local host = "proxy.crawlera.com"
local port = 8010
local user = "<API_KEY>"
local password = ''
local session_header = "X-Crawlera-Session"
local session_id = "create"
splash:on_request(function (request)
request:set_header("X-Crawlera-UA", "desktop")
request:set_header(session_header, session_id)
request:set_header("X-Crawlera-Max-Retries" : 1)
request:set_proxy{host, port, username=user, password=password}
end)
splash:on_response_headers(function (response)
if type(response.headers[session_header]) ~= nil then
session_id = response.headers[session_header]
end
end)
splash:set_result_content_type("text/html; charset=utf-8")
print(splash.args.url)
splash:go{splash.args.url,
headers=splash.args.headers,
http_method=splash.args.http_method,
body=splash.args.body,}
assert(splash:wait(splash.args.wait))
return splash:html()
end
"""
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment