Skip to content

Instantly share code, notes, and snippets.

@sadams
Created August 16, 2021 22:36
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save sadams/43c7bc0cc983f81712e1e0303c294752 to your computer and use it in GitHub Desktop.
Save sadams/43c7bc0cc983f81712e1e0303c294752 to your computer and use it in GitHub Desktop.
function use_crawlera(splash)
local password = ''
local host = 'proxy.zyte.com'
local port = 8011
local session_header = 'X-Crawlera-Session'
local session_id = 'create'
splash:on_request(function (request)
request:set_proxy(host, port, 'PUT_CRAWLERA_AUTH_HERE', password)
request:set_header('X-Crawlera-Profile', 'desktop')
request:set_header('X-Crawlera-Cookies', 'disable')
request:set_header(session_header, session_id)
end)
splash:on_response_headers(function (response)
if type(response.headers[session_header]) ~= nil then
session_id = response.headers[session_header]
end
end)
end
function main(splash)
use_crawlera(splash)
splash:go(splash.args.url)
splash:wait(1)
return splash:html()
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment