Skip to content

Instantly share code, notes, and snippets.

@nickfun
Created May 18, 2022 03:43
Show Gist options
  • Save nickfun/3310b39ed94898eb0f80cb7e013b017f to your computer and use it in GitHub Desktop.
Save nickfun/3310b39ed94898eb0f80cb7e013b017f to your computer and use it in GitHub Desktop.
xpath dictionary
# What to do given one XPath
def processXpath(path, doc):
try:
result = do_xpath(path, doc)
return result.text
except:
return "!!! Failed to scrape"
# you have some way of getting the document
doc = ...
# init empty
results = {}
# a big mapping of "human readable name" => "xpath"
actions = {
"Primary Owner": '//*[@id="owner"]/h1',
"Secondary Owner": '//*[@id="other_owner"]/div',
# ... etc
}
for key in actions:
name = key
xpath = actions[key]
results[name] = processXpath(xpath, doc)
print(results)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment