Skip to content

Instantly share code, notes, and snippets.

@tkanngiesser
Created August 4, 2021 08:16
Show Gist options
  • Save tkanngiesser/1dde495e09e21f4cacbb285da87ff11c to your computer and use it in GitHub Desktop.
Save tkanngiesser/1dde495e09e21f4cacbb285da87ff11c to your computer and use it in GitHub Desktop.
get files from url (txt or json)
def get_text_from_url(url, file_name, split_pattern):
filename, file_extension = os.path.splitext(file_name)
url = os.path.join(url, file_name)
if file_extension == ".txt":
text = requests.get(url).text
else:
resp = requests.get(url)
text = json.loads(resp.text, encoding="utf-8")
if split_pattern:
text = text.split(split_pattern)
return text
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment