Skip to content

Instantly share code, notes, and snippets.

@AntonFriberg

AntonFriberg/test_partition_extract.py

Created December 4, 2019 12:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save AntonFriberg/ed4b904d81dade6e4a4c908a288ff279 to your computer and use it in GitHub Desktop.
Save AntonFriberg/ed4b904d81dade6e4a4c908a288ff279 to your computer and use it in GitHub Desktop.
Download ZIP
S3 partition extraction in Python
Raw
test_partition_extract.py
import re
regex = r"(year|month|day|hour)=(\d+)"
test_str = "s3://bucket/datalake/year=2019/month=12/day=02/hour=06"
matches = re.finditer(regex, test_str)
partition = {match.group(1): int(match.group(2)) for match in matches}
print(partition)
# {'year': 2019, 'month': 12, 'day': 2, 'hour': 6}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment