Skip to content

Instantly share code, notes, and snippets.

@arihantsurana
arihantsurana / AthenaToAthenaExample.py
Created July 22, 2020 07:30
And example use of writing SQL to generate performant Athena tables
from hip_data_tools.etl.athena_to_athena import AthenaToAthena, AthenaToAthenaSettings
from hip_data_tools.aws.common import AwsConnectionSettings
aws_conn = AwsConnectionSettings(
region="ap-southeast-2",
secrets_manager=None,
profile="default") # to access the default aws profile on your machine
etl = AthenaToAthena(
settings=AthenaToAthenaSettings(
@arihantsurana
arihantsurana / AthenaToDataFrameExample.py
Last active July 23, 2020 09:57
An example for using hip_data_tools for Athena to Dataframe utility
from hip_data_tools.aws.common import AwsConnectionSettings
from hip_data_tools.etl.athena_to_dataframe import AthenaToDataFrameSettings, AthenaToDataFrame
aws_conn = AwsConnectionSettings(
region="ap-southeast-2",
secrets_manager=None,
profile="default") # to access the default aws profile on your machine
etl = AthenaToDataFrame(
settings=AthenaToDataFrameSettings(

Keybase proof

I hereby claim:

  • I am arihantsurana on github.
  • I am arisurana (https://keybase.io/arisurana) on keybase.
  • I have a public key whose fingerprint is 5400 D2A8 5A6F 1BF8 C5ED 8161 320A 4DCB 2F6C 3D6A

To claim this, I am signing this object:

Keybase proof

I hereby claim:

  • I am arisurana on github.
  • I am arisurana (https://keybase.io/arisurana) on keybase.
  • I have a public key whose fingerprint is 5400 D2A8 5A6F 1BF8 C5ED 8161 320A 4DCB 2F6C 3D6A

To claim this, I am signing this object: