Skip to content

Instantly share code, notes, and snippets.

@qooba

qooba/README.md

Created June 9, 2021 21:22
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save qooba/edd1753874b0397688e9798c0cb03632 to your computer and use it in GitHub Desktop.
Save qooba/edd1753874b0397688e9798c0cb03632 to your computer and use it in GitHub Desktop.
Download ZIP
Feast with s3 usage
Raw
README.md

Installation

git clone -b feature/filesource_s3 https://github.com/qooba/feast.git cd feast && make install-python-ci-dependencies

Usage

You have to set environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY

In feature store python definition you can use:

driver_hourly_stats = FileSource(
    path="s3://propensity/driver_stats.parquet",
    event_timestamp_column="datetime",
    created_timestamp_column="created",
    s3_endpoint_override="http://minio:9000"
)

the s3_endpoint_override argument is optional if you want to use internal S# storage eg. Minio

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment