This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
purchase table example | |
transaction_date | customer_id | purchase_price | category | |
2022-01-01 customer_A $10 jewelry | |
2022-01-02 customer_A $1 t-shirts | |
... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import boto3 | |
dynamodb = boto3.resource("dynamodb") | |
params = { | |
"TableName": "customer_purchase_ts", | |
"KeySchema": [ | |
{"AttributeName": "customer_id", "KeyType": "HASH"} | |
], | |
"LocalSecondaryIndexes": [ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import boto3 | |
dynamodb = boto3.resource("dynamodb") | |
params = { | |
'TableName': "customer_purchase_ts", | |
"KeySchema": [ | |
{"AttributeName": "customer_id", "KeyType": "HASH"}, | |
{"AttributeName": "transaction_date_epoch", "KeyType": "RANGE"} | |
], |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import boto3 | |
from datetime import datetime | |
import numpy as np | |
from fastapi import FastAPI | |
app = FastAPI() | |
dynamodb = boto3.resource("dynamodb") | |
s3 = boto3.client('s3') | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import ... | |
import fire | |
class CustomerModel(BaseModel): | |
def __init__(self): | |
super().__init__() | |
def train_model(lookback_days=30: int): |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from airflow.models import Variable | |
from airflow import DAG | |
from airflow.operators.python_operator import PythonOperator | |
MODEL_ENTRY_POINT = "customer_model.py" | |
# This dynamic configuration mechanism is not ideal but makes for an easy demo | |
model_version = Variable.get("model_version") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from airflow.models import Variable | |
from airflow import DAG | |
from airflow.operators.python_operator import PythonOperator | |
def train_model(lookback_days=30: int): | |
""" | |
Trains a model using data from the past <lookback_days> and persists to a model store | |
""" | |
... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WITH transaction AS ( | |
SELECT transaction_id, | |
customer_id, | |
state, | |
amount_spent_usd | |
FROM < FACT_TABLE_TRANSACTION > | |
), | |
customer_spend AS ( | |
SELECT customer_id, | |
state, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WITH transaction AS ( | |
SELECT transaction_id, | |
customer_id, | |
state, | |
amount_spent_usd | |
FROM FACT_TABLE_TRANSACTION | |
), | |
customer_spend AS ( | |
SELECT customer_id, | |
state, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WITH month_0 AS | |
( | |
SELECT user_id, | |
metric_a | |
FROM analytics_table | |
WHERE execution_date = < month_0 > | |
), | |
month_1 AS | |
( | |
SELECT user_id, |
NewerOlder