Skip to content

Instantly share code, notes, and snippets.

@khuyentran1401

khuyentran1401/data_engineering.py

Last active November 6, 2021 15:43
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save khuyentran1401/ca16a8275b6c3874501b102a1d6ef3f2 to your computer and use it in GitHub Desktop.
Save khuyentran1401/ca16a8275b6c3874501b102a1d6ef3f2 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
data_engineering.py
from prefect.engine.results import LocalResult
@task(result = LocalResult(dir='data/processed'))
def split_data(data: pd.DataFrame, test_data_ratio: float, classes: list) -> Dict[str, Any]:
"""Task for splitting the classical Iris data set into training and test
sets, each split into features and labels.
"""
X_train, X_test, y_train, y_test = ...
return dict(
train_x=X_train,
train_y=y_train,
test_x=X_test,
test_y=y_test,
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment