Skip to content

Instantly share code, notes, and snippets.

@LordBastor
Created February 5, 2021 16:22
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save LordBastor/097f39a500be9346413d064f12fa675d to your computer and use it in GitHub Desktop.
Save LordBastor/097f39a500be9346413d064f12fa675d to your computer and use it in GitHub Desktop.
from datetime import timedelta
from prefect import Flow
from prefect import task
from tasks.pokemontasks import ExtractPokemon, TransformPokemon, LoadPokemon
from utils.config import kubernetes_run, azure_store
extract_pokemon = ExtractPokemon()
transform_pokemon = TransformPokemon()
load_pokemon = LoadPokemon()
with Flow("PokeFlow", run_config=kubernetes_run ,storage=azure_store) as flow:
extracted_pokemon = extract_pokemon()
transformed_pokemon = transform_pokemon.map(extracted_pokemon['results'])
loaded_pokemon = load_pokemon(transformed_pokemon)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment