Created
April 4, 2019 12:25
-
-
Save somiandras/40ccefc39b68ee896687babd4d0258a6 to your computer and use it in GitHub Desktop.
Logging function to be used in pd.DataFrame.pipe().
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def pipe_log(data, logger, message=None, level='DEBUG'): | |
''' | |
Logging function to be used in pd.DataFrame.pipe(). Insert anywhere | |
in a method chain to log head and shape of dataframe/series along | |
with a custom message with the set logging level, then returns the | |
dataframe unchanged. | |
Params: | |
- data: pandas.DataFrame or pandas.Series | |
- logger: logging.Logger | |
- message: str (optional, default=None) | |
- level: str (optional, default='DEBUG') | |
Example: | |
# Logs message, head and shape before calling sum() on dataframe | |
df.pipe(pipe_log, logger, message='Test', level='INFO').sum() | |
''' | |
lvl = getattr(logging, level, 10) | |
if message is not None: | |
logger.log(lvl, message) | |
logger.log(lvl, data.head()) | |
logger.log(lvl, data.shape) | |
return data |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Could be further abstracted by accepting list of DataFrame properties and methods that should be logged, instead of hard coded
.head()
and.shape
.