fernandojunior

Quais tipos de Analytics ("data science")?

• https://docs.google.com/presentation/d/1NCWYA8vuYmw5n4rrOXJX47dJPSNawGjLsIOdbfsWNJA/edit#slide=id.gf0229e45ee_0_129
• https://www.kdnuggets.com/2017/07/4-types-data-analytics.html

Quais Profissões de Dados?

• https://sites.google.com/picpay.com/picpayspace/carreiras-do-picpay/trilhas-de-carreira
• https://www.researchgate.net/publication/350428076_O_perfil_do_cientista_de_dados_no_Brasil_competencias_e_niveis_de_senioridade (pg 250)
• https://www.kdnuggets.com/2018/08/top-10-roles-ai-data-science.html
• https://towardsdatascience.com/what-is-mlops-everything-you-must-know-to-get-started-523f2d0b8bd8

fernandojunior

from typing import Any, Callable
from dataclasses import dataclass

import numpy as np
import pandas as pd
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import MinMaxScaler
from sklearn.preprocessing import FunctionTransformer

fernandojunior

fdsf

fernandojunior

I suggest you to use some lint tools like flake8 to find some PEP8 warnings. Optionally, you can use flake8-black to automatically find and fix them. This way, you will have a more consistent and unified code style.

• Add flake8 and flake8-black (optional) as a dependency to requirements.txt and test_requirements.txt
• Create a .flake8 config file in project's root folder ROOT_FOLDER/.flake8 to setup some global flake8 configurations to customize or ignore some warnings/violations (see more here and here).

Example:

# .flake8 file
[flake8]
ignore = D203, E501, W504, W503, E731, E712

fernandojunior

https://www.python.org/dev/peps/pep-0008/#pet-peeves

Indentation

Use 4 spaces per indentation level.

Correct:

# Aligned with opening delimiter.

fernandojunior

"Help, I keep getting a 'Permission Denied (publickey)' error when I push!"

This means, on your local machine, you haven't made any SSH keys. Not to worry. Here's how to fix:

1. Open git bash (Use the Windows search. To find it, type "git bash") or the Mac Terminal. Pro Tip: You can use any *nix based command prompt (but not the default Windows Command Prompt!)
2. Type cd ~/.ssh. This will take you to the root directory for Git (Likely C:\Users\[YOUR-USER-NAME]\.ssh\ on Windows)
3. Within the .ssh folder, there should be these two files: id_rsa and id_rsa.pub. These are the files that tell your computer how to communicate with GitHub, BitBucket, or any other Git based service. Type ls to see a directory listing. If those two files don't show up, proceed to the next step. NOTE: Your SSH keys must be named id_rsa and id_rsa.pub in order for Git, GitHub, and BitBucket to recognize them by default.
4. To create the SSH keys, type ssh-keygen -t rsa -C "your_email@example.com". Th

fernandojunior

https://www.kaggle.com/dansbecker/data-leakage
https://towardsdatascience.com/data-leakage-in-machine-learning-10bdd3eec742
https://towardsdatascience.com/preventing-data-leakage-in-your-machine-learning-model-9ae54b3cd1fb
https://stats.stackexchange.com/questions/90711/can-independent-variables-with-low-correlation-with-dependent-variable-be-signif

https://medium.com/@rafaelpierre/data-leakage-part-i-think-you-have-a-great-machine-learning-model-think-again-ad44921fbf34

fernandojunior

fernandojunior

'''
This is an example of how to send data to Slack webhooks in Python with the
requests module.
Detailed documentation of Slack Incoming Webhooks:
https://api.slack.com/incoming-webhooks

'''

import json
import requests

fernandojunior

https://databricks.com/session_na20/continuous-delivery-of-ml-enabled-pipelines-on-databricks-using-mlflow
https://databricks.com/blog/2020/06/05/automate-continuous-integration-and-continuous-delivery-on-databricks-using-databricks-labs-ci-cd-templates.html
https://github.com/databrickslabs/cicd-templates
https://github.com/mshtelma/cicdtestdev
https://github.com/mshtelma/lendingclubsscoringdemo