requirements.md

Requirements

---

• pandas
• scikit-learn
• sklearn-pandas
• Dive
• Google hosted jupyter notebook: Colab. Introductory tutorial on how to use colab

Topics Covered

---

• Data Exploration & Validation
  • Statistical Properties
    • Mean, variance etc.
    • Correlation
    • Entropy
    • Mutual information, Point-wise mutual information
  • Missing values
  • Outlier Detection
  • Feature imputation
  • Fairness in Machine Learning
• Feature Engineering
  • Feature scaling
  • Feature normalization
  • Feature standardization
  • Feature encoding
    • One hot encoding (why do we need one hot encoding)
  • Feature selection
    • Feature Correlation: Whether two features are correlated
    • Mutual information
    • chi2 test
    • Dimensionality reduction - Principal Component Analysis (PCA)
  • Feature Importance
• Algorithms
  • Linear Classification
  • Linear Regression
• Validation
  • k-fold cross validation

Dataset

• Titanic Dataset for classification
  • Train
  • Test
• Housing Price Dataset for regression
  • Train
  • Test

Other Useful Resources

Data Exploration & Visualization Tools

• https://pair-code.github.io/facets/
• http://vega.github.io/voyager/
• https://github.com/altair-viz/jupyterlab_voyager
• https://pypi.org/project/ipython-sql/
• https://datastudio.google.com/

Interpretable Machine Learning Tools

• https://pair-code.github.io/what-if-tool/
• https://github.com/marcotcr/lime/

AutoML

• https://www.datarobot.com/ (Commercial)

Datasets

• https://archive.ics.uci.edu/ml/datasets.html
• https://www.kaggle.com/
• https://opendata.stackexchange.com/

Few off-the-shelf good enough tools

• https://github.com/Microsoft/LightGBM
• https://github.com/dmlc/xgboost