ryanbehdad

def compare_df_columns(df1, df2):
    """
    Compare the columns of two dataframes (including their types)
    """
    matched = True
    
    # Compare number of rows
    if df1.shape[0] != df2.shape[0]:
        print(f'Row numbers do not match {df1.shape[0]:,} vs {df2.shape[0]:,}')
        matched=False   

ryanbehdad

# SHAP's force plot does not label all the important features
# We usually need to get the top (20) feautures that affect a decision for a particular instance
# In addition to their name, the features' values and their shapley values are also required.
# The below snippet 
# 1. creates a dataframe containing all the features, their shapley value and their actual value
# 2. and exports the dataframe to a csv file
# 3. It also displays the force plot

import shap
shap.initjs()

ryanbehdad

# =======================================================================
# Print a summary of a pandas dataframe and its columns
# =======================================================================
def df_summary(df):
    print(f'Dataframe has {df.shape[0]:,} rows and {df.shape[1]:,} columns')
    if len(df) > 1:
        summary = pd.DataFrame(df.dtypes, columns=['dtype']).reset_index()
        summary.rename(columns={'index': 'feature'}, inplace=True)
        summary['missing'] = df.isnull().sum().values
        summary['uniques'] = df.nunique().values

ryanbehdad

Conda cheat sheet

Setup a conda environment

  conda create --name ds python=3.11 numpy pandas scikit-learn matplotlib seaborn jupyter plotly ipykernel pyodbc

Activate the environment

activate ds

ryanbehdad

bash commands

mkdir

Make directory mkdir folder_name

Can create multiple directories in one command

mkdir Jan Feb Mar

ryanbehdad

Running RStudio in docker

Run docker

To start the docker container, run the following command:

docker run -d -p 8787:8787 -v C:\source:/home/rstudio -e ROOT=TRUE -e PASSWORD=rstudio rocker/rstudio

In the above command, I'm mounting "C:\source" to "/home/rstudio", thus providing the container access to all the contents of "C:\source".

If you want to mount multiple paths use -v multiple time. Example:

ryanbehdad

Python venv virtual environment cheat sheet

Create a venv

To create a virtual environment, go to the root of your project and run

python -m venv venv

It will create a virtual environment called venv

Activate venv

.\venv\Scripts\activate

ryanbehdad

Python venv virtual environment cheat sheet

Create a venv

To create a virtual environment, go to the root of your project and run

python -m venv venv

It will create a virtual environment called venv

Activate venv

.\venv\Scripts\activate

ryanbehdad

Useful VS Code Keyboard Shortcuts

• Ctrl + R Switching workspaces
• Ctrl + P Go to file
• Ctrl+B Toggle Sidebar Visibility
• F4 Focus Next Search Result
• Shift+F4 Focus Previous Search Result
• Ctrl+` Toggle Integrated Terminal
• Ctrl+Space Auto-complete
• Crtl + Alt + up (or down): repeat cursor
• ALT + CLICK — Repeat cursor

ryanbehdad

Using Visual Studio Code for Programming in R

Extensions

The extensions to install:

• R (https://marketplace.visualstudio.com/items?itemName=Ikuyadeu.r)
• R LSP Client (https://marketplace.visualstudio.com/items?itemName=REditorSupport.r-lsp)
• R Tools (https://marketplace.visualstudio.com/items?itemName=Mikhail-Arkhipov.r)

R Language Server

For R LSP, you need to have R language server installed.