Skip to content

Instantly share code, notes, and snippets.

View elisim's full-sized avatar
:octocat:
Deep Learning

Eli Simhayev elisim

:octocat:
Deep Learning
View GitHub Profile
@elisim
elisim / pytest.yml
Created August 10, 2023 13:07
Run PyTest GitHub Actions
name: Run PyTest
on:
[ pull_request ]
jobs:
pytest:
name: Run Tests
runs-on: ubuntu-latest
steps:
# This workflow finds which files was changed, print them, and then
# run `pre-commit` on those files. I was inspired by sktime:
# https://github.com/alan-turing-institute/sktime/blob/main/.github/workflows/build-and-test.yml
name: Code Quality
on:
pull_request:
branches:
- main
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.4.0
hooks:
# list of supported hooks: https://pre-commit.com/hooks.html
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
- id: debug-statements
@elisim
elisim / driver_code.py
Last active September 2, 2021 09:35
Hydra-Sklearn Blog - Code example 6
import hydra
from omegaconf import DictConfig
@hydra.main(config_path="configs/", config_name="config.yaml")
def main(config: DictConfig):
# instantiate our preprocessing pipeline to
# sklearn Pipeline object using hydra configuration
preprocessing_pipeline = hydra.utils.instantiate(
@elisim
elisim / config.yaml
Created September 2, 2021 09:30
Hydra-Sklearn Blog - Code example 5
# @package _global_
# specify here default preprocessing configuration
defaults:
# can be any config file in the preprocessing_pipeline folder
- preprocessing_pipeline: decision_tree.yaml
@elisim
elisim / make_pipeline.py
Created September 2, 2021 09:26
Hydra-Sklearn Blog - Code example 4
import hydra
from omegaconf import DictConfig
from sklearn.pipeline import Pipeline
def make_pipeline(steps_config: DictConfig) -> Pipeline:
"""Creates a pipeline with all the preprocessing steps specified in `steps_config`, ordered in a sequential manner
Args:
steps_config (DictConfig): the config containing the instructions for
@elisim
elisim / hydra_config_file.yaml
Last active September 2, 2021 10:54
Hydra-Sklearn Blog - Code example 3
# Hydra config file for the decision_tree preprocessing steps
_target_: hydra_sklearn_pipeline.make_pipeline
#StepName:
# _target_: <class to instantiate the step>
# param1: <step's first parameter>
# param2: <step's second parameter, etc.>
steps_config: # use yaml list syntax to preserve to order
- StandardScaler:
@elisim
elisim / different_pipelines_for_different_models.py
Last active September 2, 2021 09:16
Hydra-Sklearn Blog - Code example 2
from sklearn.decomposition import PCA
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
pipeline_lr = Pipeline(
[
("imputer", SimpleImputer(strategy="constant", fill_value="-1")),
("VarianceThreshold", VarianceThreshold(threshold=0.1)),
("scalar1", StandardScaler()),
@elisim
elisim / standard_pre_processing_pipeline.py
Last active September 2, 2021 09:17
Hydra-Sklearn Blog - Code example 1
from sklearn.feature_selection import VarianceThreshold
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
num_pipeline = Pipeline(
[
("scaler", StandardScaler()),
("VarianceThreshold", VarianceThreshold(threshold=0.1)),
("imputer", SimpleImputer(strategy="constant", fill_value="-1")),
@elisim
elisim / unrar-multiple-files.ps1
Last active May 3, 2021 13:27
Powershell script to unrar RAR files in a directory a given destination
# # This script will unrar the RARs in $src to $dest. Based on:
# https://serverfault.com/questions/356184/powershell-unrar-with-wildcard/356208?newreg=44a1363382a04e7a95a915434574dec2
param (
[Parameter(Mandatory=$true)][string]$src,
[Parameter(Mandatory=$true)][string]$dest
)
$files = @()