This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
name: Run PyTest | |
on: | |
[ pull_request ] | |
jobs: | |
pytest: | |
name: Run Tests | |
runs-on: ubuntu-latest | |
steps: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# This workflow finds which files was changed, print them, and then | |
# run `pre-commit` on those files. I was inspired by sktime: | |
# https://github.com/alan-turing-institute/sktime/blob/main/.github/workflows/build-and-test.yml | |
name: Code Quality | |
on: | |
pull_request: | |
branches: | |
- main |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
repos: | |
- repo: https://github.com/pre-commit/pre-commit-hooks | |
rev: v3.4.0 | |
hooks: | |
# list of supported hooks: https://pre-commit.com/hooks.html | |
- id: trailing-whitespace | |
- id: end-of-file-fixer | |
- id: check-yaml | |
- id: check-added-large-files | |
- id: debug-statements |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import hydra | |
from omegaconf import DictConfig | |
@hydra.main(config_path="configs/", config_name="config.yaml") | |
def main(config: DictConfig): | |
# instantiate our preprocessing pipeline to | |
# sklearn Pipeline object using hydra configuration | |
preprocessing_pipeline = hydra.utils.instantiate( |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# @package _global_ | |
# specify here default preprocessing configuration | |
defaults: | |
# can be any config file in the preprocessing_pipeline folder | |
- preprocessing_pipeline: decision_tree.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import hydra | |
from omegaconf import DictConfig | |
from sklearn.pipeline import Pipeline | |
def make_pipeline(steps_config: DictConfig) -> Pipeline: | |
"""Creates a pipeline with all the preprocessing steps specified in `steps_config`, ordered in a sequential manner | |
Args: | |
steps_config (DictConfig): the config containing the instructions for |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Hydra config file for the decision_tree preprocessing steps | |
_target_: hydra_sklearn_pipeline.make_pipeline | |
#StepName: | |
# _target_: <class to instantiate the step> | |
# param1: <step's first parameter> | |
# param2: <step's second parameter, etc.> | |
steps_config: # use yaml list syntax to preserve to order | |
- StandardScaler: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from sklearn.decomposition import PCA | |
from sklearn.ensemble import RandomForestClassifier | |
from sklearn.linear_model import LogisticRegression | |
from sklearn.tree import DecisionTreeClassifier | |
pipeline_lr = Pipeline( | |
[ | |
("imputer", SimpleImputer(strategy="constant", fill_value="-1")), | |
("VarianceThreshold", VarianceThreshold(threshold=0.1)), | |
("scalar1", StandardScaler()), |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from sklearn.feature_selection import VarianceThreshold | |
from sklearn.impute import SimpleImputer | |
from sklearn.pipeline import Pipeline | |
from sklearn.preprocessing import StandardScaler | |
num_pipeline = Pipeline( | |
[ | |
("scaler", StandardScaler()), | |
("VarianceThreshold", VarianceThreshold(threshold=0.1)), | |
("imputer", SimpleImputer(strategy="constant", fill_value="-1")), |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# # This script will unrar the RARs in $src to $dest. Based on: | |
# https://serverfault.com/questions/356184/powershell-unrar-with-wildcard/356208?newreg=44a1363382a04e7a95a915434574dec2 | |
param ( | |
[Parameter(Mandatory=$true)][string]$src, | |
[Parameter(Mandatory=$true)][string]$dest | |
) | |
$files = @() |
NewerOlder