Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.
angeligareta
/ generate_class_weights.py
Created
June 8, 2021 10:28
Method to generate class weights given a multi-class or multi-label set of classes using Python, supporting one-hot-encoded labels.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import numpy as np | |
| from sklearn.utils.class_weight import compute_class_weight | |
| from sklearn.preprocessing import MultiLabelBinarizer | |
| def generate_class_weights(class_series, multi_class=True, one_hot_encoded=False): | |
| """ | |
| Method to generate class weights given a set of multi-class or multi-label labels, both one-hot-encoded or not. | |
| Some examples of different formats of class_series and their outputs are: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| FROM alpine:3.13 | |
| WORKDIR /opt/draft-proxy | |
| RUN apk update && apk add git curl | |
| RUN curl -L -o package.tgz https://github.com/oauth2-proxy/oauth2-proxy/releases/download/v6.1.1/oauth2-proxy-v6.1.1.linux-amd64.tar.gz && \ | |
| tar xvzf package.tgz && \ | |
| mv oauth2-proxy-*.linux-amd64/oauth2-proxy . | |
| CMD ["./oauth2-proxy", \ | |
| "--provider=github", "--github-org=YOUR_GITHUB_ORG", "--email-domain=*", \ | |
| "--http-address=0.0.0.0:8080", \ | |
| "--reverse-proxy=true", \ |
hvy
/ quadratic_joblib_arbitrary_arguments_simple.py
Last active
December 7, 2020 04:45
Optuna example that optimizes a simple quadratic function in parallel using joblib with arbitrary arguments to the objective function.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """ | |
| Optuna example that optimizes a simple quadratic function in parallel using `joblib` allowing | |
| arbitrary arguments to the objective function. | |
| Run the example as follows. | |
| $ python quadratic_joblib_simple.py | |
| If you need to rerun the example and thus delete previous studies, you can use the Optuna CLI. |
inchoate
/ including_external_package_in_dataflow.md
Last active
February 2, 2024 11:40
Adding an extra package to a Python Dataflow project to run on GCP
The documentation for how to deploy a pipeline with extra, non-PyPi, pure Python packages on GCP is missing some detail. This gist shows how to package and deploy an external pure-Python, non-PyPi dependency to a managed dataflow pipeline on GCP.
TL;DR: You external package needs to be a python (source/binary) distro properly packaged and shipped alongside your pipeline. It is not enough to only specify a tar file with a setup.py.
Your external package must have a proper setup.py. What follow is an example setup.py for our ETL package. This is used to package version 1.1.1 of the etl library. The library requires 3 native PyPi packages to run. These are specified in the install_requires field. This package also ships with custom external JSON data, declared in the package_data section. Last, the setuptools.find_packages function searches for all available packages and returns that
As configured in my dotfiles.
start new:
tmux
start new with session name:
jaseemabid
/ git tutorials.md
Last active
March 24, 2024 00:07
— forked from netroy/git tutorials.md
Awesome git tutorials I am finding here and there
-
Simplest intro to git by github and codeschool - Try Git
-
[Intro to github]
GaelVaroquaux
/ 00README.rst
Last active
September 15, 2023 03:58
Copy-less bindings of C-generated arrays with Cython
The goal of this example is to show how an existing C codebase for numerical computing (here c_code.c) can be wrapped in Cython to be exposed in Python.
The meat of the example is that the data is allocated in C, but exposed in Python without a copy using the PyArray_SimpleNewFromData numpy