Skip to content

Instantly share code, notes, and snippets.

@fclesio
fclesio / log_config.py
Created January 25, 2021 11:05
Custom logging in Airflow.
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
• Installing prophet (1.1.3): Failed
ChefBuildError
Backend subprocess exited when trying to invoke build_wheel
running bdist_wheel
running build
running build_py
creating build
@fclesio
fclesio / darts-install-m1.sh
Last active July 14, 2023 12:32
Sequence to Install Darts in MacBook M1 - OS: MacOS Ventura
$ conda install -c conda-forge pip
$ conda install -c conda-forge 'lightgbm>=3.3.3'
$ conda install -c conda-forge cmdstan
$ export PROPHET_REPACKAGE_CMDSTAN=False
$ export PRECOMPILED_HEADERS=false
@fclesio
fclesio / get_classification_report.py
Created March 4, 2020 12:21
Scikit Learn Classification Report in Dataframe
def get_classification_report(y_test, y_pred):
'''Source: https://stackoverflow.com/questions/39662398/scikit-learn-output-metrics-classification-report-into-csv-tab-delimited-format'''
from sklearn import metrics
report = metrics.classification_report(y_test, y_pred, output_dict=True)
df_classification_report = pd.DataFrame(report).transpose()
df_classification_report = df_classification_report.sort_values(by=['f1-score'], ascending=False)
return df_classification_report
@fclesio
fclesio / multi-container-pod-cloud-sql-proxy.yaml
Created November 1, 2022 15:29
Sidecar container to connect to Google CloudSQL via Proxy
apiVersion: v1
kind: Pod
metadata:
name: test-proxy-deploy
spec:
volumes:
- name: secret-volume
secret:
secretName: cloud_sql_key_secret
containers:
import awswrangler as wr
import pandas as pd
df = pd.DataFrame({"id": [1, 2], "value": ["foo", "boo"]})
# Armazenando os dados no Data Lake
wr.s3.to_parquet(
df=df,
path="s3://bucket/dataset/",
dataset=True,
@fclesio
fclesio / docker-thanos.sh
Last active September 30, 2022 09:46
Stop all running containers, delete containers and images
## Ref: https://stackoverflow.com/questions/44785585/how-to-delete-all-docker-local-docker-images
docker container prune -f &&
docker stop $(docker ps -aq) &&
docker rm -vf $(docker ps -a -q) &&
docker rmi -f $(docker images -a -q)
[
{
"Id":"value",
"prediction":"value",
"status-code":200
}
]
@fclesio
fclesio / text_classification_logging_payload.json
Last active September 15, 2022 14:34
Generic json template for ML APIs for a Text Classification
{
"message":"OK",
"method":"POST",
"status-code":200,
"timestamp":"2022-01-01T00:00:00.000000",
"url":"http://0.0.0.0:1201/predict",
"data":{
"prediction":[
{
"input_text":"Flash crashes is a scenario where the price of stocks plunges but then quickly recovers.",
# Special thanks for the user Humberto Diogenes from Python List (answer from Aug 11, 2008)
# Link: http://python.6.x6.nabble.com/O-jeito-mais-rapido-de-remover-acentos-de-uma-string-td2041508.html
# I found the issue by chance (I swear, haha) but this guy gave the tip before me
# Link: https://github.com/scikit-learn/scikit-learn/issues/12897#issuecomment-518644215
import spacy
from unicodedata import normalize
nlp = spacy.load('pt_core_news_sm')