Skip to content

Instantly share code, notes, and snippets.

View mpangrazzi's full-sized avatar
💭
🎸

Michele Pangrazzi mpangrazzi

💭
🎸
View GitHub Profile
@mpangrazzi
mpangrazzi / update-helm-release-namespace.sh
Created October 6, 2023 20:28
Update Helm release namespace without having to delete and recreate it.
#!/bin/bash
#
# This script will patch a Helm release secret and move it to another namespace.
# It will also delete the original secret after the patching is done.
# It's useful when you want to move a Helm release to another namespace without having to delete and recreate it.
#
# Credits for the original idea: https://blog.random.io/moving-helm-release-to-another-kubernetes-namespace
#
@mpangrazzi
mpangrazzi / multiprocessing-parallelism.py
Created January 22, 2023 13:09
Process-based parallelism with Python when processing a 500k+ rows dataset
from multiprocessing import cpu_count, get_context
from tqdm.auto import tqdm
from time import sleep
# A fake function to simulate some work
def process_data(x: str) -> str:
sleep(0.001)
return x + " processed"
def main():
@mpangrazzi
mpangrazzi / categorization_fill_mask.py
Created September 20, 2022 19:54
Efficiently use a fill-mask pipeline on a large pretrained transformers model to categorise a list of words
from transformers import pipeline
pipe = pipeline("fill-mask", model="roberta-large", top_k=1)
def get_topic_using_fill_mask(words):
results = pipe(f"{words}. \n These words are about <mask>.")
return results[0]["token_str"].strip()
get_topic_using_fill_mask(["terra, mars, moon, saturn, jupyter"])
@mpangrazzi
mpangrazzi / openai_api.py
Created September 6, 2022 13:16
Call OpenAI API asking GPT-3 for a Completion
import openai
res = openai.Completion.create(
model="text-davinci-002",
prompt="How are you?\n",
temperature=0,
max_tokens=64,
top_p=1
)
@mpangrazzi
mpangrazzi / amazon_spider.py
Last active September 1, 2022 07:47
A very basic Scrapy spider for amazon.com
from scrapy import Spider
from bs4 import BeautifulSoup
class AmazonSpider(Spider):
name = "amazon"
allowed_domains = ["amazon.com"]
custom_settings = {
@mpangrazzi
mpangrazzi / gist:eb58de6a47ad62c5b79b210cdedcaa2e
Last active September 15, 2016 08:03
Delete all indexes in ElasticSearch
#!/bin/bash
curl 'localhost:9200/_cat/indices/' | awk '{ print $3 }' | xargs -I % curl -XDELETE 'localhost:9200/%'
# - useful if you can't delete using a wildcard
# - `xargs` can be swapped with `GNU parallel`
@mpangrazzi
mpangrazzi / gist:aff33f81607bdb4141f6
Last active August 29, 2015 14:10
Wrap less.render() in a Promise for using with `co` or directly
/**
* Module dependencies
*/
var less = require('less');
// wrap less.render() in a native Promise
@mpangrazzi
mpangrazzi / Lambda-PS1
Created January 25, 2014 10:11
Clean PS1 (with current git branch displayed)
#
# This is a simple, clean PS1 with current git branch displayed.
# How will look like:
#
# λ ~/home/project (master)
#
# NOTE: You may have to configure your terminal to support UTF-8 Unicode (so λ will displayed correctly)
function parse_git_branch () {