Skip to content

Instantly share code, notes, and snippets.

View lucaspar's full-sized avatar

Lucas Parzianello lucaspar

  • University of Notre Dame
View GitHub Profile
@lucaspar
lucaspar / pyarrow_conversions.py
Created June 14, 2023 20:03
Benchmark | PyArrow conversions
"""Benchmark the conversion of pyarrow arrays to polars, numpy, and lists."""
import time
from typing import Callable
import numpy as np
import pandas as pd
import polars as pl
import psutil
import pyarrow as pa
@lucaspar
lucaspar / Debugging.md
Created June 24, 2023 21:39
Debugging in containers

Debugging in containers

Tired of "command not found" - these lines will install commonly used tools to debug networking, read logs, and other things.

Start a shell as root inside the container

CONTAINER_KEYWORD="spark"
CONTAINER_NAME=$(docker ps | grep -i "${CONTAINER_KEYWORD}" | head -n 1 | awk '{print $NF}')
echo "CONTAINER_NAME: ${CONTAINER_NAME}"
@lucaspar
lucaspar / waste_my_ram.py
Created July 14, 2023 20:17
Progressively allocates all memory in a system
"""You probably don't want to run this.
Source: https://stackoverflow.com/a/66109163/2848528
"""
import logging
import secrets
from typing import Optional
from humanfriendly import format_size
@lucaspar
lucaspar / heimdall-style.css
Created July 20, 2023 15:18
Heimdall Style
/* Paste these lines in Settings > Advanced > Custom CSS */
/* ================ */
/* generic elements */
body {
background-color: rgb(16, 16, 16);
color: rgb(198, 198, 198);
/* font-family: monospace; */
line-height: 1.8;
}
@lucaspar
lucaspar / slack-notify
Created October 6, 2023 14:44
Slack notification script
#!/bin/env bash
# Slack Notifications
SLACK_WEBHOOK_URL_DEFAULT="https://hooks.slack.com/services/GET/A/LINK"
SLACK_WEBHOOK_URL="$SLACK_WEBHOOK_URL_DEFAULT"
SLACK_PREFIX="${SLACK_PREFIX:=\`$(hostname)\`}"
# notify about error on slack channel if errors happened
function _slack_notify() {
local ERROR_MESSAGE="$SLACK_PREFIX | $1"