Skip to content

Instantly share code, notes, and snippets.

View luizvbo's full-sized avatar

Luiz Otavio Vilas Boas Oliveira luizvbo

14 followers · 1 following
View GitHub Profile
@luizvbo
luizvbo / concatenator.sh
Last active July 25, 2025 09:27
Concatenate multiple files into a single output fFile, ideal for preparing data for Generative AI models. You can customize the output file, file patterns, and clearing options to streamline your workflow.
#!/bin/bash
usage="
Script to concatenate files matching patterns into a single output file
Usage:
$(basename "$0") [-h] [-o output_file] [-p patterns] [-c] input_folder
where:
-h show this help text
@luizvbo
luizvbo / pyspark_iceberg_example.md
Last active February 6, 2025 13:02
Simple example of how you can run PySpark locally with Iceberg

Introduction

Simple example of how you can run PySpark locally with Iceberg. There's no need to spin up Docker containers or install additional packages (besides PySpark). The script demonstrates how to create a Spark session with Iceberg support, create a sample DataFrame, and save it as an Iceberg table

Code