Skip to content

Instantly share code, notes, and snippets.

View vcabral19's full-sized avatar

Vitor Cabral vcabral19

  • Miro
  • Amsterdam, Netherlands
View GitHub Profile
@vcabral19
vcabral19 / EnvVarTest.java
Last active July 1, 2021 20:59
playing with variables in java
package com.myproject;
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
public class EnvVarTest {
public final Properties properties;
@vcabral19
vcabral19 / async_kafka_prod_cons.py
Created July 1, 2021 20:58
playing with async python and kafka
import asyncio
from confluent_kafka import Consumer, Producer
from confluent_kafka.admin import AdminClient, NewTopic
BROKER_URL = "PLAINTEXT://localhost:9092"
TOPIC_NAME = "my-first-python-topic"
@vcabral19
vcabral19 / sync_kafka_prod.py
Last active July 4, 2021 19:56
using dataclass with faker (to generate random values) I've played with producing messages to kafka in a sync fashion
from dataclasses import dataclass, field
import json
import random
from confluent_kafka import Consumer, Producer
from confluent_kafka.admin import AdminClient, NewTopic
from faker import Faker
faker = Faker()
@vcabral19
vcabral19 / date_utils_for_spark_etl.py
Created May 13, 2022 12:56
some useful date functions (python + pyspark)
from datetime import date, datetime
def _convert_date_string_to_epoch(date_string: str) -> Union[int, float]:
date_iso = date.fromisoformat(date_string)
return datetime.fromordinal(date_iso.toordinal()).timestamp()
def _convert_epoch_to_date_string(epoch_time: int) -> str:
date_ts = datetime.fromtimestamp(epoch_time)
return date_ts.strftime("%Y-%m-%d")