Skip to content

Instantly share code, notes, and snippets.

View nicor88's full-sized avatar
:octocat:

nicor88 nicor88

:octocat:
115 followers · 206 following
  • Berlin, Germany
View GitHub Profile
@nicor88
nicor88 / gist:1d39a19ed8d4c840c1321af463c8b642
Created August 20, 2021 10:04 — forked from CristinaSolana/gist:1885435
Keeping a fork up to date

1. Clone your fork:

git clone git@github.com:YOUR-USERNAME/YOUR-FORKED-REPO.git

2. Add remote from original repository in your forked repository:

cd into/cloned/fork-repo
git remote add upstream git://github.com/ORIGINAL-DEV-USERNAME/REPO-YOU-FORKED-FROM.git
git fetch upstream
@nicor88
nicor88 / ecs_task_definition.json
Last active October 5, 2019 12:55
dbt_serverless
[{
"name": "${container_name}",
"image": "${image}:${image_version}",
"essential": true,
"environment": [
{
"name": "DB_HOST",
"value": "${db_host}"
},
{
@nicor88
nicor88 / dbt_operator.py
Last active September 28, 2019 15:42
Airflow ECS dbt operator
import logging
import pprint
import sys
from airflow.exceptions import AirflowException
from airflow.models import BaseOperator
from airflow.plugins_manager import AirflowPlugin
from airflow.utils import apply_defaults
from airflow.contrib.hooks.aws_hook import AwsHook
@nicor88
nicor88 / mqtt_spark_streaming.py
Created July 12, 2017 20:17 — forked from kartben/mqtt_spark_streaming.py
Example of how to use Spark Streaming for MQTT data consolidation
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
@nicor88
nicor88 / jupyter_pyspark_examples.ipynb
Created May 31, 2017 16:17
Jupyter Pyspark Examples
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@nicor88
nicor88 / bootstrap_jupyter.sh
Created April 20, 2017 10:23
Bootstrap action to install Conda and Jupyter on EMR
#!/usr/bin/env bash
set -x -e
JUPYTER_PASSWORD=${1:-"myJupyterPassword"}
NOTEBOOK_DIR=${2:-"s3://myS3Bucket/notebooks/"}
# home backup
if [ ! -d /mnt/home_backup ]; then
sudo mkdir /mnt/home_backup
sudo cp -a /home/* /mnt/home_backup