Skip to content

Instantly share code, notes, and snippets.

View skozz's full-sized avatar
🏠
Working from home

Angel skozz

🏠
Working from home
View GitHub Profile
@mehd-io
mehd-io / slack_failed_task.md
Last active February 25, 2019 10:28
Airflow - proper Slack notifications

Define the custom function in the dag :

sc = SlackClient(my_slack_token)

def slack_failed_task(contextDictionary, **kwargs):
        base_url = mybasedUrl
        task_instance = contextDictionary.get('task_instance')
        log_url = '{task_instance.log_url}'.format(**locals())
 failed_alert = sc.api_call('chat.postMessage',
@rosiehoyem
rosiehoyem / getting-started-with-airflow.md
Last active January 14, 2023 18:09
Getting Started With Airflow

Install Airflow

1. Install Airflow

Follow the installation instructions on the Airflow website.

Update Airflow Configurations

To configure Airflow to use Postgres rather than the default Sqlite3, go to airflow.cfg and update this configuration to LocalExecutor:

# The executor class that airflow should use. Choices include
@Bhavdip
Bhavdip / sketch-never-ending.md
Created October 6, 2016 15:53
Modify Sketch to never ending trial

###Sketch trial non stop

Open hosts files:

$ open /private/etc/hosts

Edit the file adding:

127.0.0.1 backend.bohemiancoding.com

127.0.0.1 bohemiancoding.sketch.analytics.s3-website-us-east-1.amazonaws.com

@PavloBezpalov
PavloBezpalov / 1. ELK.install
Last active January 5, 2024 16:14
ELK Stack with Rails (Elasticsearch, Logstash, Kibana) on Ubuntu VPS
INSTALL JAVA
$ sudo apt-get update && sudo apt-get install default-jre
INSTALL ELASTIC SEARCH https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html
$ wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
$ echo "deb https://packages.elastic.co/elasticsearch/2.x/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-2.x.list
$ sudo apt-get update && sudo apt-get install elasticsearch
$ sudo update-rc.d elasticsearch defaults 95 10
$ sudo service elasticsearch restart
$ sudo service elasticsearch status
@skozz
skozz / s3.sh
Created May 18, 2016 16:59 — forked from chrismdp/s3.sh
Uploading to S3 in 18 lines of Shell (used to upload builds for http://soltrader.net)
# You don't need Fog in Ruby or some other library to upload to S3 -- shell works perfectly fine
# This is how I upload my new Sol Trader builds (http://soltrader.net)
# Based on a modified script from here: http://tmont.com/blargh/2014/1/uploading-to-s3-in-bash
S3KEY="my aws key"
S3SECRET="my aws secret" # pass these in
function putS3
{
path=$1
@vasanthk
vasanthk / System Design.md
Last active April 26, 2024 18:05
System Design Cheatsheet

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

  1. Clarify and agree on the scope of the system
  • User cases (description of sequences of events that, taken together, lead to a system doing something useful)
    • Who is going to use it?
    • How are they going to use it?
@icebeat
icebeat / tripadvisor.js
Created January 11, 2016 09:59
Tripadvisor parser with x-ray
"use strict";
const Xray = require('x-ray');
const x = Xray();
const fs = require('fs');
const tripavisorUrl = 'http://www.tripadvisor.com/Restaurant_Review-g187514-d4760195-Reviews-La_Huerta_de_Tudela-Madrid.html';
x(tripavisorUrl, {
id: '.mapContainer@data-locid',
#!/usr/bin/env ruby
require 'time'
require 'logger'
require 'aws-sdk'
AWS_REGION = "us-east-1"
AWS_KEY_ID = ""
AWS_SECRET_KEY = ""
@PurpleBooth
PurpleBooth / README-Template.md
Last active April 26, 2024 17:22
A template to make good README.md

Project Title

One Paragraph of project description goes here

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Prerequisites

@ozgun
ozgun / custom_responder.rb
Created June 23, 2015 08:17
Custom Responder
module MyModule
class CustomResponder < ActionController::Responder
include Responders::FlashResponder
include Responders::HttpCacheResponder
def has_errors?
case controller.action_name
when 'destroy'
resource.destroyed? ? false : true
else