absognety

#!/usr/bin/python
# $Id: $

# Converts Oracle, SQL-Server, and other DDL to Snowflake DDL


def usage():
    print """\
# Usage: sql2sf.py input-file [output-file]
    """

absognety

Using Docker and docker-compose to manage Apache Airflow on mac

Using our beloved docker and docker-compose, we can very quickly bring up an Apache Airflow instance on our mac.

Contents of docker-compose.yml

About the only thing you need to customize in this docker-compose.yml file is the volumes section. This will tell docker to map the given directory containing your Airflow DAGs/plugins to container file system.

version: '3'
services:

absognety

import copy

# write to a path using the Hudi format
def hudi_write(df, schema, table, path, mode, hudi_options):
    hudi_options = {
        "hoodie.datasource.write.recordkey.field": "recordkey",
        "hoodie.datasource.write.precombine.field": "precombine_field",
        "hoodie.datasource.write.partitionpath.field": "partitionpath_field",
        "hoodie.datasource.write.operation": "write_operaion",
        "hoodie.datasource.write.table.type": "table_type",

absognety

import multiprocessing #:)

def do_this(number):
  print number
  return number*2

# Create a list to iterate over. 
# (Note: Multiprocessing only accepts one item at a time)
some_list = range(0,10)

absognety

Setup multiple git identities & git user informations

/!\ Be very carrefull in your setup : any misconfiguration make all the git config to fail silently ! Go trought this guide step by step and it should be fine 😉

Setup multiple git ssh identities for git

• Generate your SSH keys as per your git provider documentation.
• Add each public SSH keys to your git providers acounts.
• In your ~/.ssh/config, set each ssh key for each repository as in this exemple:

absognety

People

:bowtie:	😄 :smile:	😆 :laughing:
😊 :blush:	😃 :smiley:	☺️ :relaxed:
😏 :smirk:	😍 :heart_eyes:	😘 :kissing_heart:
😚 :kissing_closed_eyes:	😳 :flushed:	😌 :relieved:
😆 :satisfied:	😁 :grin:	😉 :wink:
😜 :stuck_out_tongue_winking_eye:	😝 :stuck_out_tongue_closed_eyes:	😀 :grinning:
😗 :kissing:	😙 :kissing_smiling_eyes:	😛 :stuck_out_tongue:

absognety

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

1. Clarify and agree on the scope of the system

• User cases (description of sequences of events that, taken together, lead to a system doing something useful)
  • Who is going to use it?
  • How are they going to use it?

absognety

# MWS API docs at http://docs.developer.amazonservices.com/en_US/orders-2013-09-01/Orders_Datatypes.html#Order
# MWS Scratchpad at https://mws.amazonservices.com/scratchpad/index.html
# Boto docs at http://docs.pythonboto.org/en/latest/ref/mws.html?#module-boto.mws

from boto.mws.connection import MWSConnection

...

# Provide your credentials.
conn = MWSConnection(

absognety

// Find the minimum path sum (from root to leaf)
public static int minPathSum(TreeNode root) {
	if(root == null) return 0;
	int sum = root.val;
	
	int leftSum = minPathSum(root.left);
	int rightSum = minPathSum(root.right);
	
	if(leftSum < rightSum){
		sum += leftSum;

absognety

1)What is Difference between Secondary namenode, Checkpoint namenode & backupnod Secondary Namenode, a poorly named component of hadoop.

(2)What are the Side Data Distribution Techniques.

(3)What is shuffleing in mapreduce?

(4)What is partitioning?

(5)Can we change the file cached by Distributed Cache