Sumit Kumar Ghosh skghosh-invn
skghosh-invn
/ dynamo_json_to_csv.py
Created
August 23, 2019 14:27
— forked from eliocapelati/dynamo_json_to_csv.py
Convert a dynamodb result [json] to csv
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/usr/bin/python3 | |
| # -*- coding: UTF-8 -*- | |
| __author__ = "Elio Capelati Jr" | |
| __version__ = "1.0.0" | |
| import csv, json, sys, argparse | |
| from pprint import pprint |
skghosh-invn
/ emr_spark_thrift_on_yarn
Created
June 8, 2016 19:18
— forked from elliottcordo/emr_spark_thrift_on_yarn
EMR spark thrift server
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #on cluster | |
| thrift /spark/sbin/start-thriftserver.sh --master yarn-client | |
| #ssh tunnel, direct 10000 to unused 8157 | |
| ssh -i ~/caserta-1.pem -N -L 8157:ec2-54-221-27-21.compute-1.amazonaws.com:10000 hadoop@ec2-54-221-27-21.compute-1.amazonaws.com | |
| #see this for JDBC config on client http://blogs.aws.amazon.com/bigdata/post/TxT7CJ0E7CRX88/Using-Amazon-EMR-with-SQL-Workbench-and-other-BI-Tools |