Skip to content

Instantly share code, notes, and snippets.

View maatthc's full-sized avatar

Alexandre Andrade maatthc

  • MaaT Tech
  • Melbourne
View GitHub Profile
@maatthc
maatthc / Glue_Spark_job_example.yml
Last active February 10, 2022 18:02
Cloud Formation example for Glue Spark Job with metrics and scheduler
AWSTemplateFormatVersion: '2010-09-09'
Description: Cloud Formation example for Glue Spark Job with metrics and scheduler
Parameters:
ArtifactBucket:
Description: A global deployable artefact bucket
Type: String
Default: artefacts
ServiceName:
Description: Service Name that owns the stack when created
@maatthc
maatthc / find_latest_file_in_s3_bucket.py
Last active May 22, 2019 07:41
Find the latest file in a AWS S3 Bucket using Boto3
import boto3
def find_latest_file_in_folder(bucket, path):
s3 = boto3.client('s3')
s3_keys = s3.list_objects_v2(Bucket=bucket, Prefix=path)['Contents']
last_added = max(s3_keys, key=lambda x: x['LastModified'])
return "%s/%s" % (bucket,last_added['Key'])
@maatthc
maatthc / process_csv_using_stream.js
Created May 14, 2019 04:13
Avoid the "JavaScript heap out of memory" : Example of how to process huge CSV files using Streams in Nodejs.
#!/usr/bin/env node
const fs = require('fs')
const path = require('path')
const moment = require('moment')
const csv = require('fast-csv')
const highland = require('highland')
const writeableStream = fs.createWriteStream(path.join(__dirname, 'input.csv'))
const csvStream = csv.fromPath(path.join(__dirname, 'output.csv'), { headers: true, trim: true, objectMode: true })
const outputStream = csv.createWriteStream({ headers: true })
@maatthc
maatthc / 90-notify_ip_to_slack.sh
Last active May 14, 2019 04:09
Shell script to post to Slack every time the Raspberry Pi changes its IP Address via DHCP. Should be in the folder /lib/dhcpcd/dhcpcd-hooks/
#!/bin/bash
if [ "$interface" = wlan0 ]; then
if [ "$if_up" = true ]; then
IP=`ip add show wlan0|grep global |awk '{print $2}'`
PREVIOUS_IP=`cat /tmp/notify_output`
if [ "$PREVIOUS_IP" != "$IP" ]; then
echo 'Notifying ip: '$IP' to Slack Channel..'
echo $IP > /tmp/notify_output
curl -X POST -H 'Content-type: application/json' --data '{"text":"RaspBerry TV IP is now: '${IP}'","icon_emoji":":tv:", "username":"RaspBerry"}' https://hooks.slack.com/services/XXXX
fi
@maatthc
maatthc / sad.js
Last active November 19, 2022 09:25
Search and Destroy - Free up disk space deleting "node_modules" folders recursively
#!/usr/bin/env node
// Search and Destroy
//
const path = require('path')
const fs = require('fs')
const createPackageJson = () => {
const packageJsonContent = `
{
"name": "sad",
@maatthc
maatthc / api-gw-template.yml
Last active October 13, 2023 14:52
AWS CloudFormation template example for allowing uploading files to S3 via API Gateway
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
ApiGatewayStack:
Description: The stack where the api gateway rest api is defined
Type: String
Default: dev-maat-apigw-app
EventBucket:
Type: String
Default: dev-maat-bucket
CorsAllowOrigin:
AWSTemplateFormatVersion: "2010-09-09"
Description: "An example lambda template to demonstrate persistent state between executions"
Resources:
LambdaExecutionRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
@maatthc
maatthc / aws_sam_template.yml
Created March 13, 2019 22:57
Basic AWS SAM template to run NodeJs Lambdas locally: sam local invoke -t sam_template.yml -e test/fixtures/s3-to-lambda-put-event.json
AWSTemplateFormatVersion : '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
LambdaFunction:
Type: AWS::Serverless::Function
Properties:
Handler: index.handler
Runtime: nodejs8.10
MemorySize: 128
Timeout: 15
### BEGIN INIT INFO
# Provides: access-point
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Control hostapd access point
# Description: Control hostapd Wifi access point on Raspberry Pi 3 running Kali Linux
### END INIT INFO
MENUDIR=./
@maatthc
maatthc / nativescript_androidStudio.sh
Created May 14, 2018 06:44
How to integrate Android Studio without SDK to NativeScript without sudo
# Download the "Android Studio" (version 3.2.1 as today) from
# https://developer.android.com/studio/#downloads
# Extract and move to the Desktop
# Download the "Command line tools only" from the same page
# Extrat to ~/Downloads and then:
cd ~/Desktop/Android\ Studio.app/
mv ~/Downloads/tools ./
# Download the "SDK Platform Tools" from