Skip to content

Instantly share code, notes, and snippets.

View monkut's full-sized avatar

monkut monkut

28 followers · 1 following
View GitHub Profile
@monkut
monkut / boto3_service_endpoints.py
Created September 20, 2019 06:00
boto3 defined default service endpoints definition for use with localstack
import os
AWS_REGION = os.getenv('AWS_DEFAULT_REGION', 'ap-northeast-1')
DEFAULT_S3_SERVICE_ENDPOINT = f'https://s3.{AWS_REGION}.amazonaws.com'
DEFAULT_SQS_SERVICE_ENDPOINT = f'https://sqs.{AWS_REGION}.amazonaws.com'
DEFAULT_SNS_SERVICE_ENDPOINT = f'https://sns.{AWS_REGION}.amazonaws.com'
AWS_SERVICE_ENDPOINTS = {
's3': os.getenv('S3_SERVICE_ENDPOINT', DEFAULT_S3_SERVICE_ENDPOINT),
@monkut
monkut / mediainfo_xml_parse.py
Created September 20, 2019 05:56
a python parser for `mediainfo` output
import json
import subprocess
from xml.etree import ElementTree
import boto3
from . import settings
S3 = boto3.client(
's3',
endpoint_url=settings.AWS_SERVICE_ENDPOINTS['s3'],
)
@monkut
monkut / tilemill_ubuntu18.04_install.md
Last active September 11, 2019 09:22
install tilemill in ubuntu 18.04

install libs

sudo apt-get install autoconf automake libtool curl make g++ unzip -y

clone and install protobuf

cd ~
@monkut
monkut / alllow-aws-crossaccount-role-access.md
Last active September 9, 2019 09:54
notes on creating an AWS cross account role

  1. Create User (User1A) in Account-A

    • Get AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

  2. Create Group in Account-A

  3. Create Role (MyRoleB) in Account-B with Permissions you want user to have:

@monkut
monkut / parse_aws_arn.py
Last active August 8, 2019 23:51
parse AWS ARN
def parse_arn(arn_str: str) -> Tuple[str, str, str, Optional[str], str, Optional[List[str]]]:
"""
https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html
arn:partition:service:region:account-id:resource
arn:partition:service:region:account-id:resourcetype/resource
arn:partition:service:region:account-id:resourcetype/resource/qualifier
arn:partition:service:region:account-id:resourcetype/resource:qualifier
arn:partition:service:region:account-id:resourcetype:resource
arn:partition:service:region:account-id:resourcetype:resource:qualifier
@monkut
monkut / gist:1d00900b4a633f407cd1a00c2217666c
Created June 30, 2019 07:12
video_transfer.sh
#!/bin/bash
# Command to transfer
export VIDEO_DIRECTORY=/storage/VIDEO/HOME/$(python3 -c "import datetime;print(datetime.datetime.now().strftime('%Y%m%d-video-dump'), end='')")
echo Making directory: ${VIDEO_DIRECTORY}
mkdir ${VIDEO_DIRECTORY}
echo Copying...
rsync --progress -h /media/monkut/JVCCAM_MEM/AVCHD/BDMV/STREAM/*.MTS ${VIDEO_DIRECTORY}
@monkut
monkut / PrepareAWSClientVPN.md
Created June 14, 2019 06:58
Prepare AWS Client VPN for connecting to a VPC without NAT/IGW

Prepare AWS Client VPN README

Prerequisites

  • AWS Account
  • Linux(like) System
  • awscli
@monkut
monkut / get_s3_latest_sagemaker_model_assets_key.py
Last active March 26, 2019 07:25
"""
Get the latest model assets key from resulting sagemaker training session
"""
import boto3
S3 = boto3.client('s3')
def get_latest_model_assets_key(bucket: str, prefix: str):
model_assets = []
paginator = S3.get_paginator('list_objects')
@monkut
monkut / ssh-tunnel-through-bastion-server.md
Created February 14, 2019 01:30
SSH Tunnel Trhough Bastion Server

SSH Tunnelにより、同じPCにDBがあるように接続する

  • Committing deleted app
  1. 自分のPCからTunnel貼る：

postgresqlを使用例

# LOCAL_PORT:REMOTE_ADDRESS:REMOTE_PORT
ssh -nNT -L 5432:{VPC内のRDSIPADDRESS}:5432 -i ~/.ssh/bastion.pem ubuntu@{IP ADDRESS} &
@monkut
monkut / add_python_to_WINDOWS_WSL_ubuntu.md
Created February 13, 2019 13:19
Add python to Windows Subsystem for Linux (WSL) [ubuntu]