Skip to content

Instantly share code, notes, and snippets.

View doi-t's full-sized avatar
🧗‍♂️
Be open-minded

Toshiya Doi doi-t

🧗‍♂️
Be open-minded
25 followers · 2 following
View GitHub Profile
@doi-t
doi-t / listup_ami_ids.sh
Last active April 14, 2022 11:41
List up all Amazon Linux AMI (HVM / 64-bit) IDs
aws ec2 describe-images --owners self amazon --filters \
Name=virtualization-type,Values=hvm \
Name=architecture,Values=x86_64 \
Name=block-device-mapping.volume-type,Values=gp2 \
| jq -r '.Images | sort_by(.Name)| .[] | select(.Platform != "windows") | .Name + ": " + .ImageId' \
| egrep 'amzn-ami-hvm|ecs-optimized|amzn2-ami-hvm|amzn2-ami-ecs'
@doi-t
doi-t / generate_cloudwatch_metric_url.sh
Created September 11, 2018 07:33
View a stacked graph of an EC2 Per-Instance metric for EC2 instances that has prefix you specified.
# As of 2018/09/11
# METEIC can be either NetworkPacketsOut, NetworkPacketsIn, DiskReadOps, StatusCheckFailed, StatusCheckFailed_Instance, NetworkOut, NetworkIn, DiskWriteBytes, StatusCheckFailed_System, CPUUtilization, DiskWriteOps or DiskReadBytes
METRIC='NetworkIn'; REGION='ap-northeast-1'; NAME_TAG_VALUE='your_instance_name_prefix*'; echo "https://${REGION}.console.aws.amazon.com/cloudwatch/home?region=${REGION}#metricsV2:graph=~(view~'timeSeries~stacked~false~metrics~(`aws ec2 describe-instances --filter Name=tag:Name,Values="${NAME_TAG_VALUE}" | jq '.[][].Instances[].InstanceId' --raw-output | sed "s/^/~(~'AWS*2fEC2~'${METRIC}~'InstanceId~'/g" | sed "s/$/~(stat~'Sum))/g" | tr -d '\n'`)~view~'timeSeries~stacked~true~region~'${REGION})"
@doi-t
doi-t / README.md
Created June 1, 2018 11:52
Full Example of Terratest

Full Example of Terratest

Usage

$ export VPC_ID=<VPC ID> \
export SUBNET_ID=<subnet id> \
export PEM_KEY_NAME=<pem key name> \
export YOUR_NAME=<your name for resource tag>; \
go test -v
@doi-t
doi-t / aws_lambda_cloudwatch_logs_exporter.md
Last active February 10, 2022 23:46
Export logs from CloudWatch Logs to S3 with AWS Lambda

event example

TASK_NAME=${1:-'export_logs'}
FROM=${2:-'2017-12-24 12:00'}
TO=${3:-'2017-12-24 12:05'}
cat <<-EOF > event.json
{
    "taskName": "$TASK_NAME",
    "fromTime": `gdate --date="$FROM" +%s%3N`,
 "toTime": `gdate --date="$TO" +%s%3N`
@doi-t
doi-t / getting-start-with-AWS-Batch.md
Last active January 2, 2023 09:50
A log of getting start with AWS Batch.
@doi-t
doi-t / get_cloudwatch_logs_records_from_kinesis_stream.py
Last active September 25, 2019 21:17
CloudWatch Logs --> subscription filter --> Kinesis stream --> boto3.client('kinesis').get_records()
import base64
import datetime
import gzip
import json
import sys
import time
from typing import Any, Dict, List
import boto3
@doi-t
doi-t / example_envattrs.py
Last active May 28, 2019 03:05
Simple example of envattrs for aws lambda function
# echo '{}' > event.json; mkdir -p libs; pip install attrs envattrs -t libs; \
# export YOUR_PREFIX_SLACK_CHANNEL=your_project-staging; \
# export YOUR_PREFIX_S3_BUCKET=pyconapac-bucket; \
# export YOUR_PREFIX_THRESHOLD=50; \
# python-lambda-local -l libs/ -f handler -t 5 lambda_pyconapac.py event.json
from typing import Dict
import attr
import envattrs
@doi-t
doi-t / mk_bulk_insert.bash
Created July 16, 2014 06:07
単純なinsertの繰り返しと、バルクインサートのDDLを生成するスクリプト
#!/bin/bash
if [ $# -ne 2 ]; then
echo "$0 [the number of insert] [output file]"
exit 1
fi
nmemb_insert=$1
ddlfile=$2
cat <<- EOF > $ddlfile
@doi-t
doi-t / test_double_dig.c
Created June 21, 2014 07:19
0.1 + 0.1 + 0.1 == 0.3 ?
#include <stdio.h>
int
main(void)
{
double answer = 0.1 + 0.1 + 0.1;
if(answer == 0.3){
printf("True:%.100f\n", answer);
} else {
@doi-t
doi-t / awk_rand.bash
Created June 18, 2014 03:21
awkで乱数生成
#!/bin/bash
echo "---> sleep 0"
for loop in `seq 10`; do awk 'BEGIN{ srand(); print rand() }'; done
echo "---> sleep 0.5"
for loop in `seq 10`; do awk 'BEGIN{ srand(); print rand() }'; sleep 0.5; done
echo "---> sleep 1.0"
for loop in `seq 10`; do awk 'BEGIN{ srand(); print rand() }'; sleep 1.0; done
echo "---> srand(\$RANDOM)"
for loop in `seq 10`; do awk 'BEGIN{ srand('"$RANDOM"'); print rand() }'; done