Skip to content

Instantly share code, notes, and snippets.

View crabba's full-sized avatar

Andrew Crabb crabba

  • AWS
  • Baltimore, MD
View GitHub Profile
@crabba
crabba / multithread_01.cpp
Last active February 28, 2024 20:11
2402-pcluster-threads
#include <chrono>
#include <iostream>
#include <thread>
#include <unistd.h>
#include <vector>
void whoami(int i) {
int sleep_ms = rand() % 5000;
std::thread::id this_id = std::this_thread::get_id();
#! /bin/bash
# install-singularity-al2.sh
# Install Singularity (also Apptainer, your choice) runtime on ParallelCluster with Amazon Linux 2
# Assumes this script is installed on a shared home directory: uses srun to call nodes
set -o errexit # abort on nonzero exitstatus
# set -o nounset # abort on unbound variable
set -o pipefail # don't hide errors within pipes
#! /usr/bin/env bash
# Set up venv inside a wdl directory
mkdir wdl
cd wdl
python3 -m venv venv
source ./venv/bin/activate
# AL2 version of openssl causes error
# miniwdl-run urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'OpenSSL 1.0.2k-fips 26 Jan 2017'.
# Process to build R with MKL support on Amazon Linux 2
# Install R v4
sudo amazon-linux-extras install R4
wget https://mac.r-project.org/benchmarks/R-benchmark-25.R
which R # /usr/bin/R
R --version # 4.0.2
# Install Intel MKL library
# https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl-download.html?operatingsystem=linux&distributions=yumpackagemanager
@crabba
crabba / get_os.sh
Created August 26, 2023 00:26
Detect OS and set useful variables
#! /bin/bash
# Detect OS and set useful environment variables
# Sets:
# AWS_ACCOUNT_ID
# AWS_REGION
TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
curl http://169.254.169.254/latest/meta-data/profile -H "X-aws-ec2-metadata-token: $TOKEN"
@crabba
crabba / boto3_sample.py
Last active February 23, 2024 03:40
Basis for a boto3 script
#! /usr/bin/env python3
import argparse
import boto3
import logging
parser = argparse.ArgumentParser(description='Retrieve Omics test run results')
parser.add_argument('-p', '--profile', dest='profile', metavar='PROFILE', help='AWS profile to use')
parser.add_argument('-n', '--name', dest='name', metavar='NAME', help='Name of test (run prefix)')
parser.add_argument('-v', '--verbose', dest='verbose', action='count', default=0, help='Verbosity (up to 3x)')
@crabba
crabba / delete_iam_user.sh
Created September 28, 2022 23:59 — forked from hernandanielg/delete_iam_user.sh
Bash script to delete IAM users using AWS cli tool
#!/bin/bash
user=$1
echo "User: $user"
user_policies=$(aws iam list-user-policies --user-name $user --query 'PolicyNames[*]' --output text)
echo "Deleting user policies: $user_policies"
for policy in $user_policies ;
do
@crabba
crabba / s3-object-exists.py
Last active November 10, 2021 16:26
S3 test if an object exists
#! /usr/bin/env python3
import boto3
import sys
s3client = boto3.client('s3')
def s3_object_exists(bucket, key):
ret = False
try:
#!/bin/bash
# Specify the desired volume size in GiB as a command line argument. If not specified, default to 50 GiB.
SIZE=${1:-50}
# Get the ID of the environment host Amazon EC2 instance.
INSTANCEID=$(curl http://169.254.169.254/latest/meta-data/instance-id)
# Get the ID of the Amazon EBS volume associated with the instance.
VOLUMEID=$(aws ec2 describe-instances \
@crabba
crabba / datetime00.py
Last active February 22, 2020 23:03
Python Datetimes
from datetime import datetime, timezone, timedelta
# Using Boto 3 EC2
launch_time = ec2.launch_time
print("launch_time {} tz {}".format(launch_time, launch_time.tzinfo))
now = datetime.now()
print("now {} tz {}".format(now, now.tzinfo))
try:
diff = now - launch_time
except TypeError as err:
print("Error: {}".format(err))