This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# Check if a file path was provided | |
if [ "$#" -ne 1 ]; then | |
echo "Usage: \$0 <path-to-python-file>" | |
exit 1 | |
fi | |
# The file path is the first argument to the script | |
FILE_PATH="$1" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Process lifecycle management controller, the main purpose of such controller will accept requests from the client, transform incoming batch requests to multiple parallel process inside, and execute those processes to initialize associated job and distributed to multiple backend server considering the workload even & request reliable, the controller will keep monitoring the job result, respond to client, handle the error exception and recycle the process resource. | |
Sample below only demonstrates the basic concept of how to manage the process lifecycle, it's not a complete solution, the real-world scenario will be more complex and need to consider more factors like the process resource limitation, the process priority, the process dependency, the process retry etc. it mainly focus on the process state transition and managament in parallel processing. | |
Below are Mermaid state diagram illustrating the transitions between these Linux process states: | |
stateDiagram-v2 | |
[*] --> Running | |
Running --> Interrup |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# disable the restart dialogue and install several packages | |
sudo sed -i "/#\$nrconf{restart} = 'i';/s/.*/\$nrconf{restart} = 'a';/" /etc/needrestart/needrestart.conf | |
sudo apt-get update | |
sudo apt install wget git python3 python3-venv build-essential net-tools awscli -y | |
# install CUDA (from https://developer.nvidia.com/cuda-downloads) | |
wget https://developer.download.nvidia.com/compute/cuda/12.0.0/local_installers/cuda_12.0.0_525.60.13_linux.run | |
sudo sh cuda_12.0.0_525.60.13_linux.run --silent | |
# install git-lfs |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# install python 3.9 in AWS EC2 | |
cd /opt | |
wget https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tgz | |
tar -xvf Python-3.9.16.tgz | |
cd Python-3.9.16 | |
./configure --enable-optimizations | |
make altinstall | |
python3.9 -V | |
python3.9 -m pip install --upgrade pip | |
python3.9 -m pip install --upgrade setuptools |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# -*- coding: utf-8 -*- | |
""" | |
Python daemon implementation from: | |
http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/ | |
""" | |
import atexit | |
import os | |
import sys | |
import time |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env bash | |
set -o errexit | |
set -o nounset | |
set -o pipefail | |
if [[ "${TRACE-0}" == "1" ]]; then | |
set -o xtrace | |
fi | |
if [[ "${1-}" =~ ^-*h(elp)?$ ]]; then |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import * as cdk from "aws-cdk-lib"; | |
import {CfnUserPoolUserToGroupAttachment, IUserPool} from "@aws-cdk/aws-cognito"; | |
import {AwsCustomResource, AwsCustomResourcePolicy, PhysicalResourceId} from "@aws-cdk/custom-resources"; | |
export class UserPoolUser extends cdk.Stack { | |
constructor(scope: cdk.App, id: string, props: { | |
userPool: IUserPool, | |
username: string, | |
password: string, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# script to auto generate ssl certs and upload to s3 bucket | |
import os | |
import boto3 | |
import logging | |
import subprocess | |
import sys | |
import time | |
from multiprocessing import Pool | |
# set logging to stdout |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from cmath import log | |
from godaddypy import Client, Account | |
import configparser | |
import os | |
import logging | |
import sys | |
logging.basicConfig( | |
level=logging.INFO, | |
format="%(asctime)s [%(levelname)s] %(message)s", | |
handlers=[ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# scan given domain with dig command | |
# | |
# Usage: | |
# ./dig_scan.py <domain> | |
# | |
# Example: | |
# ./dig_scan.py example.com | |
# | |
# Output: | |
# |
NewerOlder