Skip to content

Instantly share code, notes, and snippets.

View mlconnor's full-sized avatar

Michael Connor mlconnor

11 followers · 4 following
View GitHub Profile
@mlconnor
mlconnor / country_date_formats.csv
Created February 22, 2012 20:49
Listing of countries with their preferred date formats, ISO3166 code, ISO629-2
ISO 3166 Country Code ISO639-2 Country Code Country ISO 3166 Country Code ISO639-2 Lang Language Date Format
ALB AL Albania sqi sq Albanian yyyy-MM-dd
ARE AE United Arab Emirates ara ar Arabic dd/MM/yyyy
ARG AR Argentina spa es Spanish dd/MM/yyyy
AUS AU Australia eng en English d/MM/yyyy
AUT AT Austria deu de German dd.MM.yyyy
BEL BE Belgium fra fr French d/MM/yyyy
BEL BE Belgium nld nl Dutch d/MM/yyyy
BGR BG Bulgaria bul bg Bulgarian yyyy-M-d
BHR BH Bahrain ara ar Arabic dd/MM/yyyy
@mlconnor
mlconnor / stablediffusion_on_ec2.yaml
Last active December 14, 2023 02:07
Cloudformation Template for Stable Diffusion
# aws cloudformation create-stack --stack-name sd-webui-stack --template-body file://vpc-sf.yaml --parameters ParameterKey=KeyPair,ParameterValue=StableDiffusion
# ssh -i StableDiffusion.pem ubuntu@ip
# https://towardsdatascience.com/create-your-own-stable-diffusion-ui-on-aws-in-minutes-35480dfcde6a
# ssh -N -L 7860:127.0.0.1:7860 ubuntu@IP_ADDR
# cd /home/ubuntu/stable-diffusion-webui && nohup /home/ubuntu/stable-diffusion-webui/webui.sh --listen --theme dark --enable-insecure-extension-access --gradio-auth UNAME:PWD &
# systemctl - https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/6049
# sdxl - a1111 https://github.com/FurkanGozukara/Stable-Diffusion/blob/main/Tutorials/How-To-Use-Stable-Diffusion-SDXL-Locally-And-Also-In-Google-Colab.md
AWSTemplateFormatVersion: "2010-09-09"
Description: StableDiffusion server
Parameters:
@mlconnor
mlconnor / tomcat_memory.md
Created September 17, 2014 13:57
Memory settings for Java JVM in Amazon Elastic Beanstalk

Elastic Beanstalk Java JVM Settings

Determining the right memory settings (MX & MS) for the JVM is not trivial. If you set memory too low then your machine will trash as it runs out and eventually crash. If you set it too high then other critical processes such as Apache or the OS itself may become memory starved and also cause crashes.

In general, I think it best to set the initial memory setting (MS) to be small, around 200M. The real variable we need to calculate is the limit we will place on JVM memory (MS).

In order to make this determination, we need to calculate a few things such as the memory that Apache and the Linux OS need to operate efficiently.

Apache Memory Needs

@mlconnor
mlconnor / frontmatter.js
Created January 23, 2015 20:29
A simple front matter (YAML) with Handlebars template. Pass the name of the file into the program.
yaml = require('js-yaml');
fs = require('fs');
Handlebars = require('hbs');
Handlebars.registerHelper("sep", function(options){
if(options.data.last) {
return options.inverse();
} else {
return options.fn();
}
@mlconnor
mlconnor / tlds.php
Last active August 2, 2023 12:49
PHP list of top level domains and a TLD domain checker for URL parsing
/**
* Returns true or false
* indicating whether or not the URL
* passes a TLD check.
*/
function check_tld($url) {
$parsed_url = parse_url($url);
if ( $parsed_url === FALSE ) return false;
return preg_match('/\.(aero|asia|biz|cat|com|coop|info|int|jobs|mobi|museum|name|net|org|post|pro|tel|travel|mlcee|xxx|ac|ad|ae|af|ag|ai|al|am|an|ao|aq|ar|as|at|au|aw|ax|az|ba|bb|bd|be|bf|bg|bh|bi|bj|bl|bm|bn|bo|bq|br|bs|bt|bv|bw|by|bz|ca|cc|cd|cf|cg|ch|ci|ck|cl|cm|cn|co|cr|cu|cv|cw|cx|cy|cz|de|dj|dk|dm|do|dz|ec|ee|eg|eh|er|es|et|eu|fi|fj|fk|fm|fo|fr|ga|gb|gd|ge|gf|gg|gh|gi|gl|gm|gn|gp|gq|gr|gs|gt|gu|gw|gy|hk|hm|hn|hr|ht|hu|id|ie|il|im|in|io|iq|ir|is|it|je|jm|jo|jp|ke|kg|kh|ki|km|kn|kp|kr|kw|ky|kz|la|lb|lc|li|lk|lr|ls|lt|lu|lv|ly|ma|mc|md|me|mf|mg|mh|mk|ml|mm|mn|mo|mp|mq|mr|ms|mt|mu|mv|mw|mx|my|mz|na|nc|ne|nf|ng|ni|nl|no|np|nr|nu|nz|om|pa|pe|pf|pg|ph|pk|pl|pm|pn|pr|ps|pt|pw|py|qa|re|ro|rs|ru|rw|sa|sb|sc|sd|se|sg|sh|si|sj|sk|sl|sm|sn|so|sr|st|su|sv|sx|sy|sz|tc|td|tf|tg|th|tj|tk|tl|tm|tn|to|tp|tr|tt|tv|tw|tz|ua|ug|
@mlconnor
mlconnor / flan-t5-xxl-sharded-fp16.ipynb
Last active February 27, 2023 19:17
This is a Python Jupyter notebook built for Amazon SageMaker notebooks. To launch, go to AWS Console | SageMaker | Notebooks | Create Notebook Instance (ml.g5.2xlarge, config volume size - 500G) | Open Jupyter Lab | Upload. Based on https://www.philschmid.de/fine-tune-flan-t5
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@mlconnor
mlconnor / walk_array.php
Created June 25, 2012 17:19
php function to walk an array/object recursively
/**
* works for json objects. will replace all
* keys and values with the result of the
* closure.
*/
function walk_recursive($obj, $closure) {
if ( is_object($obj) ) {
$newObj = new stdClass();
foreach ($obj as $property => $value) {
$newProperty = $closure($property);
@mlconnor
mlconnor / aws_iam_permissions.csv
Last active November 11, 2022 21:38
CSV file containing all AWS service names and permissions
Service Permission
Auto Scaling CreateAutoScalingGroup
Auto Scaling CreateLaunchConfiguration
Auto Scaling CreateOrUpdateScalingTrigger
Auto Scaling CreateOrUpdateTags
Auto Scaling DeleteAutoScalingGroup
Auto Scaling DeleteLaunchConfiguration
Auto Scaling DeleteNotificationConfiguration
Auto Scaling DeletePolicy
Auto Scaling DeleteScheduledAction
@mlconnor
mlconnor / elastic_beanstalk_external_sessions.md
Created October 17, 2014 20:26
Analaysis and recommendation for externalizing session in Elastic Beanstalk using Tomcat and memcached.

Session Management in an Autoscaling Environment

Problem Statement

User sessions in J2EE and LAMP stacks have traditionally been handled in memory by the application server handling the user request. Because of that, load balancers have been configured to use sticky sessions. By sticky sessions we mean that once the user has visited the site, they will be assigned an app server and will return to that server for subsequent requests. The load balancers typically handle that by referencing the users session cookie.

Elastic cloud environments differ from traditional server configurations in that they have a variable number of servers based on traffic loads whereas traditional configurations had a fixed number of servers. When traffic volumes decline it is necessary to vaporize servers. In doing so, we would lose user sessions (essentially forcing a logout) unless we come up with a new strategy for session management.

A new approach

After much research, it is clear that the best

@mlconnor
mlconnor / census_data.csv
Last active February 15, 2022 21:02
US Census Data - https://aws.amazon.com/blogs/machine-learning/analyze-us-census-data-for-population-segmentation-using-amazon-sagemaker/
We can't make this file beautiful and searchable because it's too large.
CensusId,State,County,TotalPop,Men,Women,Hispanic,White,Black,Native,Asian,Pacific,Citizen,Income,IncomeErr,IncomePerCap,IncomePerCapErr,Poverty,ChildPoverty,Professional,Service,Office,Construction,Production,Drive,Carpool,Transit,Walk,OtherTransp,WorkAtHome,MeanCommute,Employed,PrivateWork,PublicWork,SelfEmployed,FamilyWork,Unemployment
1001,Alabama,Autauga,55221,26745,28476,2.6,75.8,18.5,0.4,1.0,0.0,40725,51281.0,2391.0,24974,1080,12.9,18.6,33.2,17.0,24.2,8.6,17.1,87.5,8.8,0.1,0.5,1.3,1.8,26.5,23986,73.6,20.9,5.5,0.0,7.6
1003,Alabama,Baldwin,195121,95314,99807,4.5,83.1,9.5,0.6,0.7,0.0,147695,50254.0,1263.0,27317,711,13.4,19.2,33.1,17.7,27.1,10.8,11.2,84.7,8.8,0.1,1.0,1.4,3.9,26.4,85953,81.5,12.3,5.8,0.4,7.5
1005,Alabama,Barbour,26932,14497,12435,4.6,46.2,46.7,0.2,0.4,0.0,20714,32964.0,2973.0,16824,798,26.7,45.3,26.8,16.1,23.1,10.8,23.1,83.8,10.9,0.4,1.8,1.5,1.6,24.1,8597,71.8,20.8,7.3,0.1,17.6
1007,Alabama,Bibb,22604,12073,10531,2.2,74.5,21.4,0.4,0.1,0.0,17495,38678.0,3995.0,18431,1618,16.8,27.9,21.5,17.9,