Skip to content

Instantly share code, notes, and snippets.

View MichaelCurrie's full-sized avatar

Michael Currie MichaelCurrie

View GitHub Profile
@MichaelCurrie
MichaelCurrie / git-lfs-example.sh
Created March 13, 2018 11:02
Use git-lfs on existing repository
# Install git-lfs
# (From https://github.com/git-lfs/git-lfs/wiki/Installation)
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:git-core/ppa
sudo apt-get update
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
sudo apt-get install git-lfs
git lfs install
# Add git-lfs to an existing repo
@MichaelCurrie
MichaelCurrie / Dropzone with Python 3
Last active December 14, 2023 10:19
Drag-and-drop upload files, via JavaScript, to a simple Python 3 HTTP server
#Drag-and-drop upload files, via JavaScript, to a simple Python 3 HTTP server
@MichaelCurrie
MichaelCurrie / dataframe_fast.py
Last active September 7, 2023 23:09
Fast pandas DataFrame read/write to mariadb
#!/usr/bin/env python3
"""
Drop-in replacement for pandas.DataFrame methods for reading and writing to database:
pandas.DataFrame.to_sql
pandas.read_sql_table
For some reason the current (Jan 2023) implementation in Pandas
using sqlalchemy is really slow. These methods are ~300x faster.
@MichaelCurrie
MichaelCurrie / yield_to_lamb_chop.py
Last active December 14, 2022 10:19
Shari Lewis teaches `yield`
def the_song_lyrics():
while True:
yield "This is the song that doesn't end"
yield "Yes it goes on and on my friends"
yield "Some people started singing it, not knowing what it was"
yield "And they'll continue singing it forever, just because"
for lyric in the_song_lyrics():
print(lyric)
@MichaelCurrie
MichaelCurrie / autopep8 Travis-CI.md
Last active April 26, 2021 17:18
Automatically fix PEP-8 issues using Travis-CI

PEP-8 is a set of Python style recommendations. pep8 is a module that checks your .py file for violations. To make your Travis-CI build fail if you have any violations, you could add these lines to your .travis.yml:

before_install:
    - pip install pep8
    
script:
    # Run pep8 on all .py files in all subfolders
    # (I ignore "E402: module level import not at top of file"
    # because of use case sys.path.append('..'); import <module>)
@MichaelCurrie
MichaelCurrie / awss3connect.md
Last active March 25, 2021 12:05
Connecting your local machine to an S3 bucket

Connecting your local machine to an S3 bucket

Steps to get a file on your local machine uploaded to an AWS S3 bucket, and then downloaded and used in an AWS EC2 Ubuntu Instance. (In this case we do the reverse, but the process is similar)

STEP 1

Get security credentials to be able to use the AWS command line utility:

@MichaelCurrie
MichaelCurrie / backup_mediawiki
Last active October 12, 2020 05:29
How to back up a MediaWiki server hosted on a remote Debian machine
# A script showing how to back up a MediaWiki server hosted on a remote Debian machine
# I run each line manually myself
# SUMMARY: Basically there are five things you should extract from the mediawiki server:
# 1. LocalSettings.php: Get this from /var/www/mediawiki/LocalSettings.php
# 2. images.zip: A zip file of /var/www/mediawiki/images which contains all the image data
# 3. dbdump20201011.sql: A single .sql file which is a dump of every table in the /var/www/ MySQL mediawiki database,
# which you can create by running mysqldump
# 4. pedia_full_dump.xml: The content, by running php /var/www/mediawiki/maintenance/dumpBackup.php --full > pedia_full_dump.xml
# 5. mediawiki.zip: A zip file of /var/www/mediawiki entirely (just in case we have other media or config stuff we need)
We can make this file beautiful and searchable if this error is corrected: Unclosed quoted field in line 9.
Asset,Type,Asset Value,Debt,Net Value,Change vs 2015,Change,Stake,Latitude,Longitude,Notes
Trump Tower (New York City),Office and retail,471,100,371,-159,Down,1,40.768277,-73.981455,Opened 1983
1290 Avenue of the Americas (New York City),Office and retail,2310,950,408,-62,Down,0.3,40.768277,-73.981455,
Niketown (New York City),Office and retail,400,10,390,-52,Down,1,40.768277,-73.981455,Ground lease through 2079
40 Wall Street (New York City),Office and retail,501,156,345,-28,Down,1,40.768277,-73.981455,
Trump Park Avenue (New York City),Residential and retail,191,14.3,176.7,-27,Down,1,40.768277,-73.981455,"49,564 sq. ft. of condos; 27,467 sq. ft. of retail"
Trump Parc/Trump Parc East (New York City),Residential and retail,88,0,88,17,Up,1,40.768277,-73.981455,"11,750 sq. ft. of condos; 14,963 sq. feet of retail; 13,108 sq. ft. of garage"
"Trump International Hotel and Tower, Central Park West (New York City)","Hotel, condos and retail",38,0,38,21,Up,1,40.768277,-73.981455,
"Trump World Tower, 845 United Natio
@MichaelCurrie
MichaelCurrie / python_multiprocessing_example.py
Last active May 23, 2020 02:49
Python multiprocessing example
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
A basic example using multiple processes
We spawn 4 processes. Each process is made to draw N/4 random
integers from 1 to 10, for some fixed large N.
Then the processes adds up its list of numbers, passes it into a queue,
and the main program adds up the subtotals to get a grand total.
@MichaelCurrie
MichaelCurrie / ..setup_git-lfs.sh
Last active March 14, 2018 09:33
Set up Git LFS for a new repo
# Instructions for Windows developers working on the Fling Unity Visualization Tool
# 1. Install the latest version of Unity
# 2. (NOT NEEDED) Install "SmartMerge" for Unity by following https://docs.unity3d.com/Manual/SmartMerge.html
# 3. Install GitHub Desktop
# 4. Install git-lfs
# 5. Get a GitHub developer account
# 6. Get invitated and accept the invitation to the fling-asia GitHub organization
# 7. Clone the fling-unity repository to your computer via the GitHub desktop app
# 8. Open the program in Unity