Skip to content

Instantly share code, notes, and snippets.

View mapmeld's full-sized avatar

Nick Doiron mapmeld

  • Chicago, IL
View GitHub Profile
@mapmeld
mapmeld / llama2-langchain.py
Last active February 2, 2024 16:22
llama2-langchain
# this should run on a GPU CoLab notebook
# pip install langchain xformers transformers datasets bitsandbytes accelerate --quiet
# get access to the meta-llama models, accept license, and get a read token
hf_auth = '######'
from langchain.chains import ConversationChain
from langchain.llms import HuggingFacePipeline
from langchain.memory import ConversationSummaryBufferMemory
from langchain.prompts.prompt import PromptTemplate
import json
j = json.load(open('./census-reviewed.json', 'r'))
headers = None
total_vars = {
'P1_047N': 0,
'P1_063N': 0,
'P1_070N': 0,
'P2_072N': 0,
'Hisp': 0,
@mapmeld
mapmeld / split-multi.py
Created December 29, 2015 06:11
Split a GeoJSON MultiPolygon FeatureCollection into GeoJSON Polygons
# split-multi.py
# open source, MIT license
import json
js = open('multipolygon.geojson', 'r').read()
gj = json.loads(js)
output = { "type": "FeatureCollection", "features": [] }
@mapmeld
mapmeld / kanji.md
Created July 25, 2023 20:41
Chinese characters example

累令直漢刃

累令直漢刃

@mapmeld
mapmeld / OverEncrypt.md
Last active July 25, 2023 18:55
OverEncrypt - paranoid HTTPS

OverEncrypt

This is a guide that I wrote to improve the default security of my website https://fortran.io , which has a certificate from LetsEncrypt. I'm choosing to improve HTTPS security and transparency without consideration for legacy browser support.

WARNING: if you mess up settings, lose your certificates, or decide to no longer maintain HTTPS certs, these steps can and will make your domain inaccessible.

I would recommend these steps only if you have a specific need for information security, privacy, and trust with your users, and/or maintain a separate secure.example.com domain which won't mess up your main site. If you've been thinking about hosting a site on Tor, then this might be a good option, too.

The best resources that I've found for explaining these steps are https://https.cio.gov , https://certificate-transparency.org , and https://twitter.com/konklone

@mapmeld
mapmeld / chatgpt-on-gptnyc.md
Created March 12, 2023 02:21
ChatGPT on GPT-NYC type questions

Date: February 25, 2023

Questions in quotes

My comments in bold italics

Hi, I'm going to ask some questions about New York City as a new visitor, and you should respond as an expert resident.

Sure, I'm happy to help! What would you like to know about New York City?

@mapmeld
mapmeld / face_classifier.py
Created December 22, 2019 22:07
face_classifier.py
"""
# BASH dependencies
apt-get install python-opencv ffmpeg
pip install keras numpy shap matplotlib pillow
rm ./drive/My\ Drive/mlin/training/*/*.jpg
rm ./drive/My\ Drive/mlin/validation/*/*.jpg
"""
# native imports
@mapmeld
mapmeld / example.py
Created December 16, 2021 17:04
How to write an ML example
# All I'm looking for on an ML example:
# ! pip install name_of_library
from name_of_library import model, other_stuff
tdata = load_data_from_file() # not a built-in datasets source where I'd need to write python to add data
tdata.apply(changes) # whose dataset is so perfect we don't edit it
model.train(tdata, **explained_params)
@mapmeld
mapmeld / patching_models_bigsci_proposal.md
Last active December 14, 2021 03:11
Patching Models BigSci Proposal

Patching Models with New Words, People, and Events

May 6 - June 15, 2021

Scope

Once a large pre-trained language model is published, it is a snapshot of language when its corpus was collected. What are ways to update models to support new or newly-frequent terms (BIPOC), phrasing (social distancing), or people and events (Fyre Festival)? What are reliable, low-cost ways to test and benchmark these methods of updating?

Current status

@mapmeld
mapmeld / Vanguard-Sortfix.js
Last active December 1, 2021 16:43
Sort stocks by percent change or my holdings change
/*
Generally, don't run random JS in your browser console, especially on financial sites, but here we are
By default this sorts by Percent Change. If you uncomment the next line it sorts by myDelta (price x your shares)
Caveats:
- I'm not affiliated with Vanguard or any licensed financial advisor or tax preparer. I don't have a clue what's going on with your finances.
- The script assumes you did NOT trade today; it uses today's change and current shares
- Delta-sort does not handle penny stocks as well because the UI says 0.01 and we reverse-engineer from current balance
*/
let sortRule = 'pct';