Skip to content

Instantly share code, notes, and snippets.

View dannguyen's full-sized avatar
💭
havin a normal one

Dan Nguyen dannguyen

💭
havin a normal one
View GitHub Profile
@dannguyen
dannguyen / bq-sfpd-query.sql
Created July 26, 2022 16:00
Example of querying BigQuery's public dataset of SFPD crime incidents
SELECT
unique_key
, pddistrict AS pd_district
, DATE(timestamp) AS incident_date
, category
, descript AS description
, dayofweek AS day_of_week
, resolution
, UPPER(address) AS address
, longitude
@dannguyen
dannguyen / fetch_ghstars.md
Last active January 2, 2024 07:24
fetch_ghstars.py: quick CLI script to fetch from Github API all of a user's starred repos and save it as raw JSON and wrangled CSV

fetch_ghstars.py: quick CLI script to fetch and collate from Github API all of a user's starred repos

  • Requires Python 3.6+
  • Creates a subdir 'ghstars-USERNAME' at the current working directory
  • the raw JSON of each page request is saved as: 01.json, 02.json 0n.json
  • A flattened, filtered CSV is also created: wrangled.csv

Example usage:

@dannguyen
dannguyen / aws-transcribe-2020-10-biden-palin.md
Last active February 10, 2021 01:29
i only created this gist to respond to someone responding to my older aws-transcribe-via-cli gist

Amazon Transcribe (real-time) streaming sample, with speakers identified (2020-10-09)

Note: This gist refers this older gist that shows the AWS transcribe API: https://gist.github.com/dannguyen/9b8c51f5bb853209f19f1a0f18f0f74c

I went into the AWS console for Transcription, which has an interface for real-time transcription here: https://console.aws.amazon.com/transcribe/home?region=us-east-1#realTimeTranscription

Then I used my phone to play out this snippet of the 2008 VP presidential debate, featuring speech from Biden and Palin: https://twitter.com/dancow/status/1313951588428517385

fieldname value
act 1
scene 5
speaker Horatio
lines Propose the oath, my lord.
~~~~~~~~~
act 1
scene 5
speaker Hamlet
@dannguyen
dannguyen / README-xsv-split-windows.md
Last active August 27, 2020 07:00
How to install and use xsv to split a large CSV file (Windows)

How to use xsv (in Windows) to split up a CSV file too big for Excel

I wrote these instructions on how to install and use xsv – a powerful CSV-handling command-line tool, because someone asked how to deal with a data file that was too big to open in Excel or even Notepad. I didn't know how familiar the person was with installing/running downloadable .exe files or with Powershell, so I've tried to include some general instructions that hopefully are useful to even novices.

This mini-guide is not at all meant to be exhaustive as it basically shows just one of xsv's many useful functions. But if you're new to the idea of using command-line tools to do things, hopefully this can be a friendly intro to it.


Here's an example of a CSV that, at 3 million rows, is too big for Excel to open: https://burntsushi.net/stuff/worldcitiespop.csv

@dannguyen
dannguyen / bash-prompt.md
Last active August 19, 2020 00:05
my bash prompt with a ghost and stuff

this goes in my bash profile:

XRESET='\[\033[00m\]'
PROMPT_PATH="\[\033[0;33m\]\W${XRESET} \[\033[1;37m\]\$${XRESET}"
PROMPT_GHOST="༼ つ\[\033[1;33m\]°${XRESET}\[\033[1;31m\]︻\[\033[1;33m\]゜${XRESET}༽つ🐕"

export PS1="${PROMPT_GHOST} ${PROMPT_PATH} "
@dannguyen
dannguyen / normalize-ascii-google-sheet-README.md
Last active August 25, 2020 22:17
A modified Google App Script hack to normalize Vietnamese characters into ASCII
@dannguyen
dannguyen / DANS SECRET STUFF.md
Created July 16, 2020 22:16
DANS SECRET STUFF

test test test

@dannguyen
dannguyen / autotimestamp-column-in-google-sheets.md
Last active April 30, 2021 17:07
a Google Sheets script that automatically adds a timestamp to a corresponding row when the first column is created

How to automatically timestamp a column when a row in Google Sheets is created

Couldn't be bothered to look up the latest workout tracking app, so decided to stick to Google Sheets for now. One problem with this approach is that while it is easy on desktop GSheets to fill out a current timestamp (Command-Option-Shift-semicolon in MacOS), no such shortcut is available AFAIK in the iOS version of Google Sheets.

image

So I wrote a little custom function in Google Apps Script to do the following:

In an active Google Sheets spreadsheet, given a tab/sheet named myworkouts with a header named datetime, the following script updates a row's corresponding "datetime" column when a value is entered into a first column cell:

@dannguyen
dannguyen / nyt-local-news-sites.json
Created May 5, 2020 18:24
JSON extracted from the NYT's "Local journalism helps tell the full story" interactive https://www.nytimes.com/interactive/2020/us/support-local-journalism.html
[
{
"id": 0,
"name": "Adirondack Explorer",
"homepageUrl": "https://www.adirondackexplorer.org",
"supportUrl": "https://www.adirondackexplorer.org/subscription-center/new-combined-digital-and-print-subscription",
"city": "Saranac Lake",
"state": 36,
"lat": "44.329496",
"long": "-74.1312662"