Skip to content

Instantly share code, notes, and snippets.

Avatar

Mark Litwintschik marklit

View GitHub Profile
@marklit
marklit / cmd.sh
Last active Nov 16, 2017
World Map in ANSI
View cmd.sh
wget https://eol.jsc.nasa.gov/videos/crewearthobservationsvideos/World_map_blank_small.png
convert World_map_blank_small.png World_map_blank_small.jpg
jp2a --colors --term-fit -i World_map_blank_small.jpg | sed -r 's/[XM]/ /g'
View kebase.md

Keybase proof

I hereby claim:

  • I am marklit on github.
  • I am marklit (https://keybase.io/marklit) on keybase.
  • I have a public key ASARIRAX9pGFl6YOQvWpSrl-DusmWqCP_cFNr002Z29pEwo

To claim this, I am signing this object:

View kebase.md

Keybase proof

I hereby claim:

  • I am marklit on github.
  • I am marklit (https://keybase.io/marklit) on keybase.
  • I have a public key ASARIRAX9pGFl6YOQvWpSrl-DusmWqCP_cFNr002Z29pEwo

To claim this, I am signing this object:

@marklit
marklit / install.sh
Last active Feb 22, 2020
Bot Detection Script. Works with Apache and Nginx Log Files.
View install.sh
sudo apt-get update
sudo apt-get install \
python-dev \
python-pip \
python-virtualenv
virtualenv findbots
source findbots/bin/activate
curl -O http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz
@marklit
marklit / _run.sh
Created Feb 25, 2017
CommonCrawl / AWS EMR / MRJob
View _run.sh
git clone https://github.com/Smerity/cc-mrjob.git ~/cc-mrjob && cd ~/cc-mrjob
pip install -r requirements.txt
pip install awscli
aws configure
curl -O https://commoncrawl.s3.amazonaws.com/crawl-data/CC-MAIN-2017-04/wat.paths.gz
gunzip wat.paths.gz
python server_analysis.py \
-r emr \
--conf-path mrjob.conf \
View rethinkdb-failure-reasons-according-to-hn.md
- Inexplicable perversity of human nature.
- The clever machinations of MongoDB's marketing people.
- The AGPL license killed it.
- We spent too long development before monetizing.
- Bad performance.
- Numeric types limited to a 64-bit `float`.
- Great product, but didn't/couldn't translate to revenue.
- Bad business model.
- Failure in timezones/timestamp nuances.
@marklit
marklit / deep fizz buzz.md
Created Sep 24, 2016
Perfectly predicted Fizz buzz
View deep fizz buzz.md

See GitHub for the source code and TensorFlow on a GTX 1080 for installation notes.

$ python fizz_buzz.py \
    --hidden_units 500 \
    --iterations 4000 \
    --learning_rate 0.04
@marklit
marklit / sqlite3_insert.sh
Created May 18, 2016
Insert compressed CSV data into an SQLite3 table
View sqlite3_insert.sh
$ (echo -e "PRAGMA synchronous=OFF; PRAGMA count_changes=OFF;\n.mode csv\n.separator ,\n.import /dev/stdin trips";
gunzip -c ~/taxi-data/*.csv.gz) | \
sqlite3 trips.db
@marklit
marklit / es-dsl.md
Created May 11, 2016
Elasticsearch DSL queries generated by SQL plugin
View es-dsl.md

Query 1

SELECT cab_type,
       count(*)
FROM trips
GROUP BY cab_type
@marklit
marklit / mappings.txt
Created Apr 8, 2016
Taxi Data Mapping w/ ES-compatible data types
View mappings.txt
trip_id integer
vendor_id string
pickup_datetime TIMESTAMP
dropoff_datetime TIMESTAMP
store_and_fwd_flag string
rate_code_id integer
pickup_longitude float
pickup_latitude float
dropoff_longitude float
dropoff_latitude float
You can’t perform that action at this time.