Skip to content

Instantly share code, notes, and snippets.

@netj
netj / memusg
Last active January 29, 2024 15:04
memusg -- Measure memory usage of processes
#!/usr/bin/env bash
# memusg -- Measure memory usage of processes
# Usage: memusg COMMAND [ARGS]...
#
# Author: Jaeho Shin <netj@sparcs.org>
# Created: 2010-08-16
############################################################################
# Copyright 2010 Jaeho Shin. #
# #
# Licensed under the Apache License, Version 2.0 (the "License"); #
set-option -g prefix `
set -g base-index 1
# send `
bind-key a send-prefix
# start window index of 1 instead of 0
set-option -g base-index 1
# Start panes at 1 instead of 0. tmux 1.6 only
@chanks
chanks / gist:7585810
Last active February 29, 2024 03:50
Turning PostgreSQL into a queue serving 10,000 jobs per second

Turning PostgreSQL into a queue serving 10,000 jobs per second

RDBMS-based job queues have been criticized recently for being unable to handle heavy loads. And they deserve it, to some extent, because the queries used to safely lock a job have been pretty hairy. SELECT FOR UPDATE followed by an UPDATE works fine at first, but then you add more workers, and each is trying to SELECT FOR UPDATE the same row (and maybe throwing NOWAIT in there, then catching the errors and retrying), and things slow down.

On top of that, they have to actually update the row to mark it as locked, so the rest of your workers are sitting there waiting while one of them propagates its lock to disk (and the disks of however many servers you're replicating to). QueueClassic got some mileage out of the novel idea of randomly picking a row near the front of the queue to lock, but I can't still seem to get more than an an extra few hundred jobs per second out of it under heavy load.

So, many developers have started going straight t

@tsiege
tsiege / The Technical Interview Cheat Sheet.md
Last active May 9, 2024 13:54
This is my technical interview cheat sheet. Feel free to fork it or do whatever you want with it. PLEASE let me know if there are any errors or if anything crucial is missing. I will add more links soon.

ANNOUNCEMENT

I have moved this over to the Tech Interview Cheat Sheet Repo and has been expanded and even has code challenges you can run and practice against!






\

@wrobstory
wrobstory / .gitignore
Last active May 13, 2019 12:19
PyData2014
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
# C extensions
*.so
# Distribution / packaging
.Python
env/
@karpathy
karpathy / min-char-rnn.py
Last active May 10, 2024 18:13
Minimal character-level language model with a Vanilla Recurrent Neural Network, in Python/numpy
"""
Minimal character-level Vanilla RNN model. Written by Andrej Karpathy (@karpathy)
BSD License
"""
import numpy as np
# data I/O
data = open('input.txt', 'r').read() # should be simple plain text file
chars = list(set(data))
data_size, vocab_size = len(data), len(chars)
@gigamonkey
gigamonkey / criteria.txt
Last active January 5, 2020 06:21
Hiring criteria: looking for the ability to …
Write a program that does what it’s supposed to do
Write idiomatic code
Debug a program that you wrote
Debug a program someone else wrote
Debug the interaction between a system you wrote and one you didn’t
File a good bug report
Modify a program you didn’t write
Test a program you wrote
Test a program you didn’t write
Learn a new programming language
@audreyfeldroy
audreyfeldroy / pypi-release-checklist2.md
Last active March 9, 2020 19:26
My PyPI Release Checklist 2 (now with bumpversion)
  • Update HISTORY.rst
  • Commit the changes:
git add HISTORY.rst
git commit -m "Changelog for upcoming release 0.1.1."
  • Update version number (can also be patch or major)
bumpversion minor
@kastnerkyle
kastnerkyle / minibatch_ocr.py
Last active September 9, 2022 11:34
Minibatch OCR using modified CTC from Shawn Tan and Mohammad Pezeshki
"""
bitmap utils and much of the ctc code modified
From Shawn Tan, Rakesh and Mohammad Pezeshki
"""
# Author: Kyle Kastner
# License: BSD 3-clause
from theano import tensor
from scipy import linalg
import theano
import numpy as np

A Few Useful Things to Know about Machine Learning

The paper presents some key lessons and "folk wisdom" that machine learning researchers and practitioners have learnt from experience and which are hard to find in textbooks.

1. Learning = Representation + Evaluation + Optimization

All machine learning algorithms have three components:

  • Representation for a learner is the set if classifiers/functions that can be possibly learnt. This set is called hypothesis space. If a function is not in hypothesis space, it can not be learnt.
  • Evaluation function tells how good the machine learning model is.
  • Optimisation is the method to search for the most optimal learning model.