Skip to content

Instantly share code, notes, and snippets.

View eggie5's full-sized avatar
💭
Working on TensorFlow Ranking

Alex Egg eggie5

💭
Working on TensorFlow Ranking
View GitHub Profile

FWIW: I (@rondy) am not the creator of the content shared here, which is an excerpt from Edmond Lau's book. I simply copied and pasted it from another location and saved it as a personal note, before it gained popularity on news.ycombinator.com. Unfortunately, I cannot recall the exact origin of the original source, nor was I able to find the author's name, so I am can't provide the appropriate credits.


Effective Engineer - Notes

What's an Effective Engineer?

@mblondel
mblondel / letor_metrics.py
Last active April 24, 2024 19:43
Learning to rank metrics.
# (C) Mathieu Blondel, November 2013
# License: BSD 3 clause
import numpy as np
def ranking_precision_score(y_true, y_score, k=10):
"""Precision at rank k
Parameters
@ed-alertedh
ed-alertedh / validate_tfrecords.py
Last active April 16, 2024 18:57
Utility functions to check for corruption in tfrecord files
import tensorflow as tf
def validate_dataset(filenames, reader_opts=None):
"""
Attempt to iterate over every record in the supplied iterable of TFRecord filenames
:param filenames: iterable of filenames to read
:param reader_opts: (optional) tf.python_io.TFRecordOptions to use when constructing the record iterator
"""
i = 0
@jgillman
jgillman / restore.sh
Last active March 8, 2024 17:51
pg_restore a local db dump into Docker
# Assumes the database container is named 'db'
DOCKER_DB_NAME="$(docker-compose ps -q db)"
DB_HOSTNAME=db
DB_USER=postgres
LOCAL_DUMP_PATH="path/to/local.dump"
docker-compose up -d db
docker exec -i "${DOCKER_DB_NAME}" pg_restore -C --clean --no-acl --no-owner -U "${DB_USER}" -d "${DB_HOSTNAME}" < "${LOCAL_DUMP_PATH}"
docker-compose stop db
@omoindrot
omoindrot / tensorflow_finetune.py
Last active February 25, 2024 15:00
Example TensorFlow script for fine-tuning a VGG model (uses tf.contrib.data)
"""
Example TensorFlow script for finetuning a VGG model on your own data.
Uses tf.contrib.data module which is in release v1.2
Based on PyTorch example from Justin Johnson
(https://gist.github.com/jcjohnson/6e41e8512c17eae5da50aebef3378a4c)
Required packages: tensorflow (v1.2)
Download the weights trained on ImageNet for VGG:
```
wget http://download.tensorflow.org/models/vgg_16_2016_08_28.tar.gz
@epicserve
epicserve / redis_key_sizes.sh
Last active February 21, 2024 18:30
A simple script to print the size of all your Redis keys.
#!/usr/bin/env bash
# This script prints out all of your Redis keys and their size in a human readable format
# Copyright 2013 Brent O'Connor
# License: http://www.apache.org/licenses/LICENSE-2.0
human_size() {
awk -v sum="$1" ' BEGIN {hum[1024^3]="Gb"; hum[1024^2]="Mb"; hum[1024]="Kb"; for (x=1024^3; x>=1024; x/=1024) { if (sum>=x) { printf "%.2f %s\n",sum/x,hum[x]; break; } } if (sum<1024) print "1kb"; } '
}
@fchollet
fchollet / classifier_from_little_data_script_3.py
Last active September 13, 2023 03:34
Fine-tuning a Keras model. Updated to the Keras 2.0 API.
'''This script goes along the blog post
"Building powerful image classification models using very little data"
from blog.keras.io.
It uses data that can be downloaded at:
https://www.kaggle.com/c/dogs-vs-cats/data
In our setup, we:
- created a data/ folder
- created train/ and validation/ subfolders inside data/
- created cats/ and dogs/ subfolders inside train/ and validation/
- put the cat pictures index 0-999 in data/train/cats
@codeinthehole
codeinthehole / run.py
Created November 21, 2012 13:46
Sample Celery chain usage for processing pipeline
from celery import chain
from django.core.management.base import BaseCommand
from . import tasks
class Command(BaseCommand):
def handle(self, *args, **kwargs):
@wrburgess
wrburgess / gist:5528649
Last active November 24, 2022 15:29
Backup Heroku Postgres database and restore to local database

Grab new backup of database

Command: heroku pgbackups:capture --remote production

Response: >>> HEROKU_POSTGRESQL_COLOR_URL (DATABASE_URL) ----backup---> a712

Get url of backup download

Command: heroku pgbackups:url [db_key] --remote production

@lioutasb
lioutasb / mrr_metric.py
Created July 26, 2018 23:09
Tensorflow implementation of Mean Reciprocal Rank (mrr) metric compatible with tf.Estimator
import tensorflow as tf
def mrr_metric(labels, predictions, weights=None,
metrics_collections=None,
updates_collections=None,
name=None):
with tf.name_scope(name, 'mrr_metric', [predictions, labels, weights]) as scope: