Skip to content

Instantly share code, notes, and snippets.

@W4ngatang
W4ngatang / download_glue_data.py
Last active May 23, 2024 12:55
Script for downloading data of the GLUE benchmark (gluebenchmark.com)
''' Script for downloading all GLUE data.
Note: for legal reasons, we are unable to host MRPC.
You can either use the version hosted by the SentEval team, which is already tokenized,
or you can download the original data from (https://download.microsoft.com/download/D/4/6/D46FF87A-F6B9-4252-AA8B-3604ED519838/MSRParaphraseCorpus.msi) and extract the data from it manually.
For Windows users, you can run the .msi file. For Mac and Linux users, consider an external library such as 'cabextract' (see below for an example).
You should then rename and place specific files in a folder (see below for an example).
mkdir MRPC
cabextract MSRParaphraseCorpus.msi -d MRPC
@moodoki
moodoki / exportgraph.py
Last active February 14, 2023 05:58
Freeze and export Tensorflow graph from checkpoint files
import os, argparse
import tensorflow as tf
from tensorflow.python.framework import graph_util
dir = os.path.dirname(os.path.realpath(__file__))
def freeze_graph(model_folder, output_nodes='y_hat',
output_filename='frozen-graph.pb',
rename_outputs=None):
@mauri870
mauri870 / tf-serving-client.go
Last active April 2, 2022 08:43
Tensorflow Serving Go client for the inception model
// Tensorflow Serving Go client for the inception model
// go get github.com/golang/protobuf/ptypes/wrappers google.golang.org/grpc
//
// Compile the proto files:
//
// git clone https://github.com/tensorflow/serving.git
// git clone https://github.com/tensorflow/tensorflow.git
//
// mkdir -p vendor
@zombiezen
zombiezen / .profile
Last active November 15, 2023 07:34
zsh on Google Cloud Shell
# Copyright 2017 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
@WillieMaddox
WillieMaddox / k_means_anchor_points.py
Created April 13, 2017 13:15
create darknet anchor points using k-means. Uses fast numpy array ops: pascal ~= 1.0 s coco ~= 16 s
# -*- coding: utf-8 -*-
import numpy as np
import xml.etree.ElementTree as ET
from pycocotools.coco import COCO
def convert_coco_bbox(size, box):
dw = 1. / size[0]
dh = 1. / size[1]
x = box[0] + box[2] / 2.0
@alansouzati
alansouzati / drone_deployment.yml
Created February 11, 2017 00:05
Drone sample config for kubernetes
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: grommet-drone
namespace: drone
labels:
app: grommet-drone
annotations:
description: drone with interface
spec:
@yaroslavvb
yaroslavvb / smart_initialize.py
Created October 13, 2016 23:20
Better initialize_all_variables which respects variable dependencies and doesn't rerun initializers
# testing variable order init
import tensorflow as tf
def initialize_all_variables(sess=None):
"""Initializes all uninitialized variables in correct order. Initializers
are only run for uninitialized variables, so it's safe to run this multiple
times.
Args:
@yaroslavvb
yaroslavvb / local_distributed_benchmark.py
Last active September 16, 2021 10:26
Benchmark distributed tensorflow locally by adding vector of ones on worker2 to variable on worker1 as fast as possible
"""Benchmark tensorflow distributed by adding vector of ones on worker2
to variable on worker1 as fast as possible.
On 2014 macbook, TensorFlow 0.10 this shows
Local rate: 2175.28 MB per second
Distributed rate: 107.13 MB per second
"""
@morgangiraud
morgangiraud / nvidia-reinstall.sh
Last active December 11, 2020 15:48
Script to reinstall manually nvidia drivers,cuda 9.0 and cudnn 7.1 on Ubuntu 16.04
# Remove anything linked to nvidia
sudo apt-get remove --purge nvidia*
sudo apt-get autoremove
# Search for your driver
apt search nvidia
# Select one driver (the last one is a decent choice)
sudo apt install nvidia-370
import tensorflow as tf
from tensorflow.python.framework import ops
import numpy as np
# Define custom py_func which takes also a grad op as argument:
def py_func(func, inp, Tout, stateful=True, name=None, grad=None):
# Need to generate a unique name to avoid duplicates:
rnd_name = 'PyFuncGrad' + str(np.random.randint(0, 1E+8))