Skip to content

Instantly share code, notes, and snippets.

View bhavika's full-sized avatar

Bhavika Tekwani bhavika

View GitHub Profile
@sgillies
sgillies / cfrw.py
Last active April 26, 2023 08:37
Rasterio concurrency example
"""Concurrent read-process-write example"""
import concurrent.futures
from itertools import islice
from time import sleep
import rasterio
CHUNK = 100
@alsrgv
alsrgv / horovod_model_parallelism.py
Created January 27, 2018 06:20
Model parallelism in Horovod
# Copyright 2018 Uber Technologies, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
@kevinzakka
kevinzakka / data_loader.py
Last active April 19, 2024 23:42
Train, Validation and Test Split for torchvision Datasets
"""
Create train, valid, test iterators for CIFAR-10 [1].
Easily extended to MNIST, CIFAR-100 and Imagenet.
[1]: https://discuss.pytorch.org/t/feedback-on-pytorch-for-kaggle-competitions/2252/4
"""
import torch
import numpy as np
@kmurugulla
kmurugulla / s3-cp-speed-test.sh
Created May 2, 2017 17:38
To test upload speeds to S3 (uses AWS CLI)
#! /bin/bash
#Author : Kiran Murugulla
#Description : Script to create an asset and upload binary from local machine into an S3 bucket
usage="Usage: s3-cp-speed-test.sh bucketname filepath \n"
BUCKET=$1
FILE=$2
if [ ! $# -eq 2 ] ; then
echo -e "$usage"
exit 2
@panovr
panovr / finetune.py
Created March 2, 2017 23:04
Fine-tuning pre-trained models with PyTorch
import argparse
import os
import shutil
import time
import torch
import torch.nn as nn
import torch.nn.parallel
import torch.backends.cudnn as cudnn
import torch.optim
@varver
varver / redshift_connect.py
Created February 15, 2017 11:02
Python script to connect with Redshift on AWS with SCHEMA support.
############ REQUIREMENTS ####################
# sudo apt-get install python-pip
# sudo apt-get install libpq-dev
# sudo pip install psycopg2
# sudo pip install sqlalchemy
# sudo pip install sqlalchemy-redshift
##############################################
import sqlalchemy as sa
from sqlalchemy.orm import sessionmaker
@jaron
jaron / urban-sound-cnn-salamon.py
Last active July 12, 2019 15:07
A Keras/Tensorflow implementation of the 5-layer CNN described in Salamon and Bello's paper (https://arxiv.org/pdf/1608.04363.pdf). See http://aqibsaeed.github.io/2016-09-24-urban-sound-classification-part-2/ for a description on how to create the data this uses.
import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Convolution2D, MaxPooling2D
from keras.optimizers import SGD
from keras.regularizers import l2, activity_l2
from keras.utils import np_utils
from sklearn import metrics
# to run this code, you'll need to load the following data:
@criccomini
criccomini / test_dags.py
Created June 22, 2016 15:18
test_dags.py
import os
import unittest
from airflow.models import DagBag
class TestDags(unittest.TestCase):
"""
Generic tests that all DAGs in the repository should be able to pass.
"""
@danikin
danikin / tar_test.c
Last active December 18, 2020 07:28
Tarantool Quick Test
// Tarantool quick test
// Copyright, Dennis Anikin 2016
//
// Quick disclaimer:
//
// This test shows 500K-1000K transactions per second on one CPU core
// and 600K-1600K queries per second on one CPU core.
//
// Based on the $6.57 per-month-price for the AWS t2.micro instance we can afford the tremendous number of 630bln queries for just $1
//