Skip to content

Instantly share code, notes, and snippets.

View bhavika's full-sized avatar

Bhavika Tekwani bhavika

View GitHub Profile
@kevinzakka
kevinzakka / data_loader.py
Last active April 19, 2024 23:42
Train, Validation and Test Split for torchvision Datasets
"""
Create train, valid, test iterators for CIFAR-10 [1].
Easily extended to MNIST, CIFAR-100 and Imagenet.
[1]: https://discuss.pytorch.org/t/feedback-on-pytorch-for-kaggle-competitions/2252/4
"""
import torch
import numpy as np
@alsrgv
alsrgv / horovod_model_parallelism.py
Created January 27, 2018 06:20
Model parallelism in Horovod
# Copyright 2018 Uber Technologies, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
@sgillies
sgillies / cfrw.py
Last active April 26, 2023 08:37
Rasterio concurrency example
"""Concurrent read-process-write example"""
import concurrent.futures
from itertools import islice
from time import sleep
import rasterio
CHUNK = 100