Skip to content

Instantly share code, notes, and snippets.

:octocat:
doing open source

Valentin Haenel esc

View GitHub Profile
@esc
esc / gulpio_results.txt
Created Sep 8, 2017
GulpIO Blogpost GulpIO Loader results
View gulpio_results.txt
(torch) root@7396071ace7e:/home/rgoyal/GulpIO-benchmarks# CUDA_VISIBLE_DEVICES=0,1 python train_gulp.py --config configs/config_gulpio.json
=> active GPUs: 0,1
=> Output folder for this run -- jester_conv_example
> Using 10 processes for data loader.
> Training is getting started...
> Training takes 999999 epochs.
> Current LR : 0.001
Epoch: [0][0/1852] Time 12.410 (12.410) Data 9.079 (9.079) Loss 3.2856 (3.2856) Prec@1 9.375 (9.375) Prec@5 17.188 (17.188)
Epoch: [0][100/1852] Time 0.457 (0.854) Data 0.000 (0.429) Loss 2.5770 (3.0944) Prec@1 26.562 (12.655) Prec@5 68.750 (37.036)
Epoch: [0][200/1852] Time 0.456 (0.818) Data 0.000 (0.432) Loss 2.6250 (2.8789) Prec@1 25.000 (18.043) Prec@5 56.250 (47.528)
@esc
esc / jpeg_results.txt
Created Sep 8, 2017
GulpIO Blogpost JPEG Loader results
View jpeg_results.txt
(torch) root@7396071ace7e:/home/rgoyal/GulpIO-benchmarks# CUDA_VISIBLE_DEVICES=0,1 python train_jpeg.py --config configs/config_jpeg.json -g 0,1
=> active GPUs: 0,1
=> Output folder for this run -- jester_conv_example
> Using 10 processes for data loader.
> Training is getting started...
> Training takes 999999 epochs.
> Current LR : 0.001
Epoch: [0][0/1852] Time 77.101 (77.101) Data 73.654 (73.654) Loss 3.3249 (3.3249) Prec@1 0.000 (0.000) Prec@5 21.875 (21.875)
Epoch: [0][100/1852] Time 36.009 (7.519) Data 35.798 (7.183) Loss 2.6882 (3.0521) Prec@1 21.875 (13.784) Prec@5 54.688 (40.161)
Epoch: [0][200/1852] Time 13.762 (6.756) Data 13.562 (6.433) Loss 2.5396 (2.8169) Prec@1 20.312 (19.652) Prec@5 62.500 (50.451)
@esc
esc / gulpio_ingestor_example.py
Last active Sep 8, 2017
GulpIO Blogpost Ingestor Example
View gulpio_ingestor_example.py
adapter = Custom20BNCsvJpegAdapter(input_csv, jpeg_path, output_folder,
shuffle=shuffle,
frame_size=img_size,
shm_dir_path=shm_dir
)
ingestor = GulpIngestor(adapter, output_folder, videos_per_chunk,
num_workers=num_workers)
ingestor()
@esc
esc / gulpio_adapter_example.py
Last active Sep 8, 2017
GulpIO Blogpost Adapter
View gulpio_adapter_example.py
class AbstractDatasetAdapter(ABC):
""" Base class adapter for gulping (video) datasets.
Inherit from this class and implement the `iter_data` method. This method
should iterate over your entire dataset and for each element return a
dictionary with the following fields:
id : a unique(?) ID for the element.
frames : a list of frames (PIL images, numpy arrays..)
meta : a dictionary with arbitrary metadata (labels, start_time...)
For examples, see the custom adapters below.
"""
@esc
esc / gulpio_dataloader_example.py
Created Sep 8, 2017
GulpIO Blogpost Dataloader Example
View gulpio_dataloader_example.py
from gulpio.dataset import GulpImageDataset
from gulpio.loader import DataLoader
from gulpio.transforms import Scale, CenterCrop, Compose, UnitNorm
# define data augmentations. Notice that there are different functions for videos and images
transforms = Compose([
Scale(28), # resize image by the shortest edge
CenterCrop(28),
UnitNorm(), # instance wise mean and std norm
])
@esc
esc / gulpio_reading_example.py
Last active Sep 8, 2017
GulpIO Blogpost Reading Example
View gulpio_reading_example.py
# import the main interface for reading
from gulpio import GulpDirectory
# instantiate the GulpDirectory
gulp_directory = GulpDirectory('/tmp/something_something_gulps')
# iterate over all chunks
for chunk in gulp_directory:
# for each 'video' get the metadata and all frames
for frames, meta in chunk:
# do something with the metadata
for i, f in enumerate(frames):
@esc
esc / expose_moto_bug.py
Last active Sep 29, 2016
Expose a bug in moto EC2 tag filtering
View expose_moto_bug.py
import boto3
import moto
@moto.mock_ec2
def expose():
ec2 = boto3.resource('ec2')
blue, green = ec2.create_instances(
ImageId='ANY_ID', MinCount=2, MaxCount=2)
ec2.create_tags(Resources=[blue.instance_id],
@esc
esc / gist:1f7097d6c33b037b6911
Created Apr 1, 2015
hashing with git and python
View gist:1f7097d6c33b037b6911
zsh» git init example
Initialized empty Git repository in /tmp/example/.git/
zsh» cd example
zsh» cp ../objektmodell-beispiel/hello.py .
zsh» cat hello.py
#! /usr/bin/env python
""" Hello World! """
print 'Hello World!'
@esc
esc / test_lz4_failure.py
Created Oct 24, 2014
Testing LZ4 failure with blosc
View test_lz4_failure.py
import blosc
data = [
0x93, 0xb0, 0x49, 0xaf, 0x62, 0xad, 0xe3, 0xaa, 0xe4, 0xa5, 0x43, 0x20,
0x24, 0x29, 0xc9, 0xaf, 0xee, 0xad, 0x0b, 0xac, 0x3d, 0xa8, 0x1f, 0x99,
0x53, 0x27, 0xb6, 0x2b, 0x16, 0xb0, 0x5f, 0xae, 0x89, 0xac, 0x51, 0xa9,
0xfc, 0xa1, 0xc9, 0x24, 0x59, 0x2a, 0x2f, 0x2d, 0xb4, 0xae, 0xeb, 0xac,
0x2f, 0xaa, 0xec, 0xa4, 0x53, 0x21, 0x31, 0x29, 0x8f, 0x2c, 0x8e, 0x2e,
0x31, 0xad, 0xd6, 0xaa, 0x6d, 0xa6, 0xad, 0x1b, 0x3e, 0x28, 0x0a, 0x2c,
0xfd, 0x2d, 0xf8, 0x2f, 0x45, 0xab, 0x81, 0xa7, 0x1f, 0x95, 0x02, 0x27,
View blaze_quickstart.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
You can’t perform that action at this time.