Skip to content

Instantly share code, notes, and snippets.

:octocat:
doing open source

Valentin Haenel esc

:octocat:
doing open source
Block or report user

Report or block esc

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View np vs. loop.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@esc
esc / gulpio_results.txt
Created Sep 8, 2017
GulpIO Blogpost GulpIO Loader results
View gulpio_results.txt
(torch) root@7396071ace7e:/home/rgoyal/GulpIO-benchmarks# CUDA_VISIBLE_DEVICES=0,1 python train_gulp.py --config configs/config_gulpio.json
=> active GPUs: 0,1
=> Output folder for this run -- jester_conv_example
> Using 10 processes for data loader.
> Training is getting started...
> Training takes 999999 epochs.
> Current LR : 0.001
Epoch: [0][0/1852] Time 12.410 (12.410) Data 9.079 (9.079) Loss 3.2856 (3.2856) Prec@1 9.375 (9.375) Prec@5 17.188 (17.188)
Epoch: [0][100/1852] Time 0.457 (0.854) Data 0.000 (0.429) Loss 2.5770 (3.0944) Prec@1 26.562 (12.655) Prec@5 68.750 (37.036)
Epoch: [0][200/1852] Time 0.456 (0.818) Data 0.000 (0.432) Loss 2.6250 (2.8789) Prec@1 25.000 (18.043) Prec@5 56.250 (47.528)
@esc
esc / jpeg_results.txt
Created Sep 8, 2017
GulpIO Blogpost JPEG Loader results
View jpeg_results.txt
(torch) root@7396071ace7e:/home/rgoyal/GulpIO-benchmarks# CUDA_VISIBLE_DEVICES=0,1 python train_jpeg.py --config configs/config_jpeg.json -g 0,1
=> active GPUs: 0,1
=> Output folder for this run -- jester_conv_example
> Using 10 processes for data loader.
> Training is getting started...
> Training takes 999999 epochs.
> Current LR : 0.001
Epoch: [0][0/1852] Time 77.101 (77.101) Data 73.654 (73.654) Loss 3.3249 (3.3249) Prec@1 0.000 (0.000) Prec@5 21.875 (21.875)
Epoch: [0][100/1852] Time 36.009 (7.519) Data 35.798 (7.183) Loss 2.6882 (3.0521) Prec@1 21.875 (13.784) Prec@5 54.688 (40.161)
Epoch: [0][200/1852] Time 13.762 (6.756) Data 13.562 (6.433) Loss 2.5396 (2.8169) Prec@1 20.312 (19.652) Prec@5 62.500 (50.451)
@esc
esc / gulpio_ingestor_example.py
Last active Sep 8, 2017
GulpIO Blogpost Ingestor Example
View gulpio_ingestor_example.py
adapter = Custom20BNCsvJpegAdapter(input_csv, jpeg_path, output_folder,
shuffle=shuffle,
frame_size=img_size,
shm_dir_path=shm_dir
)
ingestor = GulpIngestor(adapter, output_folder, videos_per_chunk,
num_workers=num_workers)
ingestor()
@esc
esc / gulpio_adapter_example.py
Last active Sep 8, 2017
GulpIO Blogpost Adapter
View gulpio_adapter_example.py
class AbstractDatasetAdapter(ABC):
""" Base class adapter for gulping (video) datasets.
Inherit from this class and implement the `iter_data` method. This method
should iterate over your entire dataset and for each element return a
dictionary with the following fields:
id : a unique(?) ID for the element.
frames : a list of frames (PIL images, numpy arrays..)
meta : a dictionary with arbitrary metadata (labels, start_time...)
For examples, see the custom adapters below.
"""
@esc
esc / gulpio_dataloader_example.py
Created Sep 8, 2017
GulpIO Blogpost Dataloader Example
View gulpio_dataloader_example.py
from gulpio.dataset import GulpImageDataset
from gulpio.loader import DataLoader
from gulpio.transforms import Scale, CenterCrop, Compose, UnitNorm
# define data augmentations. Notice that there are different functions for videos and images
transforms = Compose([
Scale(28), # resize image by the shortest edge
CenterCrop(28),
UnitNorm(), # instance wise mean and std norm
])
@esc
esc / gulpio_reading_example.py
Last active Sep 8, 2017
GulpIO Blogpost Reading Example
View gulpio_reading_example.py
# import the main interface for reading
from gulpio import GulpDirectory
# instantiate the GulpDirectory
gulp_directory = GulpDirectory('/tmp/something_something_gulps')
# iterate over all chunks
for chunk in gulp_directory:
# for each 'video' get the metadata and all frames
for frames, meta in chunk:
# do something with the metadata
for i, f in enumerate(frames):
@esc
esc / expose_moto_bug.py
Last active Sep 29, 2016
Expose a bug in moto EC2 tag filtering
View expose_moto_bug.py
import boto3
import moto
@moto.mock_ec2
def expose():
ec2 = boto3.resource('ec2')
blue, green = ec2.create_instances(
ImageId='ANY_ID', MinCount=2, MaxCount=2)
ec2.create_tags(Resources=[blue.instance_id],
@esc
esc / gist:1f7097d6c33b037b6911
Created Apr 1, 2015
hashing with git and python
View gist:1f7097d6c33b037b6911
zsh» git init example
Initialized empty Git repository in /tmp/example/.git/
zsh» cd example
zsh» cp ../objektmodell-beispiel/hello.py .
zsh» cat hello.py
#! /usr/bin/env python
""" Hello World! """
print 'Hello World!'
@esc
esc / test_lz4_failure.py
Created Oct 24, 2014
Testing LZ4 failure with blosc
View test_lz4_failure.py
import blosc
data = [
0x93, 0xb0, 0x49, 0xaf, 0x62, 0xad, 0xe3, 0xaa, 0xe4, 0xa5, 0x43, 0x20,
0x24, 0x29, 0xc9, 0xaf, 0xee, 0xad, 0x0b, 0xac, 0x3d, 0xa8, 0x1f, 0x99,
0x53, 0x27, 0xb6, 0x2b, 0x16, 0xb0, 0x5f, 0xae, 0x89, 0xac, 0x51, 0xa9,
0xfc, 0xa1, 0xc9, 0x24, 0x59, 0x2a, 0x2f, 0x2d, 0xb4, 0xae, 0xeb, 0xac,
0x2f, 0xaa, 0xec, 0xa4, 0x53, 0x21, 0x31, 0x29, 0x8f, 0x2c, 0x8e, 0x2e,
0x31, 0xad, 0xd6, 0xaa, 0x6d, 0xa6, 0xad, 0x1b, 0x3e, 0x28, 0x0a, 0x2c,
0xfd, 0x2d, 0xf8, 0x2f, 0x45, 0xab, 0x81, 0xa7, 0x1f, 0x95, 0x02, 0x27,
You can’t perform that action at this time.