Skip to content

Instantly share code, notes, and snippets.

View darrengarvey's full-sized avatar

Darren Garvey darrengarvey

View GitHub Profile
@darrengarvey
darrengarvey / tf_list_files_perf_test.py
Created April 3, 2018 11:21
Perf test for `tensorflow.data.Dataset.list_files()`
#!/usr/bin/env python
from __future__ import print_function
'''
Perf test for `tensorflow.data.Dataset.list_files()`.
At least on TF 1.7 all file names are loaded into memory before
training starts. This can be a bottleneck for slower disks, especially
with large datasets.