BedirYilmaz

import json

li = ["Yet each man kills the thing he loves,",
"By each let this be heard,",
"Some do it with a bitter look,",
"Some with a flattering word,",
"The coward does it with a kiss,",
"The brave man with a sword!",
"Some kill their love when they are young,",
"And some when they are old;",

BedirYilmaz

import os
from glob import glob

path = "path/to/files"
ext = "*.file_extension"

list(glob(os.path.join(path, ext)))

BedirYilmaz

In most of deep learning projects, the training scripts always start with lines to load in data, which can easily take a handful minutes. Only after data ready can start testing my buggy code. It is so frustratingly often that I wait for ten minutes just to find I made a stupid typo, then I have to restart and wait for another ten minutes hoping no other typos are made.

In order to make my life easy, I devote lots of effort to reduce the overhead of I/O loading. Here I list some useful tricks I found and hope they also save you some time.

1. use Numpy Memmap to load array and say goodbye to HDF5.

   I used to relay on HDF5 to read/write data, especially when loading only sub-part of all data. Yet that was before I realized how fast and charming Numpy Memmapfile is. In short, Memmapfile does not load in the whole array at open, and only later "lazily" load in the parts that are required for real operations.

Sometimes I may want to copy the full array to memory at once, as it makes later operations

BedirYilmaz

https://docs.tibco.com/pub/mash-local/4.1.1/doc/html/docker/GUID-BD850566-5B79-4915-987E-430FC38DAAE4.html

How to Do a Clean Restart of a Docker Instance
If you are using Docker-Machine, make sure your are talking to the right one. Execute the command docker-machine ls to find which one is currently active. It is also recommended to always redo the command:
`eval "$(docker-machine env <docker machine name>)" `
Note: Deleting volumes will wipe out their data. Back up any data that you need before deleting a container.
Procedure
Stop the container(s) using the following command:
`docker-compose down`
Delete all containers using the following command:

BedirYilmaz

name: torch
channels:
  - pytorch
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - astroid=2.3.3=py38_0
  - attrs=19.3.0=py_0
  - backcall=0.1.0=py38_0
  - blas=1.0=mkl

BedirYilmaz

name: torchold
channels:
  - albumentations
  - pytorch
  - conda-forge
  - defaults
dependencies:
  - albumentations=0.4.0=py36_0
  - geos=3.7.2=he1b5a44_2
  - giflib=5.1.4=0

BedirYilmaz

 # https://stackoverflow.com/a/11250884/776348

import matplotlib.pyplot as plt

fig, ax = plt.subplots()

# We need to draw the canvas, otherwise the labels won't be positioned and 
# won't have values yet.
fig.canvas.draw()
labels = ['m','y','l','a','b','e','l','s']

BedirYilmaz

# https://superuser.com/questions/355891/move-all-files-from-subdirectories-to-current-directory
find . -type f -mindepth 2 -exec mv -i -- {} . \;

BedirYilmaz

# https://stackoverflow.com/a/11596144/776348

fuser -k 8888/tcp

BedirYilmaz

def create_dir(path):
  if not os.path.isdir(path):
      os.makedirs(path)