Masaki Samejima harusametime

## jupyter_error_jp.md

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                harusametime
                / jupyter_error_jp.md
            
            
              Last active
              November 6, 2017 08:02
            
              
                Jupyter notebookをubuntuで起動してエラーが出たとき
              
          
    $ jupyter notebook
...
OSError: [Errno 99] Cannot assign requested address

こういう風にいれましょう．
$ jupyter notebook --ip=127.0.0.1


## fujixerox_printer_ubuntu.md

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                harusametime
                / fujixerox_printer_ubuntu.md
            
            
              Created
              January 5, 2018 10:58
            
              
                How to install Fujixerox printer in ubuntu 16.04
              
          
    What

This is how to install Fujixerox C5000-d printer in ubuntu 16.04. Unfotunately, the official printer driver is too old to be compatibe with latest ubuntu.
Step


Download "fxlinuxprint_1.1.0+ds.orig.tar.xz" in Launchpad[https://launchpad.net/ubuntu/+source/fxlinuxprint/1.1.0+ds-2]
Extract the tar.xz file.

tar Jxfv fxlinuxprint_1.1.0+ds.orig.tar.xz


Search and run "Printer" from "Dash" that is the top icon in launcher


## test.ipynb

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                harusametime
                / test.ipynb
            
            
              Created
              November 21, 2018 23:28
            
          
      Sorry, something went wrong. Reload?
      Sorry, we cannot display this file.
      Sorry, this file is invalid so it cannot be displayed.
      
          Viewer requires iframe.
      
    
## tensorflow_script.ipynb

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                harusametime
                / tensorflow_script.ipynb
            
            
              Created
              December 1, 2018 09:11
            
          
      Sorry, something went wrong. Reload?
      Sorry, we cannot display this file.
      Sorry, this file is invalid so it cannot be displayed.
      
          Viewer requires iframe.
      
    
## mnist_script.py
import os
import argparse
import tensorflow as tf

# global variables
width = 28
height = 28
n_class = 10

# tfrecord parser

## error_load_npz.md

      
              1 file
            
          
              0 forks
            
          
              4 comments
            
          
              0 stars
            
          
                harusametime
                / error_load_npz.md
            
            
              Last active
              July 12, 2019 09:07
            
              
                npz保存時の注意
              
          
    Numpyで複数のndarrayを保存するとき，npzを使うのは便利だが
import numpy as np

X = np.zeros((100,100))
y = np.zeros(10)

file_path = /home/samejima/data.npz
np.savez(file_path, x = X, y=y) 

  
## sagemaker_handson.ipynb

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                harusametime
                / sagemaker_handson.ipynb
            
            
              Last active
              September 4, 2019 15:32
            
          
      Sorry, something went wrong. Reload?
      Sorry, we cannot display this file.
      Sorry, this file is invalid so it cannot be displayed.
      
          Viewer requires iframe.
      
    
## gluoncv_deeplab_hosting.ipynb

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                harusametime
                / gluoncv_deeplab_hosting.ipynb
            
            
              Last active
              January 13, 2021 15:46
            
          
      Sorry, something went wrong. Reload?
      Sorry, we cannot display this file.
      Sorry, this file is invalid so it cannot be displayed.
      
          Viewer requires iframe.
      
    
## redshiftML.py
import boto3

redshift = boto3.client('redshift')
credentials = redshift.get_cluster_credentials(
    DbUser='demouser',
    DbName='dev',
    ClusterIdentifier='redshiftml',
    DurationSeconds=3600,
    AutoCreate=False
)

## joblib_on_docker.md

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              1 star
            
          
                harusametime
                / joblib_on_docker.md
            
            
              Last active
              February 11, 2022 20:18
            
              
                Note on joblib with docker
              
          
    Problem

When passing big arrays to joblib.Parallel in docker container, parallel processing does not start immediately.
Why?

Joblib uses the folder specified by "JOBLIB_TEMP_FOLDER" for memmap of the arrays. Without specifying, JOBLIB_TEMP_FOLDER is set to /dev/shm, which usually has small size and is not enough for the big arrays.
https://pythonhosted.org/joblib/generated/joblib.Parallel.html
Solution


Set JOBLIB_TEMP_FOLDER
	import os
	import argparse
	import tensorflow as tf

	# global variables
	width = 28
	height = 28
	n_class = 10

	# tfrecord parser
	import boto3

	redshift = boto3.client('redshift')
	credentials = redshift.get_cluster_credentials(
	DbUser='demouser',
	DbName='dev',
	ClusterIdentifier='redshiftml',
	DurationSeconds=3600,
	AutoCreate=False
	)