Skip to content

Instantly share code, notes, and snippets.

@cbkmephisto
Last active July 8, 2019 14:40
Show Gist options
  • Save cbkmephisto/b32451982d73d9e69da9d9356b38c453 to your computer and use it in GitHub Desktop.
Save cbkmephisto/b32451982d73d9e69da9d9356b38c453 to your computer and use it in GitHub Desktop.

How to add a title to Seaborn Facet Plot

Reference link

import seaborn as sns
from matplotlib import pyplot as plt

# load data and plot using sns
tips = sns.load_dataset("tips")
g = sns.FacetGrid(tips, col="sex", row="smoker", margin_titles=True)
g.map(sns.plt.scatter, "total_bill", "tip")

# adjust height of sns plot and add title
plt.subplots_adjust(top=0.9)
g.fig.suptitle('THIS IS A TITLE, YOU BET') # can also get the figure from plt.gcf()

numpy save/load array to binary file with gzipped

Need to

import gzip

original post link

Hi, I was trying to read directly from a compressed file:

data = np.array([9, 8, 7, 6, 5, 4, 3, 2, 1])
dt = data.type
with gzip.open("datafile.gz", "wb") as outfile:
    outfile.write(data.tobytes())

with gzip.open("datafile.gz", "rb") as infile:
    data = np.fromfile(infile, dtype=dt)
    print(data)

Unfortunately, this returns garbage data:

[6542475788951259935 7594864974085029634 1008947487324530028 3113290099057413416 91954333390038676 6865743131250406218]

The workaround is to load the data into a buffer first, and then let np.frombuffer read it:

with gzip.open("datafile.gz", "rb") as infile:
    data = np.frombuffer(infile.read(), dtype=dt)
    print(data)

Returns: [9 8 7 6 5 4 3 2 1] as expected.

I think silently returning corrupt data is pretty much the worst possible behaviour in this ase. Either of these options would be an improvement:

  • Teach np.fromfile() to read from gzip (and possible other compression formats) file bjects
  • Raise an error explaining that NumPy can't deal with this kind of file object
  • At least clearly document what kind of "file object" np.fromfile() expects.

Right now, the file parameter of np.fromfile is documented as:

file : file or str

Open file object or filename.

And the file object returned from gzip.open is a file object, but apparently not the right ind. That's pretty confusing and should at least be changed in the documentation.

Operating System: Fedora 27, 64-Bit

Python version: 3.6.4

NumPy version: 1.13.3


Installing Jupyterhub on cuda1, 2017-09-18

pre-req

  • python3
  • sudo user

install

sudo apt-get install npm nodejs-legacy

sudo -H pip3 install jupyterhub
sudo npm install -g configurable-http-proxy
sudo -H pip3 install notebook

SSL: assuming certbot (letsencrypt) is installed and working

jupyterhub --generate-config
# this command will generate a config file "jupyterhub_config.py" in current dir
  • edit "jupyterhub_config.py"
# optional: change "c.JupyterHub.port = 8000" to whatever port you prefer
c.JupyterHub.port = 9999
c.JupyterHub.ssl_cert = '/opt/certkeys/fullchain.pem'
c.JupyterHub.ssl_key = '/opt/certkeys/privkey.pem'
# admin user
c.Authenticator.admin_users = {'hailins', 'rohan'}
# Allowing of use LocalAuthenticator to create system users
c.LocalAuthenticator.create_system_users = True
# JupyterHub.admin_access if False: disabled
  • sudo edit "/etc/cron.d/certbot", add the cp part so that renewed pem files are available to everyone
0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(3600))' && certbot -q renew && cp /etc/letsencrypt/live/cuda1.ansci.iastate.edu/[fp]*.pem /opt/certkeys/
  • Cookie secret
openssl rand -hex 32 > jupyterhub_cookie_secret
  • SSL: WRONG_VERSION_NUMBER problem solution: mv ~/.jupyter away, say, to ~/.jupyter.bak

Using sudo to run JupyterHub without root privileges

# add user "jupyterhub" and group "jupyterhub"
sudo useradd jupyterhub
sudo -H pip3 install sudospawner
# add existing users into group jupyterhub
sudo adduser hailins jupyterhub
sudo adduser rohan jupyterhub
  • modify sudoers via visudo, adding these content
# the command(s) the Hub can run on behalf of the above users without needing a password
# the exact path may differ, depending on how sudospawner was installed
Cmnd_Alias JUPYTER_CMD = /usr/local/bin/sudospawner

# actually give the Hub user permission to run the above command on behalf
# of the above users without prompting for a password
jupyterhub ALL=(%jupyterhub) NOPASSWD:JUPYTER_CMD
  • enabling PAM for non-root
sudo usermod -a -G shadow jupyterhub
sudo passwd jupyterhub # 'I wont tell you'
  • Make a Directory for JupyterHub
sudo mkdir /etc/jupyterhub
sudo chown jupyterhub /etc/jupyterhub
# copy all the configuration files (3) here!!!
  • Finally, start the server as our newly configured user
cd /etc/jupyterhub
sudo -u jupyterhub jupyterhub --JupyterHub.spawner_class=sudospawner.SudoSpawner

Run jupyter notebook on server

Referenced from THIS LINK

1 Create the config file. In bash, server side

jupyter notebook --generate-config

2 Generate password. In python3, anywhere

from notebook.auth import passwd
print(passwd())
# copy the output to clipboard like 'sha1:0f89d3214bef:8343fc31e75b9d679c5ca6d02ac6ef530de51bb3'

3 Use SSL to encrypt communication. In bash, server side

openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mykey.key -out mycert.pem

# put these two output files somewhere

4 Edit the config file ~/.jupyter/jupyter_notebook_config.py, modify and appending (server side)

# Set options for certfile, ip, password, and toggle off browser auto-opening
c.NotebookApp.certfile = u'/absolute/path/to/your/certificate/mycert.pem'
c.NotebookApp.keyfile = u'/absolute/path/to/your/certificate/mykey.key'
# Set ip to '*' to bind on all interfaces (ips) for the public server
c.NotebookApp.ip = '*'
c.NotebookApp.password = u'sha1:bcd259ccf...<your hashed password here>'
c.NotebookApp.open_browser = False
# It is a good idea to set a known, fixed port for server access
c.NotebookApp.port = 9999

5 lauch the server. In bash, on server side,

jupyter notebook

6 MAC user: from serverside fetch "mycert.pem" to local machine, double click it, add it in any keychain; find it in the keychain app, and rightclick on that, select “get info” and try to trust it

7 You can visit the remote ijulia by typing address "https://[DOMAIN_NAME]:[PORT]" in your local browser. Fill in the password and go

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment