Skip to content

Instantly share code, notes, and snippets.

@satishgoda
Forked from mahmoud/crontab
Created February 10, 2017 17:21
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save satishgoda/f982f47f4003e52cfc7dd3219762f63f to your computer and use it in GitHub Desktop.
Save satishgoda/f982f47f4003e52cfc7dd3219762f63f to your computer and use it in GitHub Desktop.
A supervisord + jupyter notebook setup on a remote server, without root access.
# restart supervisor (and the notebook) every minute in case the machine has been restarted
*/1 * * * * /x/home/notebook_training/start_supervisord.sh >> /x/home/notebook_training/nbserver_start_log.txt
# separate user does something like:
# */1 * * * * /x/home/mhashemi/notebook_repo_update.sh >> /x/home/mhashemi/notebook_cron_log.txt 2>&1

Directions to set up

TODO: These steps are incomplete and not in the right order.

This particular setup recommends two users, one privileged, and one less-privileged. The more privileged user has SSH keys set up to back up to GitHub.

  • Privileged user grabs anaconda
  • Create a new user (notebook_training in this case)
  • New user git init --shared on the notebooks directory
# pretty much everything is default as generated except
# The IP address the notebook server will listen on.
c.NotebookApp.ip = '0.0.0.0'
# The port the notebook server will listen on.
c.NotebookApp.port = 8080
#!/bin/env bash
# installed with easy_install --user supervisor
cmd_return=$(~/.local/bin/supervisord -c ~/supervisord.conf 2>&1)
cmd_status=$?
echo $cmd_status
if [[ "$cmd_return" != "Error: Another program is already listening"* ]] ;
then
if [[ $cmd_status == 0 ]] ;
then
echo -n $(date)
echo " - supervisord started" ;
else
echo -n "Error occurred"
echo " - $cmd_return" ;
fi
; Notes:
; - Shell expansion ("~" or "$HOME") is not supported. Environment
; variables can be expanded using this syntax: "%(ENV_HOME)s".
; - Comments must have a leading space: "a=b ;comment" not "a=b;comment".
[unix_http_server]
file=/tmp/supervisor.sock ; (the path to the socket file)
;chmod=0700 ; socket file mode (default 0700)
;chown=nobody:nogroup ; socket file uid:gid owner
;username=user ; (default is no username (open server))
;password=123 ; (default is no password (open server))
[supervisord]
logfile=/x/local/notebook_training/logs/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10 ; (num of main logfile rotation backups;default 10)
loglevel=info ; (log level;default info; others: debug,warn,trace)
pidfile=/x/local/notebook_training/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false ; (start in foreground if true;default false)
minfds=1024 ; (min. avail startup file descriptors;default 1024)
minprocs=200 ; (min. avail process descriptors;default 200)
;umask=022 ; (process file creation umask;default 022)
;user=chrism ; (default is current user, required if root)
;identifier=supervisor ; (supervisord identifier, default is 'supervisor')
;directory=/tmp ; (default is not to cd during start)
;nocleanup=true ; (don't clean up tempfiles at start;default false)
;childlogdir=/tmp ; ('AUTO' child log dir, default $TEMP)
;environment=KEY="value" ; (key value pairs to add to environment)
;strip_ansi=false ; (strip ansi escape codes in logs; def. false)
; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket
[program:nbserver]
command=/x/local/notebook_training/anaconda/bin/ipython notebook --profile nbserver
directory=/x/local/notebook_training/notebooks
umask=022
stdout_logfile=/x/local/notebook_training/logs/nbserver_stdout.log
stderr_logfile=/x/local/notebook_training/logs/nbserver_stderr.log
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment