Tips and tricks for being successful with CRON

readme.md

I have amnesia when it comes to setting up bash files correctly. I will spend hours upon hours debugging the dumbest things simply because I can't remember the nuances of setting up my bash and shell files correct.

##Setting up CRON

• Open a crontab: crontab -e

• List the contents of your crontab: crontab -l

• Use bash, not sh, but adding the following line to your crontab:

  SHELL=/bin/bash
  

• Add any files to your path that you need. This is the default system one:

  PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
  

  • You might need to append e.g. :~/bin to the end.
  • You could also run echo PATH=$PATH to get what you should put there

• Using a special python configuration? Maybe add your PYTHONPATH

  PYTHONPATH=/usr/lib/python2.7/dist-packages:$PYTHONPATH
  

  • You should probably be using virtualenv tbh

• Use the following format:

  * * * * * . /full/path/to/file/shell_script.sh > /full/path/to/file/.cron.log 2>&1
  

  • Use a period, i.e. . /full.... to start it. Don't just put the file.
  • Use 2>&1 to redirect all output to a log file
  • Make sure there's a space after 2>&1
  • via https://www.thegeekstuff.com/2012/07/crontab-log

##Shell Script

• When naming your shell script, name it the exact same thing as your python file, but with a .sh extension.

• Add #!/bin/bash to the top

• If your script is shelling out to environment commands, make sure you add that to your path. You can get that by running echo PATH=$PATH

• You need to make sure you CD into the directory running the python script. make sure you have:

  DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
  cd "$DIR"
  

• Of course, if you want to end up in the directory you started, add the following towards the top:

  CURRENTDIR="$PWD"
  

  and the following to the bottom:

  cd $CURRENTDIR
  

• If you're using virtualenv to load packages, you'll want to make sure you set that up before calling your python script. I don't need that right now, but something to keep in mind.

##Logging in your python file

Don't rely on bash for your only logging. It's pretty simple to set up logging in Python. Here's an example using rotating files. You need to add this to the top of each file, though :-/

import logging
from logging.handlers import RotatingFileHandler
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
# add a rotating handler
handler = RotatingFileHandler(".python_script.log", maxBytes=2000000, backupCount=5)
formatter = logging.Formatter('%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)

And then when logging, do something like:

logger.info("this is a message about %s", "something that happened")

example_cron.txt

* * * * * env > /tmp/env.output
SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:~/bin
PYTHONPATH=/usr/lib/python2.7/dist-packages:$PYTHONPATH
* * * * * env > /tmp/env2.output
*/2 * * * * . /home/ben/example.sh > /home/ben/.example.log 2>&1

* * * * * date > .date.tmp

shell_script.sh

#!/bin/bash
PATH=/home/ben/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
CURRENTDIR="$PWD"
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
cd "$DIR"
python python_script.py
cd $CURRENTDIR

SHELL=/bin/bash saved my day!