A general solution to python socket library lacking a default timeout.

python_socket_timeout.md

Python Socket Timeout

The Problem

It is well known that the standard python socket library has no default timeout (returns None).

[ec2-user@worker-node ~]$ python
>>> import socket
>>> socket.getdefaulttimeout()

Unfortunately without a default timeout, many dependent libraries will run without a timeout through inheritence. This means that these applications will also block while waiting for a connection. For queued applications this means a build up of jobs if there is a network problem, which might cause the host machine to OOM (out of memory).

A Solution

Setup your python environment with a default timeout and don't wory about the dilegence of third parties to set a default timeout for you.

To either globally set a timeout for all python libraries, or within your virtualenv, simply add a startup script in your site-packages folder called sitecustomize.py. See documentation for information.

#/usr/lib/python2.7/site-packages/sitecustomize.py

"""
Load and run before all python scripts are executed.
"""

def set_default_python_socket_timeout():
  import os
  import socket

  try:
      socket_timeout = float(os.environ.get('PYTHON_SOCKET_TIMEOUT'))
  except:
      socket_timeout = 60.0

  socket.setdefaulttimeout(socket_timeout)

This script will load and run before all python scripts are executed. Now any python application will have a default timeout of 60.0 seconds. Optionally, the environment variable PYTHON_SOCKET_TIMEOUT can override this fallback value.

Demo

Default socket timeout.

[ec2-user@worker-node ~]$ python
>>> import socket
>>> socket.getdefaulttimeout()
60.0

Now override with the environment variable.

[ec2-user@worker-node ~]$ export PYTHON_SOCKET_TIMEOUT="30.0"
[ec2-user@worker-node ~]$ python
>>> import socket
>>> socket.getdefaulttimeout()
30.0