Skip to content

Instantly share code, notes, and snippets.

@ra-tolson
Last active May 31, 2017 17:21
Show Gist options
  • Save ra-tolson/935d14b4129de120409679548adee643 to your computer and use it in GitHub Desktop.
Save ra-tolson/935d14b4129de120409679548adee643 to your computer and use it in GitHub Desktop.
Fixing the datahandler logging facility

Because the datahandler's logging was written in a huge hurry at the last minute, it's going to need a rewrite:

  • A custom object is used for logging instead of django's logging facility; it's already running if you orm.setup().
  • The root logger ('') is always hardcoded to talk to a socket; sooner or later we'll need a configuration-based logging config (which again django already provides).
  • The custom method, Logger.log, calls python's logging.log, thus losing information as funcName will always be log; the custom extra used instead, caller, is not reliable, as 3rd party libs don't know to use it (we had this problem with RQ).
  • In dhd.serve_log, jobid and caller had to be taken out due to above issues with RQ. This is because logging.basicConfig sets a configuration for the root (ie global) logger that makes poor assumptions.
  • It's a datahandler logger, not a gips logger. In the long run, there is no meaningful separation between gips logging and datahandler logging, only differential configs based on various use cases.

Fixes for the client side:

  • Use django's logging, including existing means to have django read the gips config. This can be used to set the logging server to talk to, etc etc.
  • Modules do what they're supposed to do: Log to a named log specific to that part of the application (python's logging tutorial says to use the fully-qualified module name via: l = logging.getLogger(__name__); l.info('message'))
  • Go through existing logging class and make sure it's not necessary; any additional wiring (say, saving a jobid) should be done in the pythonic way for logging, maybe via a custom LoggerAdapter:

https://docs.python.org/2/howto/logging-cookbook.html#adding-contextual-information-to-your-logging-output

Fixes for the server side:

  • Revise logging.basicConfig call so it makes no assumptions about custom logging anything.
  • Set up formatters or other wiring to handle logs whose names match 'gips' or 'gips.datahandler', and thus pick up extra custom info for those logs.
@ra-tolson
Copy link
Author

Note that github gists don't notify when you comment, so it's best to have discussion elsewhere.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment