Cloud Foundry Logging Proposal

gistfile1.md

Cloud Foundry Logging Proposal

Log Format

Each log line corresponds to a line of JSON. This is backward compatible with the existing steno output.

Common among all log-level formats:

{
  timestamp: TIMESTAMP, // formatted as fmt.Sprintf("%.9f", time.Now().UnixNano()) / 1e9)
  source: COMPONENT,
  message: COMPONENT.TASK.ACTION,
  log_level: LOG_LEVEL, // debug,info,error,fatal
  data: {}, //a hash of data (see below)
  file: FILENAME, // if present
  line: LINE_NUMBER, // if present
  method: METHOD_NAME, // if present
}

Users may add any additional JSON encoded information in the data field. However, different log levels guarantee that certain keys will be present in the data hash. Here's a breakdown:

Debug/Info

data: {
  description: "HUMAN READABLE DETAILS"
}

Error

data: {
  description: "HUMAN READABLE DETAILS",
  error:"ACTUAL ERROR MESSAGE" //For golang, this is: err.Error()
}

Fatal

data: {
  description: "HUMAN READABLE DETAILS",
  error:"ACTUAL ERROR MESSAGE" //For golang, this is: err.Error()
  trace:"FULL STACK TRACE" //blob of text with \ns in it
}

Example Go Interface

Here is a proposed Golang interface. The intent is to force the user to do The Right Thing by limiting their choice!

Logger Example

package logger

// NewLogger creates a logger that should be created at a top level and should
// be injected into all constructors.
func NewLogger(component string) Logger

// LoggerData is a JSON map of log-specific details
type LoggerData map[string]interface

// A Logger represents an active logging object that generates formatted lines
// of output to it's registered sinks. Multiple sinks may be registered and
// will be written to simultaneously.
type Logger interface{
  RegisterSink(sink Sink)
  Debug(task,action,description string, ...data LoggerData)
  Info(task,action,description string, ...data LoggerData)
  Error(task,action,description string, err error,...data LoggerData)
  // Fatal triggers a panic with "COMPONENT.TASK.ACTION - MESSAGE"
  Fatal(task,action,description string, err error,...data LoggerData)
}

// NewTestLogger returns an object which conforms to the Sink interface.
// TestLogger holds the logs in a memory buffer and has additional methods
// which query the logs for testing purposes.
func NewTestLogger(component string) *TestLogger

Sink Example

package sink

type LogLevel int

const(
  DEBUG LogLevel = iota
  INFO
  ERROR
  FATAL
)

// A Sink represents a write destination for a Logger.
type Sink interface {
  //Log to the sink.  Best effort -- no need to worry about errors.
  Log(level LogLevel, payload []byte)
}

// NewWriterSink creates a Sink for any io.Writer.  It guarantees to serialize
// access to the writer.
func NewWriterSink(writer io.Writer, minLogLevel LogLevel) Sink

// NewSyslogSink creates a Sink for any io.Writer.  It guarantees to serialize
// access to the writer.
func NewSyslogSink(tag string, minLogLevel LogLevel) Sink

CF_LOGGER Example

// cf_logger contains CloudFoundry-spefic wrappers, which contruct our default
// loggers and sinks so that we don't have to think about it, and have a
// centralized place to update and changes to logging policy.
package cf_logger

// NewCFLogger writes to a standard set of sinks.  It is comprised of:
//    NewWriterSink(os.Stdout, DEBUG)
//    NewSyslogSink(os.Stdout, INFO)
func NewCFLogger(component string) Logger