slogging summary

summary.txt

Summary of the config and files in play for slogging...
-------------------------------------------------------

config file: /etc/swift/log-processor.conf

/usr/local/bin
--------  
swift-access-log-delivery
  => {config: log-processor.conf -> [log-processor-access] }
  => {class: from slogging.access_log_delivery import AccessLogDeliveryDaemon }
  => {processor: slogging.access_processor.AccessLogProcessor }
  => {cron: None }
  => runs as a daemon on the proxy servers from swift-init and collects data from the proxy server log.
  
swift-account-stats-logger 
  => {config: log-processor.conf -> [log-processor-stats] }
  => {class: from slogging.db_stats_collector import AccountStatsCollector }
  => {processor: slogging.stats_processor.StatsLogProcessor }
  => {cron: 0 * * * * root /usr/local/bin/swift-account-stats-logger /etc/swift/log-processor.conf }
  => gets data from the account server database.
  => generates a csv file to be uploaded to a container which can be processed by 'swift-log-stats-collector'.
  => csv columns: account, container count, object count, bytes used
  => should only be run on the account servers.

swift-container-stats-logger 
  => {config: log-processor.conf -> [log-processor-container-stats] }
  => {class: from slogging.db_stats_collector import ContainerStatsCollector }
  => {processor: slogging.stats_processor.StatsLogProcessor }
  => {cron: 5 * * * * root /usr/local/bin/swift-container-stats-logger /etc/swift/log-processor.conf }
  => gets data from the container server database.
  => generates a csv file to be uploaded to a container which can be processed by 'swift-log-stats-collector'.
  => csv columns: account, container name, object count, bytes used
  => should only be run on the container servers.
  
swift-log-stats-collector 
  => {config: log-processor.conf -> [log-processor] }
  => {class: from slogging.log_processor import LogProcessorDaemon }
  => {cron: 30 * * * * root /usr/local/bin/swift-log-stats-collector /etc/swift/log-processor.conf }
  => takes the account_stats and containter_stats csv files and the access_log (from proxy) and outputs a csv
  => should be run on its own box so it does not compete for resources with customer request handling.

swift-log-uploader
  => {config: log-processor.conf -> (respective section) }
  => {class: from slogging.log_uploader import LogUploader }
  => {cron:
        5 * * * * root /usr/local/bin/swift-log-uploader /etc/swift/log-processor.conf access
        10 * * * * root /usr/local/bin/swift-log-uploader /etc/swift/log-processor.conf stats
        15 * * * * root /usr/local/bin/swift-log-uploader /etc/swift/log-processor.conf container-stats
     }
  => used to upload csv log files for 'account' and 'container' to their respectively configured containers.
  => uploads the appropriate logs from '/var/log/swift/stats'.