Skip to content

Instantly share code, notes, and snippets.

@tonyskapunk

tonyskapunk/todo_notes.md

Last active June 9, 2017 21:18
Show Gist options
  • Save tonyskapunk/ce601c67014982a8509f410bc661c974 to your computer and use it in GitHub Desktop.
Save tonyskapunk/ce601c67014982a8509f410bc661c974 to your computer and use it in GitHub Desktop.
Download ZIP
recap defaults
Raw
todo_notes.md

Status of the defaults and next steps

  • MAILTO - used in recap(default: ""). if defined it will send an email(send_mail) to that address with a report of the run.

  • LOG_COMPRESS - used in recaplog(default: 1). add to man

  • LOG_EXPIRY - used in recaplog(default: 0). add to man, defined in recap.conf(default: 15)

  • EMAIL_ON_ERROR, EMAIL_ON_ERROR_SUBJECT, EMAIL_ON_ERROR_RECIPIENTS - defined iny recaplog(default 1). not used, needs to be removed from recaplog and recap.conf

  • USEPS - used in recap(default: yes). to be commented out in recap.conf

  • USERESOURCES - used in recap(default: yes). takes are of running the resources(run_resources_report()) which includes at leasts: uptime, free, vmstat, iostat, iotop, additional resources are included if enabled. Needs to be commented out in recap.conf

  • USESAR - used in recap(default: no). recap.conf sets it to no as well, needs to be commented out in recap.conf. - People has requested this to be turned on("yes") by default.

  • USESARR - used in recap(default: no). takes care of memory data (sar -r). recap.conf sets it to no as well, needs to be removed from recap.conf.

  • USESARQ - used in recap(default: no). takes care of queue lenght(load) data (sar -q). recap.conf sets it to no as well, needs to be commented out in recap.conf.

  • USEFULLSTATUS - used in recap(default: no). recap.conf sets it to yes!, needs to be commented out in recap.conf.

  • STATUSURL - new variable used to avoid the use of apachectl, this is the URL that needs to be checked for the full status, default is "http://localhost:80/", needs to be added to the man page.

  • USEPSTREE - used in recap(default: no). recap.conf set it to no, needs to be commented out in recap.conf.

  • USENETSTAT - used in recap(default: no). recap.conf set it to yes!, needs to be changed in recap as new default: yes and needs to be commented out in recap.conf

  • USENETSTATSUM - used in recap(default: no). depends that USENETSTAT is set to yes, needs to be commented out in recap.conf

  • USEMYSQL - used in recap(default: no). needs to be commented out in recap.conf as is set to yes there.

  • USEMYSQLPROCESSLIST - used in recap(default: no). needs to be commented out in recap.conf as is set to yes!

  • USEINNODB - used in recap(default: no). needs to be commented out in recap.conf, needs to be added to man

  • DOTMYDOTCNF - used in recap(default: /root/.my.cnf). needs to be commented out in recap.conf, needs to be documented in man.

  • MYSQL_PROCESS_LIST - used in recap(default: table). needs to be commented out in recap.conf with the alternative.

  • USEDF - used in recap(default: no). needs to be changed to yes, needs to be commented out in recap.conf. Needs to be added to man.

  • USESLAB - used in recap(default: no). needs to be commented out in recap.conf, needs to be added to man.

  • USEFDISK - used in recap(default: no). needs to be commented out in recap.conf, needs to be commented out in recap.conf,

  • BACKUP_ITEMS - Used by recap and recaplog(default: "fdisk mysql netstat ps pstree resources"). needs to be commented out from recap.conf. Needs to be added to man. See NOTE 1.

  • MIN_FREE_SPACE - used in recap(default: 0). needs to be commented out from recap.conf, needs to be added to man. (Minimal required space in MB)

  • OPTS_DF - used in recap(default: "-x nfs"). needs to be commented out in recap.conf.

  • OPTS_FDISK - used in recap(default: "-l"). needs to be commented out in recap.conf, see NOTE 2.

  • OPTS_FREE - used in recap(default: ""). needs to be commented out in recap.conf.

  • OPTS_IOSTAT - used in recap(default: "-t -x 1 3"). needs to be commented out in recap.conf.

  • OPTS_IOTOP - used ony by recap(default: "-b -o -t -n 3")

  • OPTS_NETSTAT - used in recap(default: "-ntulpae"). needs to be commented out in recap.conf.

  • OPTS_NETSTAT_SUM - used in recap(default: "-s"). needs to be commented out in recap.conf.

  • OPTS_PS - only used by recap(default: "auxfww"). needs to be commented out in recap.conf

  • OPTS_PSTREE - used in recap(default: ""). changing default to "-p" to include PID, needs to be commented out in recap.conf. Need to update man

  • OPTS_VMSTAT - used in recap(default: "-S M 1 3"). needs to be commented out in recap.conf. Need to update man.

  • BASEDIR - used in recap and recaplog(default: "/var/log/recap"). was not defined as a setting that can be overriden, needs to be documented in recap.conf. Need to update man.

  • MAXLOAD - used in recap(old default: "1000", new default is dynamic 10 times the number of the CPUs identified by the kernel). Need to be added to man.

Notes

  1. Man page needs to be fixed to reflect the changes and document well the defaults and location of logs. (for USEXXX and OPTS_XXX)

  2. BACKUP_ITEMS either will have to be created dynamically from the options enabled, e.g. USE___ or the USE__ will be enabled from the BACKUP_ITEMS. IF one or the other is missing/disabled there will be inconsistenciews.

  3. FDISK needs love, needs to be more dynamic and use parted/fdisk, but this limits the options, it will have to be only "-l"

Other variables

recap

  • LOG_SUFFIX=$(date +%Y%m%d-%H%M%S)
  • PATH=/bin:/usr/bin:/sbin:/usr/sbin
  • DATE=date +%Y-%m-%d_%H:%M:%S ## In the snapshot functionality, this provides inconsistencies with LOG_SUFFIX, removing and using LOG_SUFFIX instead.
  • ROTATELOGS="yes"
  • SNAPSHOT="no"
  • BACKUP="no"
  • LOCKFILE="/var/lock/recap.lock"
  • BACKUP_ITEMS="fdisk mysql netstat ps pstree resources"
  • MAXLOAD="1000"
  • ROTATE="7" ## Does not seem to be used
  • OUTPUTDIR_MODE="750"

recaplog

  • PATH=/bin:/usr/bin:/sbin:/usr/sbin
  • LOCKFILE="/var/lock/recaplog.lock"
  • LOG_COMPRESS=1
  • LOG_EXPIRY=0
  • BACKUP_ITEMS="fdisk mysql netstat ps pstree resources"
  • LOGFILE="${BASEDIR}/recaplog.log"

  • These needs to be removed as recaplog does not send email anymore

  • MAIL_ON_ERROR=1
  • EMAIL_ON_ERROR_SUBJECT="recaplog error"
  • EMAIL_ON_ERROR_RECIPIENTS="root@localhost"

Setting Dependencies:

  • USERESOURCES

    • USESAR
    • USESARQ
    • USESARR
    • USEFULLSTATUS
    • USEDF

  • USENETSTAT

    • USENETSTATSUM

  • USEMYSQL

    • USEMYSQLPROCESSLIST
    • USEINNODB
    • DOTMYDOTCNF
    • MYSQL_PROCESS_LIST

Proposal on recap or /etc/recap.conf

- LOG_COMPRESS=1
+ #LOG_COMPRESS=1
- EMAIL_ON_ERROR=1
- EMAIL_ON_ERROR_SUBJECT="recaplog error"
- EMAIL_ON_ERROR_RECIPIENTS="root@localhost"
- USEPS=yes
+ #USEPS=yes
- USERESOURCES=yes
+ #USERESOURCES=no
- USESAR=no
+ #USESAR=yes
- USESARR=no
+ #USESARR=yes
- USESARQ=no
+ #USESARQ=yes
- USEFULLSTATUS=yes
+ #USEFULLSTATUS=yes
+ ## This setting defines the URL pre-configured on the webserver to query for the full status.
+ ## NGINX uses "nginx_status", while Apache HTTPD uses "server-status"
+ #STATUSURL="http://localhost:80/<STATUS_LOCATION>"
- USEPSTREE=no
+ #USEPSTREE=yes
- USENETSTAT=yes
+ #USENETSTAT=no
- USENETSTATSUM=no
+ #USENETSTATSUM=yes
- USEMYSQL=yes
+ #USEMYSQL=yes
- USEMYSQLPROCESSLIST=yes
+ #USEMYSQLPROCESSLIST=yes
- USEINNODB=no
- #USEINNODB=yes
- DOTMYDOTCNF=/root/.my.cnf
+ #DOTMYDOTCNF=/root/.my.cnf
- MYSQL_PROCESS_LIST=table
+ #MYSQL_PROCESS_LIST=vertical
- USEDF=yes
+ #USEDF=no
- USESLAB=no
+ #USESLAB=yes
- USEFDISK=no
+ #USEFDISK=yes
- BACKUP_ITEMS="fdisk mysql netstat ps pstree resources"
+ #BACKUP_ITEMS="fdisk mysql netstat ps pstree resources"
- MIN_FREE_SPACE=0
+ #MIN_FREE_SPACE=10
- OPTS_DF="-x nfs"
+ #OPTS_DF="-x nfs -PhT"
- USEFDISK=no
+ #USEFDISK=yes
- OPTS_FREE=""
+ #OPTS_FREE="-l"
- OPTS_IOSTAT="-t -x 1 3"
+ #OPTS_IOSTAT="-t -x 1 3"
- OPTS_IOTOP="-b -o -t -n 3"
+ #OPTS_IOTOP="-b -o -t -n5 -d2"
- OPTS_NETSTAT="-ntulpae"
+ #OPTS_NETSTAT="-punteCola"
- OPTS_NETSTAT_SUM="-s"
+ #OPTS_NETSTAT_SUM="-stuUSw"
- OPTS_PS="auxfww"
+ #OPTS_PS="-feaxww"
- OPTS_PSTREE=""
+ #OPTS_PSTREE="-psSu"
- OPTS_VMSTAT="-S M 1 3"
+ OPTS_VMSTAT="-twnSM 1 3"

Other changes

  1. STATUSURL - This was introduced to be more generic with USEFULLSTATUS, this will allow to query any webserver that supports status(httpd, nginx)
  2. Snapshots file name matches regular logs and backups, example of the differences:
## Regular format on recap
# ls -ltr /var/log/recap/ | tail -4
-rw-r--r--. 1 root root    820 Jun  9 14:30 pstree_20170609-143001.log
-rw-r--r--. 1 root root   6314 Jun  9 14:30 netstat_20170609-143001.log
-rw-r--r--. 1 root root    297 Jun  9 14:30 fdisk_20170609-143001.log
-rw-r--r--. 1 root root   5127 Jun  9 14:30 mysql_20170609-143001.log

## Backups use the same log formatting as recap regular logs:
# ls -ltr /var/log/recap/backups | tail -4
-rw-r--r--. 1 root root 6191 Jun  9 14:27 netstat_20170609-142001.log
-rw-r--r--. 1 root root 8070 Jun  9 14:27 ps_20170609-142001.log
-rw-r--r--. 1 root root  785 Jun  9 14:27 pstree_20170609-142001.log
-rw-r--r--. 1 root root 8407 Jun  9 14:27 resources_20170609-142001.log

## Snapshots uses a completely different formatting:
# ls -ltr /var/log/recap/snapshots/ | tail -4
-rw-r--r--. 1 root root  812 Jun  9 14:27 pstree.log_snapshot_2017-06-09_14:27:43
-rw-r--r--. 1 root root 6314 Jun  9 14:27 netstat.log_snapshot_2017-06-09_14:27:43
-rw-r--r--. 1 root root  297 Jun  9 14:27 fdisk.log_snapshot_2017-06-09_14:27:43
-rw-r--r--. 1 root root 5127 Jun  9 14:27 mysql.log_snapshot_2017-06-09_14:27:43

Now uses:

# ls -ltr /var/log/recap/snapshots/ | tail -4
-rw-r--r--. 1 root root  899 Jun  9 14:40 pstree_20170609-144025.log_snapshot
-rw-r--r--. 1 root root 6437 Jun  9 14:40 netstat_20170609-144025.log_snapshot
-rw-r--r--. 1 root root  297 Jun  9 14:40 fdisk_20170609-144025.log_snapshot
-rw-r--r--. 1 root root 5127 Jun  9 14:40 mysql_20170609-144025.log_snapshot
@troyengel
Copy link

One comment - the commented out config file options should match the internal defaults, so these would be:

#USEFULLSTATUS=no
#USEMYSQL=no

I tried to find out if this is written down anywhere, it's not part of an official specification that I can find - it is a cultural (UNIX/Linux) expectation that is reflected in many documents (https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/DM_Multipath/config_file_defaults.html e.g.) as "how we do things" but not an official law. We just... do it that way. :)

@tonyskapunk
Copy link
Author

tonyskapunk commented Jun 9, 2017 •
edited

@troyengel aha!, thanks! I was trying to find something about it(oficial or standard or best practice), so what I'm going to do is to leave the defaults but document examples of other options to give ideas then. Edit: Or maybe create an example file of different options...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment