logrotate Cheatsheet (by ChatGPT o1-preview)

logrotate-cheatsheet.md

Logrotate Cheatsheet

Logrotate is a powerful utility designed to manage and automate the rotation, compression, removal, and mailing of log files. This comprehensive guide covers all essential aspects of logrotate to help you effectively manage system logs.

---

Table of Contents

1. Basic Concepts
2. Configuration Files
3. Global Options
4. Log File Definition Options
5. Scripts Execution
6. Compression Options
7. Date Extension Options
8. Include Directive
9. Debugging and Testing
10. Force Rotation
11. State File
12. Common Pitfalls
13. Examples
14. Best Practices
15. References

---

Basic Concepts

• Rotation: Renaming a log file and creating a new one.
• Compression: Reducing the size of rotated log files.
• Retention: Keeping a specified number of old log files.
• Scripts: Commands executed before or after rotation.

---

Configuration Files

• Main Configuration File: /etc/logrotate.conf

• Additional Configuration: Files in /etc/logrotate.d/ are included by default.

• Syntax:

  <path/to/logfile> {
      options
  }
  

---

Global Options

Global options are set outside of any log file definition and apply to all logs unless overridden.

• frequency:
  • daily
  • weekly
  • monthly
  • yearly
• rotate <count>: Number of times to rotate before deletion.
• compress: Compresses logs after rotation.
• nocompress: Does not compress logs.
• delaycompress: Compress logs on the next rotation cycle.
• create [mode owner group]: Creates a new log file with specified permissions.
• nocreate: Does not create a new log file after rotation.
• mail <address>: Emails the log file to the specified address.
• nomail: Does not email the log file.
• ifempty: Rotates the log file even if it's empty.
• notifempty: Does not rotate empty log files.
• missingok: Ignores missing log files without error.
• sharedscripts: Runs scripts once per group of logs.
• postrotate/endscript: Commands to execute after rotation.

---

Log File Definition Options

These options are specified within a log file's braces {} and override global options.

• size <size>: Rotates if log file reaches specified size.
  • Units: bytes (default), k (kilobytes), M (megabytes), G (gigabytes).
• minsize <size>: Minimum size to trigger rotation.
• maxsize <size>: Maximum size before forceful rotation.
• maxage <days>: Removes logs older than specified days.
• olddir <directory>: Moves rotated logs to specified directory.
• su <user group>: Rotates logs using specified user and group.
• copytruncate: Copies and truncates the original log file.
• nocopytruncate: Does not truncate the original log file.
• copy: Copies the log file instead of moving/truncating.
• nocopy: Default behavior; does not copy the log file.

---

Scripts Execution

Scripts can be executed at different stages:

• prerotate/endscript: Executes before rotation.
• postrotate/endscript: Executes after rotation.
• firstaction/endscript: Executes once before any logs are rotated.
• lastaction/endscript: Executes once after all logs are rotated.

Example:

/var/log/example.log {
    weekly
    rotate 4
    postrotate
        systemctl reload example.service
    endscript
}

---

Compression Options

• compress: Compresses rotated logs using gzip by default.
• nocompress: Does not compress rotated logs.
• compresscmd <command>: Specifies a custom compression command.
• uncompresscmd <command>: Specifies a custom decompression command.
• compressoptions <options>: Options passed to the compression command.
• compressext <extension>: Sets custom extension for compressed files.

---

Date Extension Options

• dateext: Appends the date to rotated logs.
• dateformat <format>: Custom date format (used with dateext).
  • Format Specifiers:
    • %Y: Year (e.g., 2024)
    • %m: Month (01..12)
    • %d: Day (01..31)
    • %s: Seconds since epoch

Example:

dateext
dateformat -%Y-%m-%d

---

Include Directive

• include <file or directory>: Includes additional configuration files.

Example in /etc/logrotate.conf:

include /etc/logrotate.d

---

Debugging and Testing

• Dry Run: Use -d to simulate actions without making changes.

  logrotate -d /etc/logrotate.conf

• Verbose Mode: Use -v for detailed output.

  logrotate -v /etc/logrotate.conf

• Check Configuration: Use -f to force rotation and test configurations.

---

Force Rotation

• Use -f to force rotation regardless of rotation schedule.

  sudo logrotate -f /etc/logrotate.conf

---

State File

• Default State File: /var/lib/logrotate/status

• Contains timestamps of last rotation.

• Custom State File: Use -s to specify a different state file.

  logrotate -s /path/to/statefile /etc/logrotate.conf

---

Common Pitfalls

• Permissions: Ensure logrotate has appropriate permissions.
• Service Notification: Some services need to be reloaded after rotation.
• Copytruncate vs. Create: Choose based on application logging behavior.
• Script Errors: Verify scripts in prerotate and postrotate blocks.

---

Examples

Basic Rotation

/var/log/messages {
    rotate 5
    weekly
    postrotate
        systemctl reload rsyslog
    endscript
}

Compress Rotated Logs

/var/log/secure {
    rotate 4
    weekly
    compress
    missingok
    notifempty
}

Delay Compression

/var/log/wtmp {
    monthly
    create 0664 root utmp
    rotate 1
    delaycompress
}

Size-Based Rotation

/var/log/myapp.log {
    size 50M
    rotate 7
    compress
}

Rotate Multiple Logs Together

/var/log/httpd/*.log {
    daily
    rotate 14
    missingok
    notifempty
    sharedscripts
    postrotate
        systemctl reload httpd
    endscript
}

---

Best Practices

• Test Configurations: Always use -d option before applying changes.
• Monitor Disk Space: Regularly check log sizes and disk usage.
• Service Compatibility: Ensure services support log rotation mechanisms.
• Consistent Naming: Use date extensions for easier log management.
• Security: Secure log files with appropriate permissions.

---

References

• Manual Page: Access the logrotate manual.

  man logrotate

• Online Resources:

  • Linux Documentation Project
  • Official Logrotate GitHub

---

Note: This cheatsheet provides a comprehensive overview of logrotate functionalities up to October 2023. Always refer to the latest documentation or man pages for the most current information.

---

Prompt (to 4o): explain size vs maxsize vs minsize. they all sound the same

Great question! These three options in logrotate—size, maxsize, and minsize—do sound similar but serve different purposes. Here's how each works:

1. size

• Purpose: Rotates the log file as soon as it reaches the specified size, regardless of the rotation schedule (e.g., daily, weekly, etc.).
• Behavior: If a log file reaches or exceeds the specified size during a logrotate cycle, it will be rotated immediately.

Example:

/var/log/myapp.log {
    size 50M
    rotate 4
}

• In this example, myapp.log will be rotated when it reaches 50MB, even if the weekly or daily condition hasn't been met.

2. maxsize

• Purpose: Sets an upper limit on the size of a log file, forcing rotation when the file reaches or exceeds this limit, but only when the scheduled time arrives (e.g., daily, weekly).
• Behavior: The log file will not grow beyond the specified size when the time for the next rotation comes. It doesn't rotate immediately when the file exceeds the size limit, unlike size.

Example:

/var/log/myapp.log {
    weekly
    maxsize 100M
    rotate 4
}

• Here, the log file will be rotated weekly, but if during the week the log file exceeds 100MB, it will be rotated when the scheduled weekly cycle is triggered, not before.

3. minsize

• Purpose: Specifies a minimum size required to rotate the log file, even if the scheduled time arrives. The log file will only be rotated if it is at least the specified size.
• Behavior: If the log file is smaller than the specified minsize at the time of the scheduled rotation, it won’t be rotated.

Example:

/var/log/myapp.log {
    weekly
    minsize 10M
    rotate 4
}

• In this example, logrotate will check myapp.log every week, but it will only rotate it if the log file is at least 10MB. If it’s smaller, the log won’t be rotated until it meets the size requirement.

---

Summary of Differences:

• size: Rotates the log immediately once it reaches the specified size, ignoring the time-based rotation schedule.
• maxsize: Limits the size of the log file at the scheduled rotation. The log will be rotated during the next time-based cycle if it exceeds the specified maxsize.
• minsize: Ensures the log is rotated only if it’s larger than the specified size when the rotation time arrives.

Each option serves different use cases based on whether you want immediate action (size), control the upper limit of log files (maxsize), or enforce a minimum size for rotation (minsize).

Prompt (to 4o): maxsize and minsize still sound identical

It does seem like maxsize and minsize could sound similar, but they serve different roles in logrotate, especially in how they control rotation based on size in the context of the rotation schedule. Let me clarify further:

Key Difference:

• maxsize: Ensures that a log file never exceeds a certain size when the scheduled rotation time (e.g., daily, weekly) occurs.
• minsize: Ensures that a log file is rotated only if it is at least a certain size when the scheduled rotation time occurs.

---

More Detailed Explanation:

1. maxsize:

   • Think of maxsize as a hard upper limit. If the log file grows larger than this value before the next scheduled rotation, nothing happens. But at the time of the scheduled rotation, if the log file is larger than maxsize, it will immediately rotate, even if the file hasn’t yet reached its time-based rotation period.

   • Use case: You want to ensure a log file doesn’t exceed a specific size (e.g., you have limited disk space and don't want a log to grow indefinitely, so even if it's set to rotate daily or weekly, you don't want it to ever exceed the specified size).

   Example:

   /var/log/app.log {
       weekly
       maxsize 100M
   }
   

   • Behavior: Logrotate checks the file once a week (on the weekly schedule). If, at that point, the file is larger than 100MB, it will be rotated immediately. However, if the file is under 100MB, logrotate will simply follow the normal weekly rotation.

---

2. minsize:

   • Think of minsize as a threshold to prevent premature rotation. It ensures the log file is large enough to warrant rotation. If the log file is smaller than minsize at the time of the scheduled rotation, it won’t rotate at all, even if the scheduled rotation (daily, weekly, etc.) has arrived.

   • Use case: You don’t want to rotate small log files that haven’t grown enough to be worth rotating yet. For example, you may not want to create too many small rotated logs.

   Example:

   /var/log/app.log {
       weekly
       minsize 50M
   }
   

   • Behavior: Logrotate checks the file once a week (on the weekly schedule). If the log file is less than 50MB, it will not be rotated, even if it’s the end of the week. But if it's larger than 50MB, it will be rotated according to the schedule.

---

Simplified Scenarios:

1. maxsize:

   • "Rotate the log file once it gets too big."
   • Logrotate checks the file at its scheduled time (e.g., weekly). If the file exceeds maxsize, rotate it no matter what.

2. minsize:

   • "Only rotate the log file if it’s big enough."
   • Logrotate checks the file at its scheduled time (e.g., weekly). If the file hasn’t grown beyond minsize, don’t rotate it yet, even though it’s time.

---

Real-Life Example to Illustrate:

Suppose you're managing logs with the following settings:

/var/log/app.log {
    daily
    minsize 50M
    maxsize 100M
}

Day 1 (Log size: 30MB):

• Scheduled Rotation: Daily
• Action: No rotation, because the file is smaller than minsize.

Day 2 (Log size: 60MB):

• Scheduled Rotation: Daily
• Action: Rotate the log, because it is larger than minsize (50MB).

Day 3 (Log size: 105MB):

• Scheduled Rotation: Daily
• Action: Rotate the log, because it exceeds maxsize (100MB), even though it hasn’t reached a scheduled rotation.

---

Conclusion:

• maxsize: Forces a log to be rotated when it grows too large, even if it’s not yet time to rotate based on the schedule.
• minsize: Prevents a log from being rotated if it's too small, even if the scheduled rotation time has arrived.