Enable persistent storage for the systemd journal log

journalctl_enable_persistent_storage.md

Enable persistent storage for the systemd journal log

Overview

The assumed default setting in /etc/systemd/journald.conf is Storage=auto which implies that systemd journaling will only persist the journal if the expected storage location is available. Otherwise, the journal data is stored in memory and lost between reboots. On Ubuntu 16.04, /var/log/journal does not exist by default. Create it to keep and query events from previous boots.

Considerations:

• Syslog still provides the persistant log records for Ubuntu 16.04, so enabling persistant systemd journal logging does cause a level of duplicaiton.
• There are sane defaults:
  • SystemMaxUse is 10% for peristant storage in /var/log/journal, or 15% for memory use in /run/log/journal
  • SystemMaxFiles=100
  • SystemMaxFileSize = SystemMaxUse/8
• More config options such as MaxLevelStore can help control how fast storage is consumed.

Create storage and enable storage location

mkdir -p /var/log/journal
systemd-tmpfiles --create --prefix /var/log/journal
systemctl restart systemd-journald

Reference

• AskUbuntu - How to find previous boot log after Ubuntu 16.04 restarts?
• DigitalOcean - How To Use Journalctl to View and Manipulate Systemd Logs

As suggested by several other locations, use killall -USR1 systemd-journald instead of restarting the service. This way you get to keep all the logging of the current session: the memory log gets copied to the new disk location as well as all subsequent events.

Got it to work, but had this error still:

$ sudo systemd-tmpfiles --create --prefix /var/log/journal
Cannot set file attribute for '/var/log/journal', value=0x00800000, mask=0x00800000: Operation not supported

same with SamuelMarks

That's systemd/systemd#5607, probably?

As suggested by several other locations, use killall -USR1 systemd-journald

killall -USR1 /lib/systemd/systemd-journald for me (Raspbian GNU/Linux 10 (buster))