Skip to content

Instantly share code, notes, and snippets.

@turboBasic
Last active January 30, 2021 23:27
Show Gist options
  • Save turboBasic/3673e69e4728173fe933b4c5cd33919f to your computer and use it in GitHub Desktop.
Save turboBasic/3673e69e4728173fe933b4c5cd33919f to your computer and use it in GitHub Desktop.
systemd journalctl cheatsheet #systemd #journalctl #linux

systemd journalctl cheatsheet / tldr

journalctl compile options

➜  journalctl --version

systemd 237
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid

examples of journal entries

user journal entry

➜  journalctl --reverse --user --boot 0 --output json-pretty --lines 1
{
    "__CURSOR" : "s=............REDACTED............;i=65a02;b=............REDACTED............;m=31a4c0ea92;t=5ba1f6c3f158c;x=6b324240936c61f2",
    "__MONOTONIC_TIMESTAMP" : "213217503890",
    "__REALTIME_TIMESTAMP" : "1612019006379404",
    "_AUDIT_LOGINUID" : "..UID..",
    "_AUDIT_SESSION" : "2",
    "_BOOT_ID" : "............REDACTED............",
    "_CAP_EFFECTIVE" : "0",
    "_CMDLINE" : "/usr/share/code/code --no-sandbox --unity-launch",
    "_COMM" : "code",
    "_EXE" : "/usr/share/code/code",
    "_GID" : "..UID..",
    "_HOSTNAME" : "....REDACTED....",
    "_MACHINE_ID" : "............REDACTED............",
    "_PID" : "19329",
    "_SELINUX_CONTEXT" : "unconfined\n",
    "_STREAM_ID" : "............REDACTED............",
    "_SYSTEMD_CGROUP" : "/user.slice/user-..UID...slice/session-2.scope",
    "_SYSTEMD_INVOCATION_ID" : "............REDACTED............",
    "_SYSTEMD_OWNER_UID" : "..UID..",
    "_SYSTEMD_SESSION" : "2",
    "_SYSTEMD_SLICE" : "user-..UID...slice",
    "_SYSTEMD_UNIT" : "session-2.scope",
    "_SYSTEMD_USER_SLICE" : "-.slice",
    "_TRANSPORT" : "stdout",
    "_UID" : "..UID..",
    "MESSAGE" : [ 00, 00, "...", 00 ],
    "PRIORITY" : "6",
    "SYSLOG_IDENTIFIER" : "code.desktop"
}

system journal entry

➜  journalctl --reverse --system --boot 0 --output json-pretty --lines 1
{
    "__CURSOR" : "s=............REDACTED............;i=65a1e;b=............REDACTED............;m=31b8b3b4d2;t=5ba1f8031dfcc;x=244f98457aa46a7",
    "__MONOTONIC_TIMESTAMP" : "213552182482",
    "__REALTIME_TIMESTAMP" : "1612019341057996",
    "_BOOT_ID" : "............REDACTED............",
    "_HOSTNAME" : "....REDACTED....",
    "_MACHINE_ID" : "............REDACTED............",
    "_SOURCE_MONOTONIC_TIMESTAMP" : "213552004162",
    "_TRANSPORT" : "kernel",
    "MESSAGE" : "audit: type=1400 audit(1612019341.049:379): apparmor=\"DENIED\" operation=\"open\" profile=\"/usr/sbin/cupsd\" name=\"/proc/23389/cgroup\" pid=23389 comm=\"cupsd\" requested_mask=\"r\" denied_mask=\"r\" fsuid=0 ouid=0",
    "PRIORITY" : "5",
    "SYSLOG_FACILITY" : "0",
    "SYSLOG_IDENTIFIER" : "kernel"
}

command examples

list syslog identifiers

➜  journalctl --field SYSLOG_IDENTIFIER

last entry of user journal from current boot

➜  journalctl --boot 0 --lines 1 --reverse --user

output as pretty json (output all fields as side-effect)

➜  journalctl --output json-pretty

all events from ‘code.desktop’ indentifier

➜  journalctl --user --identifier code.desktop

all events from syslog identifier ‘login’ since last boot

➜  journalctl --no-pager --no-hostname --boot 0 --identifier login   --lines 3 
-- Logs begin at Fri 2021-01-15 21:01:06 EET, end at Sat 2021-01-30 19:56:27 EET. --
Jan 30 16:21:56 login[3633]: pam_env(login:session): pam_putenv("AVAR3=override-avar3-1")
Jan 30 16:21:56 login[3633]: pam_env(login:session): pam_putenv("AVAR4=default-avar4-2")
Jan 30 16:21:56 login[3633]: pam_unix(login:session): session opened for user ...REDACTED... by LOGIN(uid=0)

typical invocation with sane option grouping

➜   journalctl --no-pager --reverse                     `# global options` \
               --user --boot=0                          `# scope of events` \
               --identifier=code.desktop --lines=1      `# what to print` \
               --output=json-pretty                     `# how to print`

replace some information in log output before sending out

➜  journalctl --boot=0 --identifier=login --output=json-pretty \
|  sed --regexp-extended "
       s/$(cat /etc/machine-id)/............REDACTED............/g
       s/[0-9a-f]{32,32}/............REDACTED............/g
       s/$(hostname)/....REDACTED..../g
       s/$(id --user)/..UID../g
   "

links

appendix

usage

➜  journalctl --help

journalctl [OPTIONS...] [MATCHES...]

Query the journal.

Options:
    --system                Show the system journal
    --user                  Show the user journal for the current user
-M  --machine=CONTAINER     Operate on local container
-S  --since=DATE            Show entries not older than the specified date
-U  --until=DATE            Show entries not newer than the specified date
-c  --cursor=CURSOR         Show entries starting at the specified cursor
    --after-cursor=CURSOR   Show entries after the specified cursor
    --show-cursor           Print the cursor after all the entries
-b  --boot[=ID]             Show current boot or the specified boot
    --list-boots            Show terse information about recorded boots
-k  --dmesg                 Show kernel message log from the current boot
-u  --unit=UNIT             Show logs from the specified unit
    --user-unit=UNIT        Show logs from the specified user unit
-t  --identifier=STRING     Show entries with the specified syslog identifier
-p  --priority=RANGE        Show entries with the specified priority
-g  --grep=PATTERN          Show entries with MESSSAGE matching PATTERN
    --case-sensitive[=BOOL] Force case sensitive or insenstive matching
-e  --pager-end             Immediately jump to the end in the pager
-f  --follow                Follow the journal
-n  --lines[=INTEGER]       Number of journal entries to show
    --no-tail               Show all lines, even in follow mode
-r  --reverse               Show the newest entries first
-o  --output=STRING         Change journal output mode (short, short-precise,
                            short-iso, short-iso-precise, short-full,
                            short-monotonic, short-unix, verbose, export,
                            json, json-pretty, json-sse, cat)
    --output-fields=LIST    Select fields to print in verbose/export/json modes
    --utc                   Express time in Coordinated Universal Time (UTC)
-x  --catalog               Add message explanations where available
    --no-full               Ellipsize fields
-a  --all                   Show all fields, including long and unprintable
-q  --quiet                 Do not show info messages and privilege warning
    --no-pager              Do not pipe output into a pager
    --no-hostname           Suppress output of hostname field
-m  --merge                 Show entries from all available journals
-D  --directory=PATH        Show journal files from directory
    --file=PATH             Show journal file
    --root=ROOT             Operate on files below a root directory
    --interval=TIME         Time interval for changing the FSS sealing key
    --verify-key=KEY        Specify FSS verification key
    --force                 Override of the FSS key pair with --setup-keys

Commands:
-h  --help                  Show this help text
    --version               Show package version
-N  --fields                List all field names currently used
-F  --field=FIELD           List all values that a specified field takes
    --disk-usage            Show total disk usage of all journal files
    --vacuum-size=BYTES     Reduce disk usage below specified size
    --vacuum-files=INT      Leave only the specified number of journal files
    --vacuum-time=TIME      Remove journal files older than specified time
    --verify                Verify journal file consistency
    --sync                  Synchronize unwritten journal messages to disk
    --flush                 Flush all journal data from /run into /var
    --rotate                Request immediate rotation of the journal files
    --header                Show journal header information
    --list-catalog          Show all message IDs in the catalog
    --dump-catalog          Show entries in the message catalog
    --update-catalog        Update the message catalog database
    --new-id128             Generate a new 128-bit ID
    --setup-keys            Generate a new FSS key pair

journal fields

➜  journalctl --fields | sort | column

AUDIT_FIELD_ADDR      	GLIB_DOMAIN                 	OBJECT_SYSTEMD_UNIT        	_AUDIT_FIELD_SIG
AUDIT_FIELD_APPARMOR  	GLIB_OLD_LOG_API            	OBJECT_SYSTEMD_USER_SLICE  	_AUDIT_FIELD_SOCK_TYPE
AUDIT_FIELD_BUS       	GNOME_SHELL_EXTENSION_NAME  	OBJECT_UID                 	_AUDIT_FIELD_SYSCALL
AUDIT_FIELD_EXE       	GNOME_SHELL_EXTENSION_UUID  	PRIORITY                   	_AUDIT_ID
AUDIT_FIELD_HOSTNAME  	INVOCATION_ID               	SEAT_ID                    	_AUDIT_LOGINUID
AUDIT_FIELD_INTERFACE 	JOB_RESULT                  	SESSION_ID                 	_AUDIT_SESSION
AUDIT_FIELD_LABEL     	JOB_TYPE                    	SHUTDOWN                   	_AUDIT_TYPE
AUDIT_FIELD_MASK      	JOURNAL_NAME                	SLEEP                      	_BOOT_ID
AUDIT_FIELD_MEMBER    	JOURNAL_PATH                	SYSLOG_FACILITY            	_CAP_EFFECTIVE
AUDIT_FIELD_NAME      	KERNEL_USEC                 	SYSLOG_IDENTIFIER          	_CMDLINE
AUDIT_FIELD_OPERATION 	LEADER                      	SYSLOG_PID                 	_COMM
AUDIT_FIELD_PATH      	LIMIT                       	TIMESTAMP_BOOTTIME         	_EXE
AUDIT_FIELD_PEER_LABEL	LIMIT_PRETTY                	TIMESTAMP_MONOTONIC        	_FSUID
AUDIT_FIELD_PEER_PID  	MAX_USE                     	UNIT                       	_GID
AUDIT_FIELD_PID       	MAX_USE_PRETTY              	USERSPACE_USEC             	_HOSTNAME
AUDIT_FIELD_SAUID     	MESSAGE                     	USER_ID                    	_KERNEL_DEVICE
AUDIT_FIELD_TERMINAL  	MESSAGE_ID                  	USER_INVOCATION_ID         	_KERNEL_SUBSYSTEM
AVAILABLE             	NM_CONNECTION               	USER_UNIT                  	_LINE_BREAK
AVAILABLE_PRETTY      	NM_DEVICE                   	_AUDIT_FIELD_ADDR          	_MACHINE_ID
BOLT_DEVICE_NAME      	NM_LOG_DOMAINS              	_AUDIT_FIELD_APPARMOR      	_PID
BOLT_DEVICE_STATE     	NM_LOG_LEVEL                	_AUDIT_FIELD_ARCH          	_SELINUX_CONTEXT
BOLT_DEVICE_UID       	N_DROPPED                   	_AUDIT_FIELD_CAPABILITY    	_SOURCE_MONOTONIC_TIMESTAMP
BOLT_LOG_CONTEXT      	N_RESTARTS                  	_AUDIT_FIELD_CAPNAME       	_SOURCE_REALTIME_TIMESTAMP
BOLT_TOPIC            	OBJECT_AUDIT_LOGINUID       	_AUDIT_FIELD_CODE          	_STREAM_ID
BOLT_VERSION          	OBJECT_AUDIT_SESSION        	_AUDIT_FIELD_COMPAT        	_SYSTEMD_CGROUP
CODE_FILE             	OBJECT_CAP_EFFECTIVE        	_AUDIT_FIELD_DENIED_MASK   	_SYSTEMD_INVOCATION_ID
CODE_FUNC             	OBJECT_CMDLINE              	_AUDIT_FIELD_FAMILY        	_SYSTEMD_OWNER_UID
CODE_LINE             	OBJECT_COMM                 	_AUDIT_FIELD_INFO          	_SYSTEMD_SESSION
CURRENT_USE           	OBJECT_EXE                  	_AUDIT_FIELD_IP            	_SYSTEMD_SLICE
CURRENT_USE_PRETTY    	OBJECT_GID                  	_AUDIT_FIELD_NAME          	_SYSTEMD_UNIT
DISK_AVAILABLE        	OBJECT_PID                  	_AUDIT_FIELD_OPERATION     	_SYSTEMD_USER_SLICE
DISK_AVAILABLE_PRETTY 	OBJECT_SELINUX_CONTEXT      	_AUDIT_FIELD_OUID          	_SYSTEMD_USER_UNIT
DISK_KEEP_FREE        	OBJECT_SYSTEMD_CGROUP       	_AUDIT_FIELD_PEER          	_TRANSPORT
DISK_KEEP_FREE_PRETTY 	OBJECT_SYSTEMD_INVOCATION_ID	_AUDIT_FIELD_PEER_ADDR     	_UDEV_DEVNODE
ERRNO                 	OBJECT_SYSTEMD_OWNER_UID    	_AUDIT_FIELD_PROFILE       	_UDEV_SYSNAME
EXIT_CODE             	OBJECT_SYSTEMD_SESSION      	_AUDIT_FIELD_PROTOCOL      	_UID
EXIT_STATUS           	OBJECT_SYSTEMD_SLICE        	_AUDIT_FIELD_REQUESTED_MASK

journal syslog facilities

➜  journalctl --field SYSLOG_FACILITY | sort | column

0         5         DEVICE    RFKILL
1         9         DHCP4     SETTINGS
10        AUDIT     DHCP6     SUPPLICANT
3         BT        DNS       SUSPEND
4         CONCHECK  MB        WIFI
5         CORE      PLATFORM

journal syslog identifiers

➜  journalctl --field SYSLOG_IDENTIFIER | sort | column

/usr/lib/gdm3/gdm-waylan... centrify-sshd.sh            gnome-terminal-server          rsyslogd
/usr/lib/gdm3/gdm-x-session centrifydc.sh               google-chrome.desktop          rtkit-daemon
/usr/sbin/gpm               certgp.pl                   gparted.desktop                runmappers
05efi                       chage                       gpg-agent                      runuser
10freedos                   chfn                        gpm                            salt-minion
10qnx                       chsh                        gpu-manager                    sensors
20microsoft                 cisco-anyconnect.desktop    groupadd                       set-cpufreq
30utility                   clientid_linux64            grub-common                    sh
50mounted-tests             code.desktop                gvfs-afc-volume-monitor        signal-desktop_signal...
83haiku                     com.ubuntu.SoftwarePrope... gvfsd                          snapd
AptDaemon                   com.ubuntu.WhoopsiePrefe... hardinfo.desktop               snapd-apparmor
AptDaemon.Trans             containerd                  htop.desktop                   speech-dispatcher
AptDaemon.Worker            copyfilev2.pl               io.snapcraft.Settings          spice-vdagent
CRON                        cracklib                    kernel                         sshd
Flameshot.desktop           cron                        keyboard-setup.sh              su
ModemManager                crontab                     kitty.desktop                  sudo
MotionPro.desktop           cups-browsed                kse.desktop                    sudoedit
NetworkManager              dbus-daemon                 landscape-client               svn
PackageKit                  dhclient                    login                          synaptic.desktop
accounts-daemon             dirmngr                     lvm                            systemd
acpid                       docker.dockerd              lws-sysprep.desktop            systemd-cryptsetup
acvpnagent                  dockerd                     macosx-prober                  systemd-fsck
acvpndownloader             ecryptfs-insert-wrapped-... mgsusageag                     systemd-hostnamed
acvpninstall                eset-esets_gui.desktop  ... mtp-probe                      systemd-journald
acvpnui                     eset-esets_gui_autostart... ndtask                         systemd-logind
acwebhelper                 esets_daemon                networkd-dispatcher            systemd-modules-load
adcert                      esets_gui                   newgrp                         systemd-resolved
adclient                    esets_mac                   nm-dispatcher                  systemd-rfkill
adflush                     falcon-sensor               ntp-systemd-wrapper            systemd-shutdown
adinfo                      falconctl                   ntpd                           systemd-sleep
adjoin                      falcond                     nvim_nvim.desktop              systemd-timesyncd
adlicense                   firefox.desktop             obexd                          systemd-udevd
adquery                     firefoxDeveloperEdition...  org.debian.apt                 thermald
adreload                    flameshot_flameshot.desk... org.freedesktop.FileManager    thunderbird.desktop
adsmb                       fstrim                      org.gnome.Nautilus             unix_chkpwd
alsactl                     fwupd                       org.gnome.SettingsDaemon.Me... update-notifier.desktop
anacron                     gcr-viewer                  org.gnome.Shell.desktop        useradd
ansible-ansible.builtin...  gdm-launch-environment]     org.gnome.keyring.SystemPro    usermod
ansible-apt                 gdm-password]               os-prober                      vlc_vlc.desktop
ansible-setup               gdm3                        packagekitd                    whoopsie
apparmor                    gnome-keyring-daemon        passwd                         wpa_supplicant
apport                      gnome-keyring-pkcs11.des... pkexec                         xbrlapi.desktop
at-spi-bus-launcher         gnome-keyring-secrets.de... polkitd                        zeal.desktop
atopacctd                   gnome-keyring-ssh.desktop   polkitd(authority=local)       zeitgeist-daemon
audit                       gnome-session               pulseaudio                     zeitgeist-maybe-vacuum
avahi-daemon                gnome-session-binary        report_installed_packages.p...
bluetoothd                  gnome-software-service.d... root
deactivate                  gnome-shell                 rng-tools

 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment