Skip to content

Instantly share code, notes, and snippets.

@eloylp
Last active February 27, 2024 05:57
  • Star 64 You must be signed in to star a gist
  • Fork 7 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save eloylp/b0d64d3c947dbfb23d13864e0c051c67 to your computer and use it in GitHub Desktop.
Fedora 35 hibernation with swapfile, only for hibernation and resume

Fedora35 hibernation

This guide helps to configure the hibernation on a default Fedora35 (also worked fine in previous Fedora34) installation by using a swap file. The Fedora35 installation comes with btrfs as default filesystem. Also, it comes with a zram swap device:

$ swapon
NAME       TYPE      SIZE USED PRIO
/dev/zram0 partition   8G   0B  100

This device reserves a physical memory area in which all the content will be compressed (at its input) and uncompressed (at its output). But, we cannot hibernate with this type of swap space as it is only in memory.

After feedback received from contributors of this guide, seems we can say that this guide will work in both, encrypted and unencrypted setups.

important note before you continue: Some of us are experimenting problems with nvidia proprietary drivers. See below comments for more information on the investigation.

Motivation

Nowadays we have lots of RAM in our laptops. In my case, a guy with a laptop for programming, i only use swap space for hibernation. When i move from one location to another, i dont want to loose all my open stuff nor consuming battery while i am moving.

I want to preserve the zram swap device of the default configuration.

Solution

This are the steps we need to perform in our system:

  1. The hibernation is triggered by the user.
  2. The swap file is activated.
  3. The zram device is deactivated. If there are any memory pages present in zram, they will be moved to the activated (2) swap file.
  4. Hibernate the laptop.

And the following sequence for resuming:

  1. Power on the computer.
  2. Restore system state from the swap file at boot time.
  3. Activate the zram device.
  4. Deactivate the swap file. That could cause the zram device to start compressing data.

Seems there are efforts in order to make this much better, even with dynamic sized swap files, so only generating swap files for hibernation with the needed size on each moment, and more user friendly.

This guide will make use of a fixed size swap file, since probably for going further is better to just contribute in the mentioned systemd issue .

Steps

Default Fedora35 installations comes with btrfs as default filesystem. Such file system comes with the subvolume feature. Subvolumes are not partitions. They are just a logical separations of a filesystem at a file level. Some kind of operations like snapshots, will try to include the swap file. We need to prevent this by isolating the swap file into its own subvolume:

btrfs subvolume create /swap

The above will ensure the swap file will not be taken into account in other snaphots as they are not recursive in other subvolumes.

Next is to create our swapfile for hibernation. Ensure the specified size is enough for saving the contents of the RAM + the uncompressed contents of the zram swap space. How to determine the size ? that might be a rough estimation per use case. As root:

touch /swap/swapfile
chattr +C /swap/swapfile  ## Needed to disable Copy On Write on the file.
fallocate --length 33GiB /swap/swapfile  ## Please, calculate your size as mentioned in above comments.
chmod 600 /swap/swapfile 
mkswap /swap/swapfile 

Now lets prepare the path of the resume operation. We need add to the initramfs the necessary modules for resuming the system. We need to create a new file at /etc/dracut.conf.d/resume.conf with the following content:

add_dracutmodules+=" resume "

After that we need to regenerate the initramfs:

dracut -f

Next, we have to add the resume and resume_offset into the GRUB_CMDLINE_LINUX, so that Grub can instruct the kernel coordinates where the swap file resides, in order to resume the system.

For gathering the resume param we need the partition UUID in which the swap file is stored:

$ findmnt -no UUID -T /swap/swapfile
dbb0f71f-8fe9-491e-bce7-4e0e3125ecb8

Now lets gather the last needed data, we need the resume_offset. That is, the physical offset of our swap file in the file system. For doing that, we need to follow this guide.

Now we need to instruct GRUB to initialize the kernel with this coordinates. Edit the /etc/defaut/grub and grab there the parameters we just calculated:

GRUB_CMDLINE_LINUX="rd.luks.uuid=luks-4369a407-2be1-4f37-9764-ff848a0f2089 resume=UUID=dbb0f71f-8fe9-491e-bce7-4e0e3125ecb8 resume_offset=2459934 rhgb quiet"

Now lets re-configure the grub (UEFI setup assumed):

grub2-mkconfig -o /boot/grub2/grub.cfg

Note: We only want the swap file to be used when the hibernation takes place and in the resume stage. we are not going to configure fstab entries. To achieve that, one of the options is to use systemd. We are going to configure 2 systemd services. One for preparing the hibernation and the other one for resuming it:

For enabling the swap file and disabling the zram swap device before hibernation, lets create the file /etc/systemd/system/hibernate-preparation.service:

[Unit]
Description=Enable swap file and disable zram before hibernate
Before=systemd-hibernate.service

[Service]
User=root
Type=oneshot
ExecStart=/bin/bash -c "/usr/sbin/swapon /swap/swapfile && /usr/sbin/swapoff /dev/zram0"

[Install]
WantedBy=systemd-hibernate.service

The order is important in the above service definition. First of all we enable the swap file, which should have enough space to store the contents of the "in use RAM", plus the contents of the uncompressed zram swap device. Secondly, we disable the zram swap device. At that moment, the kernel will start moving all the memory pages from the zram swap device to the swap file if needed. After all of that, the hibernation will take place only in the swap file. Last step for the above script to have effect is to install this service in systemd:

# systemctl enable hibernate-preparation.service
Created symlink /etc/systemd/system/systemd-hibernate.service.wants/hibernate-preparation.service → /etc/systemd/system/hibernate-preparation.service.

We need to disable the swap file when the system just resumed. Lets create the /etc/systemd/system/hibernate-resume.service file:

[Unit]
Description=Disable swap after resuming from hibernation
After=hibernate.target

[Service]
User=root
Type=oneshot
ExecStart=/usr/sbin/swapoff /swap/swapfile

[Install]
WantedBy=hibernate.target

Then enable it by:

# systemctl enable hibernate-resume.service

Created symlink /etc/systemd/system/hibernate.target.wants/hibernate-resume.service → /etc/systemd/system/hibernate-resume.service.

In order to make the suspend-then-hibernate sequence to work, please take a look at this great comment and follow the instructions there.

In order to avoid false positives regarding to swap space, due to the zram device existence, we need to disable some checks:

mkdir -p /etc/systemd/system/systemd-logind.service.d/
cat <<-EOF | sudo tee /etc/systemd/system/systemd-logind.service.d/override.conf
[Service]
Environment=SYSTEMD_BYPASS_HIBERNATION_MEMORY_CHECK=1
EOF

mkdir -p /etc/systemd/system/systemd-hibernate.service.d/
cat <<-EOF | sudo tee /etc/systemd/system/systemd-hibernate.service.d/override.conf
[Service]
Environment=SYSTEMD_BYPASS_HIBERNATION_MEMORY_CHECK=1
EOF

Now you must reboot your computer, in order the steps until here take effect.

We also need to allow systemd-sleep system to read the swap file in SELinux . One option for allowing this is to make it fail. After that, use the audit2allow to do the white listing. Lets go step by step:

  1. Try to hibernate:

    # systemctl hibernate

    This should fail right now. Probably returning you to the display manager login. Also, logging details at /var/log/audit/audit.log .

  2. We can check the event happened with audit2allow inspecting the log, it will ouput something similar to this entry among others:

    # audit2allow -w -a
    
    type=AVC msg=audit(1630262756.460:2098): avc:  denied  { search } for  pid=26180 comm="systemd-sleep" name="swap" dev="dm-0" ino=256 scontext=system_u:system_r:systemd_sleep_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=dir permissive=0
    	Was caused by:
    		Missing type enforcement (TE) allow rule.
    
    		You can use audit2allow to generate a loadable module to allow this access.3.
  3. To see what rule we must allow, just type:

    # audit2allow -b
    
    #============= systemd_sleep_t ==============
    allow systemd_sleep_t unlabeled_t:dir search;

    The above rule should be the only one in the output. If not, we could be white listing other elements by accident.

  4. Instruct SELinux to allow further attemps by executing the following commands:

    # cd /tmp
    # audit2allow -b -M systemd_sleep
    ******************** IMPORTANT ***********************
    To make this policy package active, execute:
    
    semodule -i systemd_sleep.pp
    
    # semodule -i systemd_sleep.pp

We should be ready to hibernate now. Lets just try it by:

# systemctl hibernate

After resuming, the swap file should be deactivated, so continuing with the default zram swap device setup:

$ swapon
NAME       TYPE      SIZE USED PRIO
/dev/zram0 partition   8G   0B  100

As a bonus, you can enable gnome environment hibernate buttons by installing this extension.

Please check that all your devices state are properly restored after resuming.

Troubleshooting

If you are having unexpected problems, inspectioning the journal will be of help to see errors in systemd scripts:

journalctl -f
@ceiphr
Copy link

ceiphr commented Dec 25, 2022

@eloylp thank you for the reassurance!

@w4tsn thank you for the thorough response, that makes a lot of sense! Hopefully, we will see developments with this in the future.

For now, here are some interesting developments I've found related to this topic:

@alexspurling
Copy link

alexspurling commented Jan 10, 2023

I think the description already specifies an idea on how to calculate the space for the swap file. However, i added a comment in the code for the quick reader.

Firstly, thank you very much for the very well explained tutorial. Could you please explain this further? The article says you should ensure the size of the swap file "is enough for saving the contents of the RAM + the uncompressed contents of the zram swap space", but how can we calculate the uncompressed size of the contents of zram? As I understand, it will depend on the exact data that is stored as well as the selected compression algorithm.

I have 32GB of ram, an 8GB zram file using LZO-RLE compression. How can I calculate the required size of the swap file?

@eloylp
Copy link
Author

eloylp commented Jan 11, 2023

Hello @alexspurling

Thank you for taking time of writing ! Yeah, you are right. That will depend on each use case/setup. I think the idea was to provide a rough estimation, trying to be generous with the size. But it was not clear in the guide. I just fixed that.

I am not sure if this is something we could profile in an easy way. Probably it doesn't worth the effort. I think its something we can always adjust on demand, as we are not dealing with fixed partitions, but with a swap file we can reconfigure later on, after the initial setup.

Many thanks for your contribution !

@stephane-klein
Copy link

stephane-klein commented Feb 17, 2023

Hello,

I have executed all the instructions indicated in the guide Fedora35 hibernation that I found on Fedora Magazine.

  • My laptop: Thinkpad T14s Gen 3 AMD
  • OS: Fedora 37, totally up to date, freshly installed this morning
  • Secure Boot disable in BIOS
  • volume LUKS encryption enabled
# neofetch
             .',;::::;,'.                root@MiWiFi-R4A-srv 
         .';:cccccccccccc:;,.            ------------------- 
      .;cccccccccccccccccccccc;.         OS: Fedora Linux 37 (Workstation Edition) x86_64 
    .:cccccccccccccccccccccccccc:.       Host: 21CQCTO1WW ThinkPad T14s Gen 3 
  .;ccccccccccccc;.:dddl:.;ccccccc;.     Kernel: 6.1.11-200.fc37.x86_64 
 .:ccccccccccccc;OWMKOOXMWd;ccccccc:.    Uptime: 9 mins 
.:ccccccccccccc;KMMc;cc;xMMc:ccccccc:.   Packages: 1854 (rpm) 
,cccccccccccccc;MMM.;cc;;WW::cccccccc,   Shell: bash 5.2.15 
:cccccccccccccc;MMM.;cccccccccccccccc:   Resolution: 1920x1200 
:ccccccc;oxOOOo;MMM0OOk.;cccccccccccc:   WM: Mutter 
cccccc:0MMKxdd:;MMMkddc.;cccccccccccc;   WM Theme: Adwaita 
ccccc:XM0';cccc;MMM.;cccccccccccccccc'   Theme: Adwaita [GTK3] 
ccccc;MMo;ccccc;MMW.;ccccccccccccccc;    Icons: Adwaita [GTK3] 
ccccc;0MNc.ccc.xMMd:ccccccccccccccc;     Terminal: gnome-terminal 
cccccc;dNMWXXXWM0::cccccccccccccc:,      CPU: AMD Ryzen 7 PRO 6850U with Radeon Graphics (16) @ 2.700GHz 
cccccccc;.:odl:.;cccccccccccccc:,.       GPU: AMD ATI Radeon 680M 
:cccccccccccccccccccccccccccc:'.         Memory: 2017MiB / 30846MiB 
.:cccccccccccccccccccccc:;,..
  '::cccccccccccccc::;,.                

Problems encountered:

  • the resuming take 2min after entering my LUKS encryption password
  • wifi not working after resuming

Here is the systemd journal content: https://gist.github.com/stephane-klein/29198d3ed43e01a2ba9d0360e6deedd8

Do you know how to fix this issues?

Best regards,
Stéphane


Note: same post on Thinkpad Subreddit and Ask Fedora Forum

@stephane-klein
Copy link

stephane-klein commented Mar 1, 2023

In my dmesg (click to see full content), I see:

[  213.051537] Call Trace:
[  213.051539]  <TASK>
[  213.051541]  ieee80211_do_stop+0x647/0x8b0 [mac80211]
[  213.051591]  ieee80211_stop+0x49/0x170 [mac80211]
[  213.051633]  __dev_close_many+0x8e/0xf0
[  213.051642]  dev_close_many+0x7b/0x120
[  213.051648]  ? ieee80211_handle_reconfig_failure+0x69/0x90 [mac80211]
[  213.051696]  dev_close+0x59/0x80
[  213.051701]  cfg80211_shutdown_all_interfaces+0x49/0xf0 [cfg80211]
[  213.051742]  wiphy_resume+0x94/0x150 [cfg80211]
[  213.051780]  ? wiphy_suspend+0x2b0/0x2b0 [cfg80211]
[  213.051840]  dpm_run_callback+0x4a/0x150
[  213.051845]  device_resume+0xa2/0x1f0
[  213.051848]  async_resume+0x19/0x30
[  213.051852]  async_run_entry_fn+0x30/0x130
[  213.051856]  process_one_work+0x1c7/0x380
[  213.051859]  worker_thread+0x4d/0x380
[  213.051862]  ? rescuer_thread+0x380/0x380
[  213.051864]  kthread+0xe9/0x110
[  213.051868]  ? kthread_complete_and_exit+0x20/0x20
[  213.051873]  ret_from_fork+0x22/0x30
[  213.051880]  </TASK>
[  213.051881] ---[ end trace 0000000000000000 ]---
[  213.051903] ------------[ cut here ]------------

...

[  213.052301] ieee80211 phy0: PM: dpm_run_callback(): wiphy_resume+0x0/0x150 [cfg80211] returns -11
[  213.052317] ieee80211 phy0: PM: failed to restore async: error -11

I don't know if this detail is usefull to fix my issue :thinking: .

@stephane-klein
Copy link

In my dmesg (click to see full content), I see:

I see also :

[  210.043130] ath11k_pci 0000:01:00.0: failed to resume mhi: -5
[  210.043134] ath11k_pci 0000:01:00.0: failed to resume hif during resume: -5
[  210.043136] ath11k_pci 0000:01:00.0: failed to resume core: -5
[  210.043138] ath11k_pci 0000:01:00.0: PM: dpm_run_callback(): pci_pm_restore+0x0/0xe0 returns -5
[  210.043158] ath11k_pci 0000:01:00.0: PM: failed to restore async: error -5
[  213.050611] ath11k_pci 0000:01:00.0: wmi command 16387 timeout
[  213.050624] ath11k_pci 0000:01:00.0: failed to send WMI_PDEV_SET_PARAM cmd
[  213.050632] ath11k_pci 0000:01:00.0: failed to enable dynamic bw: -11

@stephane-klein
Copy link

@mt190502
Copy link

mt190502 commented Mar 5, 2023

In my dmesg (click to see full content), I see:

[  213.051537] Call Trace:
[  213.051539]  <TASK>
[  213.051541]  ieee80211_do_stop+0x647/0x8b0 [mac80211]
[  213.051591]  ieee80211_stop+0x49/0x170 [mac80211]
[  213.051633]  __dev_close_many+0x8e/0xf0
[  213.051642]  dev_close_many+0x7b/0x120
[  213.051648]  ? ieee80211_handle_reconfig_failure+0x69/0x90 [mac80211]
[  213.051696]  dev_close+0x59/0x80
[  213.051701]  cfg80211_shutdown_all_interfaces+0x49/0xf0 [cfg80211]
[  213.051742]  wiphy_resume+0x94/0x150 [cfg80211]
[  213.051780]  ? wiphy_suspend+0x2b0/0x2b0 [cfg80211]
[  213.051840]  dpm_run_callback+0x4a/0x150
[  213.051845]  device_resume+0xa2/0x1f0
[  213.051848]  async_resume+0x19/0x30
[  213.051852]  async_run_entry_fn+0x30/0x130
[  213.051856]  process_one_work+0x1c7/0x380
[  213.051859]  worker_thread+0x4d/0x380
[  213.051862]  ? rescuer_thread+0x380/0x380
[  213.051864]  kthread+0xe9/0x110
[  213.051868]  ? kthread_complete_and_exit+0x20/0x20
[  213.051873]  ret_from_fork+0x22/0x30
[  213.051880]  </TASK>
[  213.051881] ---[ end trace 0000000000000000 ]---
[  213.051903] ------------[ cut here ]------------

...

[  213.052301] ieee80211 phy0: PM: dpm_run_callback(): wiphy_resume+0x0/0x150 [cfg80211] returns -11
[  213.052317] ieee80211 phy0: PM: failed to restore async: error -11

I don't know if this detail is usefull to fix my issue thinking .

Same issue

@stephane-klein
Copy link

My laptop: Thinkpad T14s Gen 3 AMD

Problems encountered:

* the resuming take 2min after entering my LUKS encryption password
* wifi not working after resuming

Some issues to follow:

@stephane-klein
Copy link

@dmy3k
Copy link

dmy3k commented Jun 18, 2023

The guide still is relevant for Fedora 38, however some improvements can be done.

With systemd v253 we have possibility to use suspend-then-hibernate that will sleep as long as possible. When battery enters low-charge state (5%) system will hibernate. This is beneficial for several use cases, e.g sleep between business hours, sleep on AC with closed lid, as this will avoid entering hibernate. Corresponding systemd pull requests #23895, #25374

However, SElinux module to support the above is not shipped with os, logging errors in journalctl

Jun 16 07:38:38 fedora audit[3100]: AVC avc:  denied  { read } for  pid=3100 comm="systemd-sleep" name="+power_supply:BAT0" dev="tmpfs" ino=1650 scontext=system_u:system_r:systemd_sleep_t:s0 tcontext=system_u:object_r:udev_var_run_t:s0 tclass=file permissive=0

Jun 16 21:26:09 fedora audit[71808]: AVC avc:  denied  { write } for  pid=71808 comm="systemd-sleep" name="systemd" dev="dm-0" ino=143877 scontext=system_u:system_r:systemd_sleep_t:s0 tcontext=system_u:object_r:init_var_lib_t:s0 tclass=dir permissive=0

We can update systemd_sleep module with audit2allow as shown in the guide above. Alternatively here is a gist to automate this

@tstoeckler
Copy link

Note that there's also a guide for Silverblue: https://discussion.fedoraproject.org/t/setup-hibernation-on-silverblue-kionite/78834/8

Not sure if anything mentioned there, for example the btrfs commands, could be used in the guide here, as well, but in any case I hadn't realized that I need to use /var/swap/swapfile as a swapfile path, which (presumably) is why I couldn't get this to work on Silverblue.

@00sapo
Copy link

00sapo commented Oct 12, 2023

Great gist! I managed to get it working with nvidia drivers using the script in this commnet. However, just systemctl enable nvidia-{suspend,resume,hibernate} worked for hibernation only, but not for suspend-then-hibernate.

I think we could set up a repo with a small script to ease the set up. Arguments should be the drive where the directory should be created and the size of if. --nvidia could be added as optional.

@obitbef
Copy link

obitbef commented Dec 6, 2023

Please note that comment. It seems that deactivating zram and activating swapfile on hibernate and vice versa is not required.

You do not need to deactivate zram, since the hibernation logic in systemd ignores zram devices anyway when looking for a swap device to hibernate into.

if you have a swap file anyway, then just activate it always, but make sure to give it a lower priority than your zram device, so that zram is always preferred, and the swap file is only used if the system is under pressure enough that zram didn#t work. That will give you overall better behaviour.

@enmyj
Copy link

enmyj commented Dec 9, 2023

I'm sure I'm wrong here but based on the above linked comment, it seems like a bunch of config is no longer required here. I'm on Fedora 39 and systemd version 254 and I'm able to hibernate with the following setup:

  • create the swap subvolume, add it to /etc/fstab with low priority, swapon /swap/swapfile
  • I kept the memory check overrides and the dracut config, not clear if that's still necessary
  • I updated the selinux script posted above to include permissions for efivars:
allow systemd_sleep_t swapfile_t:dir search;
allow systemd_sleep_t efivarfs_t:file { create setattr getattr ioctl open read write };
allow systemd_sleep_t efivarfs_t:dir { add_name create write };

So basically, I'm able to hibernate with minimal config in /etc/systemd/* and without any kernel arguments 👍

@mardoodmuh
Copy link

mardoodmuh commented Dec 27, 2023

Hi, i am using Fedora workstation 39.
I followed the instructions of the first comment and now my system is not booting.

IMG_20231227_110321

62f7aa3b-6a00-4efd-b0ac-9816cd0cce79

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment