Skip to content

Instantly share code, notes, and snippets.

@kreeuwijk
Last active July 17, 2020 13:39
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kreeuwijk/1647a823739257fdcf8a7ff0188f26d7 to your computer and use it in GitHub Desktop.
Save kreeuwijk/1647a823739257fdcf8a7ff0188f26d7 to your computer and use it in GitHub Desktop.
#Provides automated patch management
class profile::patch_mgmt_win (
Array $blacklist = [],
Array $whitelist = [],
Optional[Hash] $patch_window = {
range => '01:00 - 04:00',
weekday => 'Sunday',
repeat => 3
}
) {
include os_patching
class { 'wsus_client':
server_url => 'http://wsus.example.com:8530',
target_group => 'AutoApproval',
enable_status_server => true,
auto_install_minor_updates => false,
auto_update_option => 'NotifyOnly',
detection_frequency_hours => 22
}
if $facts['os_patching'] {
$updatescan = $facts['os_patching']['missing_update_kbs']
}
else {
$updatescan = []
}
if $whitelist.count > 0 {
$updates = $updatescan.filter |$item| { $item in $whitelist }
} elsif $blacklist.count > 0 {
$updates = $updatescan.filter |$item| { !($item in $blacklist) }
} else {
$updates = $updatescan
}
schedule { 'patch_window':
* => $patch_window
}
if $facts['os_patching']['reboots']['reboot_required'] == true {
Windows_updates::Kb {
require => Reboot['patch_window_reboot']
}
notify { 'Reboot pending, rebooting node...':
schedule => 'patch_window',
notify => Reboot['patch_window_reboot']
}
}
reboot { 'patch_window_reboot':
apply => 'finished',
schedule => 'patch_window'
}
$updates.each | $kb | {
windows_updates::kb { $kb:
ensure => 'present',
maintwindow => 'patch_window'
}
}
}
@mike406
Copy link

mike406 commented Jul 14, 2020

Hello! I've been implementing my own patching routing based off of your code here, but I think I may have spotted an issue but I am not sure why it happens. When I run a puppet agent -t, the run hangs indefinitely with the below output

Notice: Reboot pending, rebooting node...
Notice: /Stage[main]/Windows_patch_mgmt/Notify[Reboot pending, rebooting node...]/message: defined 'message' as 'Reboot pending, rebooting node...'
Info: /Stage[main]/Windows_patch_mgmt/Notify[Reboot pending, rebooting node...]: Scheduling refresh of Reboot[patch_window_reboot]
Notice: Scheduling system reboot with message: "Puppet is rebooting the computer"
Notice: /Stage[main]/Windows_patch_mgmt/Reboot[patch_window_reboot]: Triggered 'refresh' from 1 event

Basically it never finishes so the reboot never occurs. If I switch the reboot apply => 'finished' to apply => 'immediately' it will restart the computer but I cannot figure out where it is hanging when it is set to 'finished'. I am wondering if it is because of this:

               Windows_updates::Kb {
                        require => Reboot['patch_window_reboot']
                }

My thoughts are that since this makes the windows_updates::kb wait for the reboot, the run essentially freezes because in my mind you can't wait for the reboot, while the reboot is also set to wait for the run to finish because that will make it wait forever. But I could be wrong.

Here is my full code for reference:

class windows_patch_mgmt (
        # Default class params.
        Array $blacklist = [],
        Optional[Hash] $patch_window = {
                range   => '01:00 - 04:00',
                weekday => 'Sunday',
                repeat  => 3
        }
) {
        include os_patching

        # Build array of missing KB numbers
        if $facts['os_patching'] {
                $updatescan = $facts['os_patching']['missing_update_kbs']
        }
        else {
                $updatescan = []
        }

        # Filter out updates based on blacklist
        if $blacklist.count > 0 {
                $updates = $updatescan.filter |$item| { !($item in $blacklist) }
        }
        else {
                $updates = $updatescan
        }

        # Create schedule resource
        schedule { 'patch_window':
                * => $patch_window
        }

        # Check for pending reboot and do it if necessary before installing any patches
        if $facts['os_patching']['reboots']['reboot_required'] == true {
                Windows_updates::Kb {
                        require => Reboot['patch_window_reboot']
                }

                notify { 'Reboot pending, rebooting node...':
                        schedule        => 'patch_window',
                        notify          => Reboot['patch_window_reboot']
                }
        }

        # Create reboot resource
        reboot { 'patch_window_reboot':
                apply           => 'finished',
                schedule        => 'patch_window'
        }

        # Do the patches
        $updates.each | $kb | {
                windows_updates::kb { $kb:
                        ensure          => 'present',
                        maintwindow     => 'patch_window'
                }
        }
}

@kreeuwijk
Copy link
Author

kreeuwijk commented Jul 15, 2020

Yes I think there's a problem in the logic as the windows_updates::kb resources and the reboot resource have no relationship. I've fixed all of this in the newly published patching_as_code module on the Puppet Forge. I would recommend switching over to that!

@mike406
Copy link

mike406 commented Jul 15, 2020

It actually turns out the hanging is caused by a weird Powershell bug with the "Get-WUHistory" command that the windows_updates module relies on. Documented here: noma4i/puppet-windows_updates#12

Also thank you for linking the module. I still have a question with how the reboot functions though. I see that in patchday.pp you you have this code here:

    if $facts[$patch_fact]['reboots']['reboot_required'] == true and $_reboot {
      Windows_updates::Kb {
        require => Reboot['Patching as Code - Patch Reboot']
      }
      notify { 'Found pending reboot, performing reboot before patching...':
        schedule => 'Patching as Code - Patch Window',
        notify   => Reboot['Patching as Code - Patch Reboot']
      }
    }

However the reboot resource is set with apply => 'finished' so the pending reboot is not happening before updates try to install. I tested it out today and windows_updates::kb still attempts to install if reboot_required is true and if there are $updates in the array. Would you be able to test this? I am thinking the apply => 'finished' might need to be changed to 'immediately' so that the require in the Windows_updates::Kb block will fire off immediately if there is a pending reboot instead of waiting.

Here is output I ran on a node, you'll see how it notifies the reboot, but it then also attempts to start installing updates. The reboot doesn't happen until after the updates are done.

Notice: Found pending reboot, performing reboot before patching...
Notice: /Stage[main]/Windows_patch_mgmt/Notify[Found pending reboot, performing reboot before patching...]/message: defined 'message' as 'Found pending reboot, performing reboot before patching...'
Info: /Stage[main]/Windows_patch_mgmt/Notify[Found pending reboot, performing reboot before patching...]: Scheduling refresh of Reboot[Patching as Code - Patch Window]
Notice: Scheduling system reboot with message: "Puppet is rebooting the computer"
Notice: /Stage[main]/Windows_patch_mgmt/Reboot[Patching as Code - Patch Window]: Triggered 'refresh' from 1 event
Notice: /Stage[main]/Windows_patch_mgmt/Windows_updates::Kb[KB3172729]/Exec[Install KB3172729]/returns:

@kreeuwijk
Copy link
Author

Hmmm, interesting. I guess when the reboot resource has triggered, this satisfies the require condition for the windows_updates::kb resources and they start installing while the machine counts down its 1 minute reboot delay. I'll update the code to reboot immediately for the reboot_required scenario.

@kreeuwijk
Copy link
Author

kreeuwijk commented Jul 16, 2020

@mike406 can you try the updated 0.2.0 version of the module? This now correctly handles pending reboots, and provides additional capabilities around pre/post patching commands, and pre-reboot commands.

@mike406
Copy link

mike406 commented Jul 16, 2020

This appears to be working!

Edit - a follow up:
I ran the patching process on a couple hundred Windows PC's at our various remote offices and noticed the following behavior for computers that had the "reboot_required = True" fact. Some computers repeatedly reboot - this is because the os_patching scheduled task that runs after reboot is not able to update the fact quick enough before the puppet agent run starts, which also happens at boot.

So these two essentially fight each other for who finishes first. For computers that have slower link speeds or are at remote locations, there is a high chance the os_patching fact for the reboot_required will not update before the puppet agent run starts, causing multiple pending reboots to fire. A classic "stale data" problem...To get around this one could set the puppet service on Windows machines to be Automatic (Delayed) instead of Automatic, but there is still a chance it could still be stale data even then.

https://forge.puppet.com/puppetlabs/reboot#complete-any-pending-reboots-before-installing-a-package
I think using the reboot module's "pending" feature may provide a more accurate way to check for a pending reboot than to check the os_patching fact that may or may not be stale data. For my environment I think I am going to backport some of your new code and combine it with your original example, but utilize the reboot module itself to check for pending reboots.

This is how I have my reboots set up. I'm still new to Puppet so I hope I did it right. My goal is to check for pending reboots with the require, and then notify as well at the end of the patching run to reboot the computer:

        reboot { 'pending_reboot':
                when    => 'pending',
                schedule    => 'patch_window'
        }

        reboot { 'reboot_after_updates':
                apply           => 'finished',
                schedule        => 'patch_window'
        }

        $updates.each | $kb | {
                windows_updates::kb { $kb:
                        ensure          => 'present',
                        maintwindow     => 'patch_window',
                        require         => Reboot['pending_reboot'],
                        notify          => Reboot['reboot_after_updates']
                }
        }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment