KB Installs with the kbupdate module
Here's my 5 day journal of the work I was doing on KBUpdate. It was not initially written for a super technical audience, so please pardon non-specific words like "metadata". Also, these changes have not yet been merged to kbupdate repo, but will be soon.
The ultimate goal of this patching project is to easily view, download and install files from Microsoft's Patch repo. The goal of the installer I'm working on today is to allow the easy installation of patches on a multitude of remote systems from a central workstation in an offline network. This will prevent the requirement of interactive (RDP ->= Click ->= Click ->= Next) installs, saving countless hours across offline networks. Making patching easier and faster also increases the likelihood of proactive patching.
Installing Windows Patches remotely using PowerShell is challenging but not impossible. I wanted an easy way to:
- Install KB Updates, including Windows, SQL Server, .NET, Office and whatever else is in the Windows Update Catalog
- Provide support for all update file types: MSI, MSU, EXE, whatever else.
- Easily install to remote computers from one admin station
- Not reach out ot the internet for any information, as the installer must work on air-gapped networks
There are a number of security mechanisms that intentionally make this hard. Off the top of my head, these include (don't quote me):
- Installing updates can only be performed by the SYSTEM account
- There's no "built-in" way to do this in a straightforward manner but you can use
psexec
and you can createSchedule Tasks
to run as SYSTEM. Sincepsexec
isn't available on all systems, you'd have to copy it over then execute which isn't ideal. UsingScheduled Tasks
is my least favorite solution. - One option that's built into Windows (for many years now) is
Invoke-DscResource
. Thank you for the hints Matthew Hitchcock, Jess Pomfret and Kirill Kravtsov! This appears to initiate some process that is runs as SYSTEM. There's one potential downside in that the changes may not be recorded(?) with something in DSC called an LCM but I figure if someone is using DSC in their org, they won't use my module. - Installing remote updates from a file share can introduce Kerberos double-hop issues.
It was pretty easy getting Invoke-DscResource
going with MSU and MSI files, and very specific SQL EXE files. Getting other EXE files to install is problematic, as they don't work off the same predictable MSI subsystem.
Research suggests that, in order to install an EXE file elegantly (ie. without using psexec or scheduled tasks), some additional metadata is required for Invoke-DscResource
. Getting this metadata (specifically, a GUID) is challenging, but not impossible, using PowerShell.
- Once a sample KB file from Microsoft's KB respository was obtained, I searched for ways to find the GUID that I would need to install EXEs remotely and was able to extract the information using PowerShell, though nothing seemed overtly useful.
- Using the sample .NET SDK Update file, I installed and inspected not only the file's metadata, but also the resulting metadata that showed up in the Windows registry. Unfortunately, nothing aligned with GUID I was looking for.
- From there, I moved on to see how other products such as chocolately install files remotely and discovered through a source code review, that I could likely specify command line arguments in lieu of searching for the GUID within metadata
- After uninstalling the .NET SDK using the Windows GUI, I attempted to install the patch using command line a proof of concept:
./dotnet-sdk-5.0.404-win-x64_a943fac999a30b3eb83580112b793d37de0c0700.exe /install /quiet /notrestart
and this worked. Now to make it work withInvoke-DscResource
or directly from the command line while bypassing Kerberos concerns.
Today's challenge was building an elegant, approved solution for finding a package's GUID. This GUID is required for simplified remote install methods that play nice with Kerberos.
The easiest way to find this is to use 7zip to unzip the exe, but 7zip is not allowed on the network.
- Turns out
extract
is not good enough - it extracts a whole compiled file w/o the GUID lessmsi
doesn't work either, it cannot read an exeextrac32
can't read an exedark.exe
extracts and decompiles too much- Reading the source code from Carbon from webmd shows a use of
WindowsInstaller.Installer
, which also can't open an updated stored in exe format [System.IO.Compression.ZipFile]::OpenRead($Path)
doesn't work, as it considers the exe corrupted- I explored native ways to work with MSZIP compressed files but refused to work with bytes and dig through the algorithm's "piquant history"
- 7Zip isn't allowed but I'm tempted to triple check with the org to see if 7zip can't be used lol
- msiexec and using the /extract flag installs in addition to extracting contents that do not provide the GUID
- The PowerShell-compliant solution was finally found within the WIX Library, which is an open source project available from Microsoft. Just gotta load
Microsoft.Deployment.Compression.Cab.dll
andMicrosoft.Deployment.Compression.dll
.
After the WIX solution was found, I began to integrate it into the code for the installer but realized that I needed to test other EXEs within the Patch Repository. I downloaded twenty sample executables and explored their internal file structure. Handling different file structures will help ensure that the installer can handle as many patch types as possible without human intervention. So far, I've found one outlier and will work to ensure this installer type is supported as well.
Now that all known pre-requisites have been discovered, today's goal was to install the 5 different Windows Patch types (MSI, MSU, EXE, cab + SQL EXE) to a remote Windows server within a lab.
This will make installing patches to servers as easy as:
Get-ChildItem C:\updates | Install-KbUpdate -ComputerName server01, server02, server03
The above hypothetical command will install a directory of patches to remote servers.
- I successfully installed both an EXE and a SQL EXE on a remote Windows Server 2022 machine running SQL Server! Excellent! But the output was not correct.
- While attempting to fix the output, I realized that the approach I used to discover the GUID was valid (extracting XML to disk), but didn't overwrite metadata files if they already existed and this possiblity needed to be handled.
- Reading a stream of data in memory instead of writing the data to disk is preferable anyway, so I began to explore the
OpenText()
andOpenRead()
methods of the CabInfo class instead of writing to a temp directory usingUnpackFile()
- Both methods result in error
Cabinet file does not have the correct format
and there's not much about this on the web or in their documents. Because the cab's format is valid for extracting/unpacking the file and writing to disk but not reading directly, I'm leaning to this being a bug within the WIX library. Worst case, I'll return back to writing and checking.
- Continued to try a number of different ways to avoid writing to disk, like unpacking to stream, but nothing worked, so I accepted that I'd have to write a temporary file disk
- Next up, choosing a unique name for XML file, which I've decided will be
$filename.exe.xml
- Next, I need a universal way to find the specific files I need, which ended up being
GetFiles()
- Now that I know how I'll extract the internal XML files, I sorted the 20 samples into the the type of uniquely idenitifable XML attributes they presented
- Before I begin my targeteted extraction, I need to reorganize my code to accommodate this updated technique
- Oops, the code reorganization killed the command's ability to install a SQL Server patch
- Wait, how did it ever install SQL Server updates in the first place?
- Incredible! It's because I was accidentally reusing a GUID because of the
file exists
bug I ran into yesterday. Turns out, I don't even need the patch's GUID. I can fake it with a CIM-Compatible GUID such asDEADBEEF-80C6-41E6-A1B9-8BDB8A05027F
. Uncharted waters! Thanks /PowerShell/PSDscResources - Confirmed that even though I used a fake
DEADBEEF-80C6-41E6-A1B9-8BDB8A05027F
GUID, the package installs properly and knows its own GUID. TheDEADBEEF-80C6-41E6-A1B9-8BDB8A05027F
GUID appears to be used to check if something is already installed, so I do try my best to get the right GUID. - If something is installed that uses the fake guid, it fails nicely but a good GUID would have saved the time it took to attempt an install
- Next up, I will
- Try the previous techniques to get the real GUID which helps the installer fail faster when the patch already exists
- If no GUID has been found (which is true of files like SQL Server 2019 CU 1), I can use
Get-KbInstalledSoftware
to see if the patch is already installed. Then, if not, I can use the fabricated GUID. Very exciting.
- Tested to see if it was possible to speed up the query for Get-KbInstalledUpdate, but ultimately, getting just one takes as long as getting all kbs because it can only be filtered after the fact. To address this, a per-server cache of KBs will be used.
- Encountered and troubleshot the error "Cannot invoke the Invoke-DscResource cmdlet. The Invoke-DscResource cmdlet is in progress and must return before Invoke-DscResource can be invoked. Use -Force option if that is available to cancel the current operation." Unfortunately, I could not find a resolution other than to restart the server and rename the corrupted file it was repeatedly attempting to install. This appears to have cancelled the process, though a clean, repeatable resolution would have been preferred.
- Fixed remote module detection that caused modules to reinstall with each run, even if they already existed.
- Added auto-detection for the update Title, through by using
(Get-ChildItem $file).VersionInfo.ProductName
. This helps make the output human readable, as the adminsitrator doesn't have to rely on a filename or KB number without context. - Added retry for copying files after failures with unknown causes were encountered. Looked for details as to why the failure occurred but no information was provided.
- Added auto-skip for KBs which have already been installed
- Added auto-delete for installer files that have been copied to the remote computer
Thanks for reading! I'll keep this updated if anything exciting changes.