Skip to content

Instantly share code, notes, and snippets.

@ritchiecarroll
Last active February 22, 2024 02:14
Show Gist options
  • Save ritchiecarroll/6def10ed017d8cd4b7b00daffeccfa13 to your computer and use it in GitHub Desktop.
Save ritchiecarroll/6def10ed017d8cd4b7b00daffeccfa13 to your computer and use it in GitHub Desktop.
Manual Windows Partitions Recovery and Restore from Windows Image Backup

Manual Windows Partitions Recovery and Restore from Windows Image Backup

Steps below iterate how to manually recreate the disk partitions required by Windows and restore a Windows installation from a system image backup created with the "Backup and Restore (Windows 7)" feature, especially when the official steps fail.

These steps were tested with Windows 11, but should work with prior versions of Windows as well:

Preface: Before the "how's", here are the "why's":

I had my hand forced to be on Windows 11 because of being stuck on an expired insider build of Windows 10, my options were reinstalling Windows from scratch and reinstall / reconfigure all applications or buy a TPM 2.0 chip and try to update to Windows 11. I opted for the "easy" path of the TPM 2.0 install and upgrade Windows to version 11, especially given the time I had invested in getting my development workstation just the way I want it. Seemed to go well, except I was still stuck in the "Windows 11 Insider Preview" loop until the next "major" release - you must be patient with build releases to get out of the insider program - waiting for a major release. This didn't seem too bad; I had been on the Windows 10 insider program for years with very few issues - certainly never had any update that "blocked" day-to-day work.

Everything was fine for a month or so, but as luck would have it, the latest Windows 11 Insider Preview 22523.1000 refused to install. The error was vague, the dreaded 0x80070422 type Windows error that truly means nothing useful. The thought of being "stuck" on an insider build again was not appealing - being stuck on any insider build means when it expires, you are back to the singular option of "reinstall Windows". <Day 1> I tried several things to "fix" this issue with no luck. Finally, I booted into a Windows recovery environment and tried some basic Windows recovery options -- FYI, I felt comfortable performing these types of actions since I had a recent backup from the "Backup and Restore" option found in control panel which included a system image backup of my entire C: drive. I initially tried the restore option that sounded obvious, the "roll back the latest feature release" on Windows. That was the beginning of the end, it "uninstalled" fine - but my system would not even boot now. Note to self: do not try to "roll back" anything when you are an insider build. Best I could tell the system thought it was in the middle of an upgrade to the latest insider preview when I tried to roll it back, leaving the system in some kind of "very" poor state.

Next obvious thing, try to get back to where you were using your handy backup! Just so you know, even though Windows happily includes the "Backup and Restore (Windows 7)" feature in control panel, I discovered that it is apparently considered deprecated, perhaps that is why they go out of the way to label it Windows 7. I suppose the only reason it is still there is because there is really nothing else readily available to replace the "system image backup" feature that is built into the latest Windows versions –- sorry, having backups of my files using the "File History" feature is not the same, this was an OS issue. So, to continue my diatribe, I tried many times (in many different ways) to use the built-in recovery options to restore my latest system image. The UI recovery tools could find and detect the system image on my D: drive just fine, but the tools said I needed to boot off a recovery DVD to properly restore. Interestingly, I was booted off a USB, which to me should have been fine, but, against my instincts, I decided to "follow the prompts": (1) "Insert a disc to burn a Windows Recovery DVD", OK, now (2) "Insert Windows Install media into DVD", yeah, into the same location where you are trying to burn a DVD (WTF) -- reboot, rinse, tweak, repeat. <Day 2> Went to another computer to burn a DVD with an ISO downloaded for Windows 11 Insider Preview 22523.1000; well, it won't "fit" onto a 4GB DVD, by just a few MBs. OK, I need a dual-layer DVD to give me 8GB; of course I don't have one of those and I am not even sure my DVD burner will even burn dual layers (who even uses DVDs these days?). <Day 3> Went to Office Max, bought the very last pack of dual-layer DVDs; these were probably sitting there for the last 5 years. Back home again, sure enough, my DVD burner (in a second computer) will not properly burn a dual-layer DVD; it acted like it would, but not before I waste three DVDs trying. Went to another, newer computer, in the house that had a Blue Ray burner - this one worked, finally! And... wouldn't you know it, the DVD reader in my broken computer refused to read the dual-layer DVD, ugh.

With that trial exhausted, time for other options. <Day 4> One Googled site basically said, "try expanding your EFI boot drive to give Windows more space"; the arguments made for this did sound convincing, however, this ultimately ended in utter disaster with all SSD partitions (where my C: drive / Windows install resided) gone in the end -- don't ask, but it involved booting into GParted, some user errors, and lots of cursing. Now I also needed to get my SSD back in order so that I could get back to my original step of trying to restore from the saved WindowSystemImage. At this point, I was sufficiently nervous about my "single" backup - so from the Windows Recovery command prompt (who would soon be my best friend), I made a backup of my backup. Luckily, I had plenty of space and two extra drives, so I copied my WindowSystemImage from my D: drive to what used to be my I: drive; my drives were now inconveniently labeled such that I: was now C: (although D: was still D:), all because my SSD was now just empty space.

<Days 5-10> What follows are my summarized "directions" to what ultimately restored my computer as gleaned from many days of Google research, lots of trial and error, multiple attempts and almost giving up several times. I provide here my gratitude and thanks 🖖 to those before me who had encountered similar issues, providing thoughts on resolution. FYI, a derivation of steps found in this article was the ultimate key to success. I condensed the resolution here in step-by-step form to provide (hopefully) all the detail needed to restore a system from a "Backup and Restore" system image, all in one spot – this being for me, next time, as much as the next desperate soul that happens upon this.

Restore Windows Partitions

FYI, if you found yourself here to simply to restore a Windows installation from a system image backup created with the "Backup and Restore (Windows 7)" feature, I recommend moving on to the next section. You should only need to follow these steps to restore Windows partitions if you have totally hosed your hard drive.

To get all the required Windows partitions back into proper shape from scratch, you must make DISKPART your friend. To manually make a bootable Windows drive, and all required partitions, here are the steps:

  1. Run DISKPART from a Windows Recovery command prompt
  2. Run list disk to get a feel for which disks are available - you need to find your original Windows disk, e.g., an SSD
  3. You can also run list volume to show currently active volumes, hopefully your other drives (like those with the system image backup) are still around and visible
  4. In my case, since what was originally my I: drive was now labeled C:, I needed to relabel the drive - C: drive needs to be for Windows install
  5. Disk 0, in my case, is what I want to be my I: drive, so I followed these steps:
  6. select disk 0
  7. assign letter=I
  8. Now I have a D: drive and an I: drive, like my original configuration, and drive letter C is now available
  9. Disk 3, in my case, is my SSD where I want my C: drive, so I select it:
  10. select disk 3
  11. Now for the "put things back in order" part. Note: before running next command, run list disk and double-check that the correct disk is selected.
  12. clean <- This fully wipes everything from the disk and does not ask "are you sure", yikes! Scary command
  13. convert gpt <- This converts the disk to use Guid Partition Tables (as required by a standard UEFI Windows boot)
  14. create partition efi size=100 <- This creates the EFI boot partition, yes 100MB is plenty, even for Windows 11
  15. format quick fs=fat32 label=System <- Formats the EFI partition as FAT32 (as required by a standard UEFI Windows boot)
  16. assign letter=S <- We give it drive letter S for now, will need this later when we associate the boot partition with Windows installation
  17. create partition msr size=16 <- This creates the Microsoft reserved partition (as required by a standard UEFI Windows boot)
  18. create partition primary <- This creates the partition that will hold Windows installation (takes up remainder of available disk space)
  19. shrink minimum=500 <- Free up 500MB at the end of the disk to hold the Windows recovery partition
  20. format quick fs=ntfs label=Windows <- Formats the Windows partition as NTFS
  21. assign letter=C <- Finally, a C: drive back on the SSD!
  22. create partition primary <- This creates the Windows recovery partition on the last free 500MB of space
  23. format quick fs=ntfs label=Recovery <- Formats the Windows recovery partition as NTFS
  24. assign letter=R <- Assigns the Windows partition as drive letter R (use this later to put back the needed recovery files)
  25. set id=de94bba4-06d1-4d40-a16a-bfd50179d6ac <- This is the specially defined Guid for the Windows recovery partition
  26. gpt attributes=0x8000000000000001 <- This is the specially defined GPT attribute for the Windows recovery partition
  27. exit <- We are done with DISKPART for now

Now we should have the following:

  1. a 100MB empty EFI boot/system partition (S:)
  2. a 16MB reserved partition
  3. an empty partition (C:) that will hold our Windows installation sized at all remaining available space
  4. a 500MB empty Windows recovery partition (R:)

Technically the EFI drive should now be "bootable", but there are no boot files on the drive yet and it is not associated with any Windows installation. We can only "fix" this when a valid Windows installation is available on the C: drive - so the next steps detail how to restore a Windows install from a WindowsImageBackup. The last step will setup the EFI drive boot files and associate it with the Windows install that gets restored on the C: drive.

Restore from WindowsImageBackup

All image operations from the command prompt use the DISM command. This is a powerful, if not often troublesome, tool for managing Windows disk images. Currently, the DISM tool can only use a .WIM file, i.e., a Windows Image File, to apply an image to a hard drive. The system image backup created with the "Backup and Restore (Windows 7)" feature is literally a .VHDX file, i.e., a virtual hard disk, exactly like the ones used in virtual machines. The goal then, is to convert from a .VHDX file to a .WIM file, but there is no direct feature for doing this. However, DISM can be used to mount the VHDX hard disk into a local folder.

Once a VHDX hard disk is mounted, all files are available, then you can use DISM to create a new WIM file. Sounds simple enough, but this is where I ran into trouble with Access Denied and other file access errors, reason being: Windows rights still apply. The place where I consistently ran into issues was my own local C:\Users\ritchie folder. For any files / folders where I had adjusted rights, I had issues.

You must remember, you are in a Windows recovery boot environment as some admin-type user, but file rights still apply. I had trouble creating an image from my mounted VHDX hard disk for any files where I manually adjusted file rights, and, with OneDrive folders where I assume file rights had been adjusted on my behalf. The primary issue here is that the process of creating a WIM file from a mounted VHDX hard disk takes many hours (around six for me), and sometimes it would be "hours" into the process before it crashed, unrecoverably.

To fix the issue I literally "deleted" the offending files from the mounted VHDX hard disk folder. Having a full backup of your WindowSystemImage is highly recommended before you go off deleing files, but the mounting process for a VHDX is interesting in the fact that no changes, including deletions, will be committed to the hard disk until you request it when unmounting -- there is a feature on unmount which will allow you to "discard" all changes. Discarding deletions is super handy for this operation since you can remove offending files from the mounted VHDX hard disk folder so that the WIM can be successfully created, then undo the changes upon completion. For example, you can remove all potentially problematic files in the "Documents" and "OneDrive" folders from the mounted VHDX hard disk such that these files do not end up in the WIM, and consequently will not be restored. However, you can discard changes to the VHDX when the Windows Image File creation is complete –- then upon successful Windows restoration, you can remount your VHDX hard disk from within your active Windows session, using your original user rights, and restore the removed files, original file rights and all.

Mount VHDX hard disk containing WindowsImageBackup

  1. Create a folder to mount the virtual disk, e.g., D:\Mount
  2. If it does not exist, also create a temporary folder, e.g., D:\Temp
  3. Locate the VHDX file with the system image, typically in a folder like WindowsImageBackup\{SYSTEMNAME}\Backup {YYYY-MM-DD} {HHmmss}\ where the file name is a Guid with a ".vhdx" extension
  4. Note that there may be multiple ".vhdx" files in this folder, but the largest one is the one that should be mounted -- it will also likely be the only file that will successfully mount
  5. Run the following command, replacing the /ImageFile: with the located VHDX file:
DISM /Mount-Image /ImageFile:"D:\WindowsImageBackup\RITCHIE-HOME\Backup 2021-12-21 070010\00985097-961c-471c-8a13-a55230708d3e.vhdx" /Index:1 /MountDir:D:\Mount\ /ScratchDir:D:\Temp /LogPath:D:\Temp\dism-mount-log.txt

Note that the /ScratchDir parameter is important, it needs to be on a drive with plenty of space. Otherwise, this defaults to a location on the Windows recovery environment, like the USB boot disk, which may run out of space.

Mounting a drive will should go quickly. If there are errors, check the log file. Even in this Windows recovery environment, you can still launch notepad.exe from command prompt.

Once mounted, you should be able to navigate (i.e., cd) into the D:\Mount folder from the command prompt and peruse all your backed up system files and get warm fuzzies that all is not lost.

Capture VHDX WindowsImageBackup to a Windows Image File

This next step is a crucial one, it is where the mounted VHDX gets converted to a WIM. This is also the most painful, prone to errors, and slow step – good luck!

BTW: You might think now that the VHDX is mounted, you should be able to just copy the files back to the C: drive. I tried this as well, but with no luck. Perhaps you will have better luck than me, but the /Capture-Image and/or /Apply-Image process is doing something, perhaps rights related, that makes Windows look like Windows. Regardless, as painful as this step is, once you have a good WIM file, the /Apply-Image does work.

  1. Create a folder to hold the captured Windows Image File, e.g., D:\Capture (if you have a separate HDD, it may be optimal use a different drive for the captured WIM from the source VHDX)
  2. I would recommend going ahead and deleting files in the D:\Mount\Users\{myuser}\Documents and any OneDrive folders in D:\Mount\Users\{myuser}\ that might cause you headaches, retries here are slow and annoying
  3. Remember you can always restore any removed files manually when your Windows environment is back online later. That said, be careful to keep up with what was deleted and try not to remove anything that will cause Windows to have startup issues
  4. You may need to repeat the deletes for other users as well. Regardless, be prepared to execute this operation several times should the capture fail. Each time the capture fails, you will need to remove offending files
  5. Always check the log after a failure to determine which file is causing the issue (remember you can run notepad.exe from the command prompt to check the log file)
  6. Run the following command, specifying the target WIM file to capture in the /ImageFile: parameter:
DISM /Capture-Image /ImageFile:D:\Capture\Backup.wim /CaptureDir:D:\Mount /Name:Win11Backup /Description:"Windows 11 Backup" /Compress:none /Verify /ScratchDir:D:\Temp /LogPath:D:\Temp\dism-capture-log.txt

I ran this tool with no compression since I had disk space to spare and figured this would be faster. If you are low on disk space, see the other compression options.

Prepare yourself, this operation is extremely slow, be patient. On my system it took 20 minutes before it even showed any percent complete feedback -- I was worried it was not doing anything and stopped it unnecessarily several times. Because I needed to keep deleting files when this process failed, the process was taking about 6-hours, and I was just leaving it to run, this step took me many days to finally complete.

If you are successful with this operation, you can proceed to next step.

Apply Captured Windows Image File to C: Drive

Hopefully, on your system, this step is as simple as running the DISM /Apply-Image command shown below. If you do run into trouble, this section details steps that may help bail you out. Regardless, this step tends to go quickly, at least as compared to the last one anyway.

  1. If your C: drive is not freshly formatted from restoring Windows partitions, or you are retrying this step, format the drive now to clear things out: FORMAT C: /Q /FS:NTFS
  2. For the first pass, start with step 10 below, then only come back to step 3 if needed
  3. You might be surprised to find that you can also have Access Denied type errors during this application phase that did not show up during the capture phase, go figure
  4. Should there be errors during this process, there is good news: you do not have to fully repeat the many hours-long last capture/conversion process -- instead, you can directly mount your WIM file and remove the offending files
  5. Always check the log after a failure to determine which file is causing the issue then you can mount the WIM file
  6. The process for mounting the WIM file is identical to mounting the VHDX file, just use a different mount folder and specify the D:\Capture\Backup.wim for the /ImageFile: parameter
  7. Like before, be careful to keep up with what was deleted and try not to remove anything that will cause Windows to have startup issues
  8. After deleting files in the mounted WIM, make sure to unmount the WIM file (see next section) after the deletes, with one caveat:
  9. Unlike using the /Discard option like you do on the VHDX, you instead need to use the /Commit option to save the deletes, this makes sure the files are not in the WIM before attempting the application process again
  10. Run the following command, specifying the captured WIM file:
DISM /Apply-Image /ImageFile:D:\Capture\Backup.wim /Index:1 /ApplyDir:C:\ /Verify /ScratchDir:D:\Temp /LogPath:D:\Temp\dism-apply-log.txt

Unmount VHDX hard disk containing WindowsImageBackup

  1. When the capture/apply steps are complete, do not forget to unmount the virtual hard disk that contains the WindowsImageBackup
  2. If you made deletes in the mounted VHDX, these will be discarded now (per the /Discard parameter) so that these deletions can be restored after Windows comes back online
  3. Run the following command, specifying the original mount folder:
DISM /Unmount-Image /MountDir:D:\Mount\ /Discard /ScratchDir:D:\Temp /LogPath:D:\Temp\dism-unmount-log.txt

Configure the EFI Boot Partition

Note: This is not a necessary step if you only restored the WindowsImageBackup.

If you followed the steps to restore Windows partitions, you will now need to fix the EFI boot partition.

Once the C: drive containing the Windows installation has been restored, run the following command (this assumes EFI drive is accessible as drive S:):

bcdboot C:\Windows /s S:

Reboot Your Computer

If everything went well, your computer should come back to life. Make sure to restore any files you manually removed to get to this point.

You can re-run the "mounting the VHDX file" steps from a Windows command prompt to get the original files back online.

Then you can run something like the robocopy command to restore deleted files, for example:

robocopy D:\Mount\Users\ritchie\Documents C:\Users\ritchie\Documents /ZB /E

Enjoy the peace, whilst you can.

@ritchiecarroll
Copy link
Author

PS) For those who followed the story in the preface and may be curious, yes, after my system was restored back to the point of my latest system image backup, the latest Windows 11 Insider Preview v22523.1000 curiously (thankfully) installed just fine...

@DucatiDan
Copy link

What a way to spend the holiday! 😒

@bzd314
Copy link

bzd314 commented May 24, 2023

Thank you for this! It saved me at least a few days off my own VHDX misadventure, which started when a power outage killed the HDD on my ten-year-old desktop. (Who knew -- an abrupt power outage can damage a hard drive, even if you use a surge protector.) I wanted to restore my old image to a new drive, and your guide was invaluable.

In the interest of paying it forward, here are some extra wrinkles I encountered, and how I solved them. Apologies if there are some newbie mistakes in here!

-Mounting the VHDX yielded an unformatted ("RAW" format) volume -- somehow, my image had been corrupted.

-After much research, it turns out chkdsk works on virtual drives as well as on physical ones, so it (eventually, after several hours) restored the volume to NTFS.

-Like you it seems, the capture to WIM kept getting "access denied" errors, the source of which was only visible when going to the end of the dism log file. Unlike you, these didn't come from the Documents or OneDrive folders.

-First it was cygwin files named with Windows reserved words, so I deleted the entire cygwin folder. Note that File Explorer also refuses to delete something named with a reserved word, so I had to use cmd with "rd \.\J:\Mount\cygwin64 /S /Q"

-Next, some Microsoft libraries ("msedge.dll" in two different Edge installs I hadn't asked for) didn't have the right security permissions.

-I needed to go into r-click->properties->security->advanced, change the owner (I chose "everyone", so the target PC wouldn't have the same problem), then add a permission entry (again for "everyone" ... security schma-curity, amirite?) with full access, and apply for all child objects.

-I assumed I would hit many more permissions problems like that, so I needed to fix permissions to EVERYTHING.

-However, even though I could apply that change recursively for ordinary folders, if I did it for the mounted volume/directory, it didn't apply to sub-folders. Rather than repeat the process 20-30 different times to get everything in the root of the image (i.e., the old C: drive, which was now mounted as J:\Mount), I moved almost everything there into a "CTemp" folder, applied security settings recursively to THAT, then moved everything back.

-CAUTION: You shouldn't mess around with the hidden link "C:\Documents and Settings". It's a junction to C:\Users, not a normal shortcut; importantly, the path is hard-coded and doesn't update when the image is on a different drive letter. I still don't fully understand how, but a blind cut-and-paste started moving files from the C: drive of my support PC into the image on the external (J:\Mount). I was lucky to catch the problem before it moved more than some of my music library.

-As with you, running dism /apply-image did not go smoothly at first. It ran for 30 minutes and got to 12% before giving yet another "access denied" error. This time the log file said it was for stuff in the Users\Ben\AppData\Local\lxss folder, which is for the Windows Subsystem for Linux.

-Your above walkthrough warned this was possible, and suggested mounting the WIM rather than the VHDX to fix the problem, as it would be faster. However, the /mount-image command ALSO hit "access denied" on lxss.

-An old post online told me that 7-zip can open the WIM and edit it, but [very, very long story short] this isn't workable because 7-zip breaks all symlinks and junctions in the WIM when it saves the edited version, and there were more than a hundred such links in Program Files. I had to delete lxss in the mounted VHDX and run /capture-image again (which took ~20 hours).

-Only then, at long last, did /apply-image work.

-Although I was then able to boot into Windows on the target PC normally, things were wonky -- the start menu wouldn't open and right-click did nothing for many objects. (Maybe this was related to the initial corruption that chkdsk had to fix?) I eventually did an in-place upgrade/repair Windows reinstall from a downloaded ISO, and that FINALLY finished the saga. (Well, except for setting up imaging software better than obsolete Windows 7 technology, but that's another story.)

@ritchiecarroll
Copy link
Author

@bzd314 glad you found this valuable, and thanks so much for sharing your experiences as well - hopefully these details will help others in similar situations in the future...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment