Skip to content

Instantly share code, notes, and snippets.

@XenoPanther
Forked from 0xbadfca11/00_README.md
Last active December 19, 2024 13:02
Show Gist options
  • Save XenoPanther/15d8fad49fbd51c6bd946f2974084ef8 to your computer and use it in GitHub Desktop.
Save XenoPanther/15d8fad49fbd51c6bd946f2974084ef8 to your computer and use it in GitHub Desktop.
Windows ReFS versions

Version number is reported by fsutil fsinfo refsinfo, available on Windows 10 and Windows Server 2016.

ReFS 1.1

  • Version of formatted by Windows Server 2012.
  • Version 1.1 is used already in Windows Server 8 Beta. I have never seen version 1.0.
  • Can use and store alternate data streams, when mount on 8.1/2012 R2 or later.

ReFS 1.2

  • Version of formatted by Windows 8.1, Windows 10 v1507 to v1607, Windows Server 2012 R2, and when specified ReFSv1 on Windows Server 2016 or later.
  • Cannot use alternate data streams, when mount on 2012.

ReFS 7.2

  • Version that can be formatted with Windows 10 Build 9780

ReFS 9.2

  • Version that can be formatted with Windows 10 Technical Preview build 9841 to 9860 and Windows Server 2016 TP1 (It's not default). Could not mount in 10 9879 or 2016 TP2 and later.

ReFS 11.2

  • Version that can be formatted with Windows 10 Technical Preview build 9879 (It's not default). Could not mount in 9926 and later.

ReFS 12.2

  • Version that can be formatted with Windows 10 Technical Preview build 9926 (It's not default). Could not mount in 10041 and later.

ReFS 22.2

  • Version that can be formatted with Windows 10 Technical Preview build 10041 to 10049 (It's not default). Could not mount in 10061 and later.

ReFS 2.0

  • Version of formatted by Windows Server 2016 TP2/TP3.
  • Version that can be formatted with between Windows 10 Technical Preview build 10061 or later and earlier than 10130 (It's not default). Could not mount in Windows 10 Insider Preview build 10130 and later, or Windows Server 2016 TP4 and later.

ReFS v2 Overview
http://www.snia.org/sites/default/files/SDC15_presentations/file_sys/JRTipton_ReFS_v2.pdf
http://www.snia.org/events/storage-developer/presentations15#file_sys

ReFS 3.0

  • Version of formatted by Windows Server 2016 TP4/TP5.
  • Upgrade to 3.1 when writable mount from Windows Server 2016 RTM.

ReFS 3.1

  • Version of formatted by Windows Server 2016.

ReFS 3.2

  • Version of formatted by Windows 10 v1703.
  • Version that can be formatted with Windows 10 Insider Preview 15002 or later (It's not default at 15002) and Windows Server Insider Preview build 16237.
    • It became the default between after than 15002 and 15019 or earlier.

ReFS 3.3

ReFS 3.4

  • Version of formatted by Windows 10 Enterprise v1803, Windows Server 2019 and Windows Server version 1803.
  • Version of formatted by Windows 10 Enterprise Insider Preview build 17083 and Windows Server Insider Preview build 17079.

ReFS 3.5

  • Version of formatted by Windows 10 Enterprise Insider Preview build 19536 and Windows Server Insider Preview build 19551.
  • Added hardlink support if fresh formatted volume.
    • Can't use hardlink if upgraded from previous version.
  • Upgrade to 3.6 when writable mount from Windows 10 Insider Preview build 21292.

ReFS 3.6

  • Version of formatted by Windows 10 Enterprise Insider Preview build 21292 and Windows Server Insider Preview build 20282.
  • Upgrade to 3.7 when writable mount from Windows 10 Insider Preview build 21313.

ReFS 3.7

  • Version of formatted by Windows 11 Enterprise v21H2 and Windows Server 2022.
  • Version of formatted by Windows 10 Enterprise Insider Preview build 21313 and Windows Server Insider Preview build 20303.

ReFS 3.9

  • Version of formatted by Windows 11 Enterprise v22H2.
  • Version of formatted by Windows 11 Enterprise Insider Preview build 22598 and Windows Server Insider Preview build 25099.
  • Added post process compression with LZ4 & ZSTD, transparent decompression.

ReFS 3.10

  • Version formatted by Windows 11 Enterprise Insider build 25324 through 25997 and builds 26010

ReFS 3.12

  • Version formatted by Windows 11 Enterprise Insider build 26002. Build 26010 reverts back to 3.10

Mountability

ReFS\Windows 2012 8.1/2012 R2 10 v1507 2016 10 v1703 10 v1709 10 v1803/2019 11 v21H2/2022 11 v22H2 Canary Channel
1.1 Yes Yes1 Yes1 Yes1 Yes1 Yes1 Yes1 Yes1 Yes1
1.2 Yes Yes Yes Yes Yes Yes Yes Yes Yes
3.1 No No No Yes Yes2 Yes3 Yes4 Yes56 Yes76
3.2 No No Yes Yes3 Yes4 Yes56 Yes76
3.3 No No Yes Yes4 Yes56 Yes76
3.4 No Yes Yes56 Yes76
3.7 No Yes Yes8
3.9 No Yes
3.10 No No Yes9
3.12 No No Yes

License: CC BY

Footnotes

  1. "Volume "?:" was mounted in an older version of Windows. Some features may be lost." was recorded to event log when writable mount. I don't know what's been lost. 2 3 4 5 6 7 8

  2. Upgrade to 3.2 when writable mount.

  3. Upgrade to 3.3 when writable mount. 2

  4. Upgrade to 3.4 when writable mount. 2 3

  5. Upgrade to 3.7 when writable mount. 2 3 4

  6. Can't use hardlink. 2 3 4 5 6 7 8

  7. Upgrade to 3.9 when writable mount. Can't mount read-only. 2 3 4

  8. Upgrade to 3.9 when writable mount.

  9. Builds 25324 to 25997

@CyberGonzo
Copy link

CyberGonzo commented Apr 5, 2024

I booted in W10. I'm starting to feel confident W11 crashes because of ReFS 2.0. Turns out the HDD has 4 ReFS partitions. versions 1.2, 2.0 and 3.4. W10 mounted 1.2 and 3.4 correctly but failed to see 2.0. I think W11 takes it a step further and crashes on ReFS v 2.0.

Screenshot 2024-04-05 151647

@nblog
Copy link

nblog commented Apr 5, 2024

@CyberGonzo it is possible to try to analyze the DUMP file as ReFS 3.x (refs.sys) and ReFS 1-2 (refsv1.sys) are two different drivers,

@CyberGonzo
Copy link

It says both Could not read faulting driver name and
MODULE_NAME: ReFS IMAGE_NAME: ReFS.SYS FAILURE_BUCKET_ID: AV_R_INVALID_ReFS!CreateUnicodeStringDirectorKey
The latter suggesting the ReFS 3.x driver then

@CyberGonzo
Copy link

Just FYI and to complete my earlier findings. I cloned a small ReFS v 3.2 image to a HDD and W11 crashed immediately after mounting the disk. So it's not v 2.0 as I thought. It's also v 3.x. (PS. I used the publicly available image refs-v3_2.E01 found here: https://data.mendeley.com/datasets/9d7p6mbhpz/1

@HotCakeX
Copy link

Supports Encrypted File System is now available in the build 24H2 release preview channel with ReFS v3.14

@sprog0
Copy link

sprog0 commented May 31, 2024

How do I start the compression process?
I successfully ran refsutil compression /c /f ZSTD /e 22 e:
But the compression statistic refsutil compression /q e: shows me only zeros.
Windows Server 2025 10.0.26100.560

@YexuanXiao
Copy link

YexuanXiao commented Jun 1, 2024

If Windows is installed on an ReFS partition, it may not pass OOBE. This issue has been present since at least build 26080, and it also exists in builds 26095, 26100 (.1, .560), and 26200. I haven’t tested this on more version, but I believe the problem persists in all versions from the past six months. bypassnro is not the cause of the failure, but it results in different outcomes. Specifically, users may experience the following symptoms:

UI freezing when filling out security questions.
‘OOBELOCAL’ error after completing security questions.
Spinning indefinitely after logging in with a Microsoft account.

Initially, the issue occurred on my Surface Book 3. However, today it happened on an ASUS laptop using the unmodified Microsoft released 26100.560 image. Symptoms disappear after turning off SecureBoot on Asus laptop.

Therefore, even though Microsoft has officially announced that Windows Server supports using ReFS as the boot file system, there are still some compatibility issues.

@YexuanXiao
Copy link

Build 27686:

PS C:\Users\Bizwen> fsutil fsinfo refsinfo D:
REFS Volume Serial Number :                0xUUUUUUUUUUUUUUUU
REFS Volume Version :                      3.14
REFS Driver Maximum Supported Version :    3.15
Number Sectors :                           0xUUUUUUUUUUUUUUUU
Total Clusters :                           0xUUUUUUUUUUUUUUUU
Free Clusters  :                           0xUUUUUUUUUUUUUUUU
Total Reserved :                           0xUUUUUUUUUUUUUUUU
Bytes Per Sector  :                        512
Bytes Per Physical Sector :                4096
Bytes Per Cluster :                        4096
Fast Tier Data Fill Percentage :           0.0%
Slow Tier Data Fill Percentage :           0.0%
Fast Tier to Slow Tier Rate (Clusters/s) : 0
Metadata Checksum Type :                   CHECKSUM_TYPE_CRC64
Data Checksum Type :                       CHECKSUM_TYPE_NONE

Unlike previous versions, ReFS 3.14 does not automatically upgrade to ReFS 3.15.

@YexuanXiao
Copy link

As of Build 27686, refsutil salvage still does not work. refsutil salvage only supports ReFS 3.9 and earlier. Does this mean that the tool is deprecated and not maintained?

@Karl-WE
Copy link

Karl-WE commented Sep 25, 2024

Hi everyone, thanks for keeping this gist alive and updated.
Had the same question as you @YexuanXiao stated here

I have asked Microsoft and received the following answers to share:

Is it planned to update ReFSutil to salvage 3.9+ version?
Yes, we are working on updating refsutil salvage to handle the latest format. The updates have to be backported to the appropriate release
pipelines, which takes time, please be patient.

Is the reported behaviour intended that ReFS will not autoupgrade from 3.14 to 3.15?
3.14 is Ge/WS2025 ReFS version. 3.15 is for future, there’s no way to get to 3.15 publicly.

Hope this helps!

@Karl-WE
Copy link

Karl-WE commented Sep 25, 2024

Hi everyone, thanks for keeping this gist alive and updated.
Had the same question as you @YexuanXiao stated here
I have asked around and received the following answers I may share:

Is it planned to update ReFSutil to salvage 3.9+ version?
Yes, we are working on updating refsutil salvage to handle the latest format. The updates have to be backported to the appropriate release
pipelines, which takes time, please be patient.

Is the reported behaviour intended that ReFS will not autoupgrade from 3.14 to 3.15?
3.14 is Ge/WS2025 ReFS version. 3.15 is for future, there’s no way to get to 3.15 publicly.

@Colin-Y0470
Copy link

Build 27723:
C:\Users\ThinkPad>sudo fsutil fsinfo refsinfo e:
REFS Volume Serial Number : 0x847623a07623924a
REFS Volume Version : 3.14
REFS Driver Maximum Supported Version : 3.15
Number Sectors : 0x0000000003fe0000
Total Clusters : 0x00000000007fc000
Free Clusters : 0x00000000007a2ed8
Total Reserved : 0x0000000000025048
Bytes Per Sector : 512
Bytes Per Physical Sector : 4096
Bytes Per Cluster : 4096
Fast Tier Data Fill Percentage : 0.0%
Slow Tier Data Fill Percentage : 0.0%
Fast Tier to Slow Tier Rate (Clusters/s) : 0
Metadata Checksum Type : CHECKSUM_TYPE_CRC64
Data Checksum Type : CHECKSUM_TYPE_NONE
ReFS v3.15 is still not available now.

@Karl-WE
Copy link

Karl-WE commented Oct 29, 2024

Have filed a PR to address that "additional information" will link to this gist, instead of the abandoned one. MicrosoftDocs/windowsserverdocs#7960

@swph
Copy link

swph commented Nov 16, 2024

I realize this is probably the wrong forum (and I'm going to check directly with MS, too): does anyone know of a method to do an in-place upgrade of a v3.9 refs storage space to v3.14? update-storagepool isn't doing it, and i did verify via fsutil that my space is 'at 3.9' but can 'support 3.14'. (I know i can create a new space and move stuff, but i don't have enough free space to do that and the migration would take literal weeks due to the size of the space and slowness of a parity space on spindle drives.)

@Karl-WE
Copy link

Karl-WE commented Nov 16, 2024

What was the host OS version before in-place OS upgrade
What's the current version?

Is this CSV from SAN or S2D?

@swph
Copy link

swph commented Nov 16, 2024

Hi karl! I'm using windows 11 pro for workstations, just upgraded from 23H2 to 24H2. It's just a home server with 8 disks in a storage pool, multiple storage spaces in the pool. I've validated that i can create a new space and format refs and that the new space is at the 3.14 version.

@YexuanXiao
Copy link

I realize this is probably the wrong forum (and I'm going to check directly with MS, too): does anyone know of a method to do an in-place upgrade of a v3.9 refs storage space to v3.14? update-storagepool isn't doing it, and i did verify via fsutil that my space is 'at 3.9' but can 'support 3.14'. (I know i can create a new space and move stuff, but i don't have enough free space to do that and the migration would take literal weeks due to the size of the space and slowness of a parity space on spindle drives.)

Typically, ReFS will automatically update to the highest version supported by Windows. However, there may be some limitations and changes. You can try installing an older version of Windows (it doesn't need to install on the target hard drive) and then gradually upgrade the target ReFS partition. Additionally, the following registry key affects the automatic upgrade of ReFS:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\FileSystem]
"RefsDisableVolumeUpgrade"=dword:00000000

Make sure the value of RefsDisableVolumeUpgrade is set to 0.

@swph
Copy link

swph commented Nov 16, 2024

Hi YexuanXiao! I figured the upgrade would be automatic, too, and I'll check that reg key. My storage spaces are running a bit low on space, showing "low capacity, add X drives" and I wondered if that could have held up any auto upgrade.

Edit: checked for the key and it doesn't even exist, searched entire registry to be certain.

@YexuanXiao
Copy link

Hi YexuanXiao! I figured the upgrade would be automatic, too, and I'll check that reg key. My storage spaces are running a bit low on space, showing "low capacity, add X drives" and I wondered if that could have held up any auto upgrade.

Edit: checked for the key and it doesn't even exist, searched entire registry to be certain.

If the key does not exist, it will automatically upgrade. I believe that insufficient remaining space in the partition may prevent the upgrade; you might need to reserve at least 2% of contiguous space.

@Karl-WE
Copy link

Karl-WE commented Nov 16, 2024

@YexuanXiao is this documented in learn.microsoft.com?

@YexuanXiao
Copy link

@YexuanXiao is this documented in learn.microsoft.com?

No, it is merely a general estimate based on my experience. ReFS requires 1%-2% of space to store metadata. I believe that during the update, the metadata might need to be regenerated and temporarily coexist with the old metadata.

@swph
Copy link

swph commented Nov 23, 2024

Well i've tried a number of things but it still refuses to upgrade.

Added 4x6TB drives, pool no longer shows "low capacity" message.

Optimize-volume -retrim
Optimize-volume -slabconsolidate
Optimize-volume -defrag (this one did not finish...would take days)
Extended space to have over 5% free
Multiple reboots

Any other ideas on how to force the upgrade?

Here's the fsinfo:

PS C:\WINDOWS\system32> fsutil fsinfo refsinfo p:
REFS Volume Serial Number :                0xe272022672020051
REFS Volume Version :                      3.9
REFS Driver Maximum Supported Version :    3.14
Number Sectors :                           0x00000003a4cf4000
Total Clusters :                           0x000000003a4cf400
Free Clusters  :                           0x0000000002b766ba
Total Reserved :                           0x0000000000080458
Bytes Per Sector  :                        4096
Bytes Per Physical Sector :                4096
Bytes Per Cluster :                        65536
Fast Tier Data Fill Percentage :           0.0%
Slow Tier Data Fill Percentage :           0.0%
Fast Tier to Slow Tier Rate (Clusters/s) : 0
Metadata Checksum Type :                   CHECKSUM_TYPE_CRC64
Data Checksum Type :                       CHECKSUM_TYPE_CRC64

@Karl-WE
Copy link

Karl-WE commented Dec 13, 2024

@XenoPanther how can we edit the table? W11 24H2, WS 2025 and Azure Local (AzL) 22H2 and 23H2 are missing

target:
AzL 22H2 = WS 2022
AzL 23H2 = WS 2025 = W11 24H2

@mdealer
Copy link

mdealer commented Dec 16, 2024

Coming over from the source gist, so 23H2 corrupted my ReFS volumes, but kept the file structure and file contents on disk. The files got "disconnected" from the actual data on disk. Any ideas how to connect them back? I am already exporting the most important files using refsutil and so far 99.9999% are intact, just not accessible by path ("file not found" error on every existing file but not on new ones). Version reported is 3.4. I run W11 24H2 now.

Tools that I have run so far do not report any errors, but I am no ReFS expert to know all of them.

@Karl-WE
Copy link

Karl-WE commented Dec 18, 2024

Hi @mdealer with no promises.

Could you describe what's the former setting and today's I would like to repro and then escalate it.

@mdealer
Copy link

mdealer commented Dec 19, 2024

Hi @mdealer with no promises.

Could you describe what's the former setting and today's I would like to repro and then escalate it.

I want to understand what went wrong and prevent this from happening for anyone else and find potential easy solutions, if any. I don't need to actually recover the partitions per se. refsutil will do fine for me.

I was running W10 22H2 Pro for Workstations. I don't know with which ReFS version I started with, but at the moment refsutil reports v9.4.

System disk: 1TB NVMe NTFS
CPU: Threadripper 2950x
RAM: 64GB

The problematic ReFS partitions were using AMD RAID:

  • RAID 0 with 4x NVMe disks 512GB each with around 20% free
  • RAID 1 with 2x HDDs 14TB each with around 65% free

What I roughly did:

  1. I waited years for W11 to get stable as I was already severely burned by XP->Vista upgrade.
  2. On day X, I made a backup of the system drive as I thought "ReFS will surely survive, it is named resilient after all"
  3. Then I reenabled the TPM module (disabled it so that I don't get nagged about upgrades) and was able to download and install W11 23H2.
  4. The installation took kinda long, but I saw no errors, the system booted fine.
  5. I started hearing disk access on the HDDs instantly after boot. Checking Task Manager showed 30MB/s read 1MB/s write non-stop.
  6. The system barely responded from Desktop, it appeared as if I am unable to open anything, but it just took more than 1min to do anything, like open an explorer window or Task Manager.
  7. AMD RAID displayed no errors, everything was normal in its control panel, but it showed 100K IOP/s.
  8. I started checking, disks were shown as RAW in Disk Management and the System process was doing the IO, there were the typical errors in event logs as reported from other users.
  9. I kind of panicked because I didn't understand how the System is not recognizing ReFS but writing to it, I am a noob in this particular area of Windows related to device mounting and file system upgrades.
  10. I checked for more updates and there were some, so I tried to install them. They took forever to install but succeeded. I think I let the computer run for an hour or so for this, then restarted. The restart went fine, but maybe I was not watching.
  11. The partitions were still shown as RAW and System was writing to them. Eventually the system stopped writing to the SSD partition but it kept restarting the process after OS restart I think. Both partitions got corrupted the same way.
  12. Installing more driver & co updates took forever, the machine was barely usable, the disks were already written to, so I decided to force abort this and put the disks Offline in disk management as I thought no way it can take this long to upgrade metadata on RAID 0 SSDs and wanted to avoid actual data loss on the HDDs. The machine became usable.
  13. Then after Googling I found out that 24H2 added support for ReFS. I couldn't believe my eyes, how did Microsoft nag me about W11 all this time, and basically ignored my ReFS partitions?!?
  14. The update was not offered to me. I had to force it in the registry. It installed fine without errors.
  15. Partitions became "readable", no disk activity. I thought all is fine. Two days later I tried to open a file and was greeted by "File not found".
  16. I am now slowly recovering the files using refsutil (need to procure disks) and so far I see no obvious problems after recovering around 1TB of data with around 40K files.

Let me know if you need more. I understand that somewhere it might be this platform specific, the Threadripper has problems with too slow internal CPU bus that slows down literally everything when accessing PCIe lanes like the RAID 0.

@swph
Copy link

swph commented Dec 19, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment