Skip to content

Instantly share code, notes, and snippets.

@XenoPanther
Forked from 0xbadfca11/00_README.md
Last active April 27, 2024 10:02
Show Gist options
  • Star 9 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save XenoPanther/15d8fad49fbd51c6bd946f2974084ef8 to your computer and use it in GitHub Desktop.
Save XenoPanther/15d8fad49fbd51c6bd946f2974084ef8 to your computer and use it in GitHub Desktop.
Windows ReFS versions

Version number is reported by fsutil fsinfo refsinfo, available on Windows 10 and Windows Server 2016.

ReFS 1.1

  • Version of formatted by Windows Server 2012.
  • Version 1.1 is used already in Windows Server 8 Beta. I have never seen version 1.0.
  • Can use and store alternate data streams, when mount on 8.1/2012 R2 or later.

ReFS 1.2

  • Version of formatted by Windows 8.1, Windows 10 v1507 to v1607, Windows Server 2012 R2, and when specified ReFSv1 on Windows Server 2016 or later.
  • Cannot use alternate data streams, when mount on 2012.

ReFS 7.2

  • Version that can be formatted with Windows 10 Build 9780

ReFS 9.2

  • Version that can be formatted with Windows 10 Technical Preview build 9841 to 9860 and Windows Server 2016 TP1 (It's not default). Could not mount in 10 9879 or 2016 TP2 and later.

ReFS 11.2

  • Version that can be formatted with Windows 10 Technical Preview build 9879 (It's not default). Could not mount in 9926 and later.

ReFS 12.2

  • Version that can be formatted with Windows 10 Technical Preview build 9926 (It's not default). Could not mount in 10041 and later.

ReFS 22.2

  • Version that can be formatted with Windows 10 Technical Preview build 10041 to 10049 (It's not default). Could not mount in 10061 and later.

ReFS 2.0

  • Version of formatted by Windows Server 2016 TP2/TP3.
  • Version that can be formatted with between Windows 10 Technical Preview build 10061 or later and earlier than 10130 (It's not default). Could not mount in Windows 10 Insider Preview build 10130 and later, or Windows Server 2016 TP4 and later.

ReFS v2 Overview
http://www.snia.org/sites/default/files/SDC15_presentations/file_sys/JRTipton_ReFS_v2.pdf
http://www.snia.org/events/storage-developer/presentations15#file_sys

ReFS 3.0

  • Version of formatted by Windows Server 2016 TP4/TP5.
  • Upgrade to 3.1 when writable mount from Windows Server 2016 RTM.

ReFS 3.1

  • Version of formatted by Windows Server 2016.

ReFS 3.2

  • Version of formatted by Windows 10 v1703.
  • Version that can be formatted with Windows 10 Insider Preview 15002 or later (It's not default at 15002) and Windows Server Insider Preview build 16237.
    • It became the default between after than 15002 and 15019 or earlier.

ReFS 3.3

ReFS 3.4

  • Version of formatted by Windows 10 Enterprise v1803, Windows Server 2019 and Windows Server version 1803.
  • Version of formatted by Windows 10 Enterprise Insider Preview build 17083 and Windows Server Insider Preview build 17079.

ReFS 3.5

  • Version of formatted by Windows 10 Enterprise Insider Preview build 19536 and Windows Server Insider Preview build 19551.
  • Added hardlink support if fresh formatted volume.
    • Can't use hardlink if upgraded from previous version.
  • Upgrade to 3.6 when writable mount from Windows 10 Insider Preview build 21292.

ReFS 3.6

  • Version of formatted by Windows 10 Enterprise Insider Preview build 21292 and Windows Server Insider Preview build 20282.
  • Upgrade to 3.7 when writable mount from Windows 10 Insider Preview build 21313.

ReFS 3.7

  • Version of formatted by Windows 11 Enterprise v21H2 and Windows Server 2022.
  • Version of formatted by Windows 10 Enterprise Insider Preview build 21313 and Windows Server Insider Preview build 20303.

ReFS 3.9

  • Version of formatted by Windows 11 Enterprise v22H2.
  • Version of formatted by Windows 11 Enterprise Insider Preview build 22598 and Windows Server Insider Preview build 25099.
  • Added post process compression with LZ4 & ZSTD, transparent decompression.

ReFS 3.10

  • Version formatted by Windows 11 Enterprise Insider build 25324 through 25997 and builds 26010

ReFS 3.12

  • Version formatted by Windows 11 Enterprise Insider build 26002. Build 26010 reverts back to 3.10

Mountability

ReFS\Windows 2012 8.1/2012 R2 10 v1507 2016 10 v1703 10 v1709 10 v1803/2019 11 v21H2/2022 11 v22H2 Canary Channel
1.1 Yes Yes1 Yes1 Yes1 Yes1 Yes1 Yes1 Yes1 Yes1
1.2 Yes Yes Yes Yes Yes Yes Yes Yes Yes
3.1 No No No Yes Yes2 Yes3 Yes4 Yes56 Yes76
3.2 No No Yes Yes3 Yes4 Yes56 Yes76
3.3 No No Yes Yes4 Yes56 Yes76
3.4 No Yes Yes56 Yes76
3.7 No Yes Yes8
3.9 No Yes
3.10 No No Yes9
3.12 No No Yes

License: CC BY

Footnotes

  1. "Volume "?:" was mounted in an older version of Windows. Some features may be lost." was recorded to event log when writable mount. I don't know what's been lost. 2 3 4 5 6 7 8

  2. Upgrade to 3.2 when writable mount.

  3. Upgrade to 3.3 when writable mount. 2

  4. Upgrade to 3.4 when writable mount. 2 3

  5. Upgrade to 3.7 when writable mount. 2 3 4

  6. Can't use hardlink. 2 3 4 5 6 7 8

  7. Upgrade to 3.9 when writable mount. Can't mount read-only. 2 3 4

  8. Upgrade to 3.9 when writable mount.

  9. Builds 25324 to 25997

@Karl-WE
Copy link

Karl-WE commented Mar 6, 2024

Thank you for continuing the great work! Much appreciated!

@nblog
Copy link

nblog commented Mar 7, 2024

I tested it directly after installing it using the original image, and the test report I got was slightly different from the above.
Here are the results of my tests:

WIN10 1709 / ReFS 3.4
WIN10 1809 / ReFS 3.4
WIN10 1909 / ReFS 3.4
WIN10 20H2 / ReFS 3.4
WIN10 21H1 / ReFS 3.4
WIN10 21H2 / ReFS 3.4
WIN10 22H2 / ReFS 3.4

WIN11 21H2 / ReFS 3.7
WIN11 22H2 / ReFS 3.10
WIN11 23H2 / ReFS 3.10

The above report is built on my quick test of the environment

@Karl-WE
Copy link

Karl-WE commented Mar 9, 2024

Windows 11 24H2 (currently still preview Channel) and Windows Server 2025 come with 3.14.
in my role as a Windows Server MVP I have kindly asked for an update to the official documentation, pointed to this fork and original work.

@YexuanXiao
Copy link

YexuanXiao commented Mar 14, 2024

Build 26080 supports booting the system from a ReFS3.14 volume and works fine, but it's WindowsRE does not recognize the ReFS3.14 volume.

2024/3/22 UPDATE: Build 26085 fixes the issue.

@XenoPanther
Copy link
Author

I'll get around to updating this, this weekend^^
Have been fairly busy

@Karl-WE
Copy link

Karl-WE commented Mar 15, 2024

^ thanks for the service! I will talk to MSFT if they can reveal more on the changes made in the respective versions

@Karl-WE
Copy link

Karl-WE commented Mar 28, 2024

Hi there it this is explanation why ReFS now supports boot of the OS

@CyberGonzo
Copy link

Perhaps this is not the place to ask, do let me know and forgive me if I clutter the flow with my Q.
I parse ReFS myself and have (from all my testing in the past) a decent working version. 1.x, 2.x and 3.x. But .. I neglected ReFS a bit and I haven't looked at it for years. My last tests were on v 3.7 and it worked fine. Yesterday, when I plugged in an old HDD (not sure what version of ReFS) my W11 system crashed horribly. It was a trigger to start looking into ReFS again. I formatted a partition with ReFS using an up to date W11 system and it appears to be version 3.10. But ... I can't seem to find the root anymore. My code appears to be broken for this version of ReFS. Frankly I did not expect data structures to change in a major version range, and I'm completely in the dark Are there people here who look at the data structures at a lower level ? I need a clue .. what changed between 3.7 and 3.10 that could explain decent working code to stop working entirely.

@Karl-WE
Copy link

Karl-WE commented Apr 2, 2024

Greetings @CyberGonzo I read your question twice but still require more information to provide an answer.

Not maintainer. I just can assume but I believe it would be good to move a technical question from this gist to techcommunity.microsoft.com, but that is on @XenoPanther to decide.
If not appropriate., I would like to encourage you to repost on techcommunity
Windows Server for IT Pro
or
Windows and provide a link to the question?

May I ask you to elaborate on your question:

"I can't seem to find the root anymore. My code appears to be broken for this version of ReFS"

Q1: What do you mean by "cannot find the root? You mean the drive become inaccessible, appearing as RAW in File Explorer

Issue:
you attached an external drive formatted with ReFS to your Windows 11 Enterprise
assuming as Windows 11 pro cannot format ReFS anymore.
your PC crashed.
you choose to reformat the drive with Windows 11 Enterprise. The drive was then healthy and working.

Q2: Are you looking into debugging the crash at all or is it wasted time?

"my code appears to be broken for this version of ReFS."

Q3: Could you tell more about your your code, at best posting it on techcommunity, too. If you don't want to post it publicly you can provide me a DM.
I will treat confidential when required.

What would be needed as a bare minimum is a synopsis of your code and the parsing process you describe, the use case and goal.
You can enter it in Edge > Copilot GPT-4 and "write a synopsis" and enter the code.
In addition, what does fail with your code with ReFS 3.10 (error code, expected behaviour, actual behaviour)

Looking forward to hearing from you!

@CyberGonzo
Copy link

Thank you @Karl-WE, for getting back to me. I admit the question is rather superficial and mainly opportunistic as I was probably hoping for a "yes, they added this or removed that or .. " which could potentially lead me to the culprit. The question was born out of late night desperation and frustration before I signed off. Meanwhile I am knee deep in the code again, trying to figure out if it's really something new or a corner case or ..

Thanks for the links. When I'm totally out of ideas after looking into the code, I might post there, but do people comment there on the deeper mechanics (data structures etc) ?

Q1: What do you mean by "cannot find the root? You mean the drive become inaccessible, appearing as RAW in File Explorer

I meant my software can't find the root anymore. Explorer finds it just fine, the issue is in my software.

Q2: Are you looking into debugging the crash at all or is it wasted time?

I have several ReFS partitions I now find that crash my W11 immediately. Scary how the W11 driver is handling ReFS it doesn't support (anymore - I assume). Only the Windows file system driver team can fix this I'm afraid / I assume.

Q3: Could you tell more about your your code, at best posting it on techcommunity, too.

I can't share the code, but my software, that contains the code, is IsoBuster. Add .com and you'll find it online ;)

What I have found so far is that problems start with the B-Tree parsing of the (what I call $CT) container files.
I identified an issue yesterday, which I will fix today, but it's not the whole story. There's more that is working differently, in particular the From Node, To Node type headers seem not to work as expected .. but we're really getting to the low level mechanics now.

@CyberGonzo
Copy link

I should add that I have a working version again. Sorry if I wasted anybody's time. It was a combination of an untested situation (which was not handled well in the code) and unexpected data in one of the headers. I fixed the former and now work around the latter.

@Karl-WE
Copy link

Karl-WE commented Apr 3, 2024

HI @CyberGonzo thanks for the reply and good to know you have addressed it somehow.

Do I understand correctly you are the maintainer of the said software?

I have several ReFS partitions that crash my W11 immediately. Scary how the W11 driver is managing ReFS it doesn't support (anymore - I assume). Only the Windows file system driver team can fix this I'm afraid / I assume.

It is hard to provide any support if you are a software developer I can try to connect you with capable folks that might want to investigate it once we have data and repro steps. Be prepared to raise a Microsoft support ticket, too, so we can keep it formal.
Without a SR and more information, it is hard to understand why you experience the crashes, this is not usually the case I am collaborating with ReFS on W11 every day. Thus, my interest into your pain in the first place.

@CyberGonzo
Copy link

Hi @Karl-WE, sure if you know of people who are interested in this, I'm certainly willing to provide more information, though I have only W11 machines at the moment and W11 crashes the moment Windows accesses the ReFS partition.
I am the developer of said software, yes.

@nblog
Copy link

nblog commented Apr 4, 2024

@CyberGonzo, As far as I know there is no relatively well-developed structure in the open source projects, and the unsound/refsprogs project is currently a little bit of resolution to the file\folder name of the project.

@CyberGonzo
Copy link

Thank you @nblog, but I was able to fix the issue and all is working well again. Thanks for the open source suggestion but upon reading its description I think my implementation is more advanced already anyway.

@CyberGonzo
Copy link

CyberGonzo commented Apr 5, 2024

As a test I wanted to see if I could produce refsutil output on a healthy refs volume, for the sake of investigation and possibly parsing.
I am baffled by what I got:

C:\Windows\System32>Refsutil salvage -QS w: m:\refstest -x -v
Microsoft ReFS Salvage [Version 10.0.11070]
Copyright (c) 2015 Microsoft Corp.

Local time: 4/5/2024 10:29:49

Option(s) specified: -v -x

ReFS version: 3.10
Error: The volume is an unsupported ReFS version. This utility supports versions up to 3.9. Volume is 3.10.
Error: Command failed.
Error: The volume does not contain a recognized file system. Please make sure that all required file system drivers are loaded and that the volume is not corrupt.

How is it possible that on my Windows 11 Pro version: 10.0.22631 Build: 22631 I can format a partition with ReFS (default settings) which then turns out to be version 3.10, but the refsutil version that is present on that same system cannot handle the (default) refs version !? That is mind boggling.

Any idea how I can update refsutil to v3.10 without messing my system up ?

@Karl-WE
Copy link

Karl-WE commented Apr 5, 2024

Yes, it is mindbloggling and I catched that some time ago. https://techcommunity.microsoft.com/t5/windows-server-insiders/b25951-refs-3-10-volume-became-raw-refsutil-not-compatible-with/m-p/3947990/highlight/true#M3139
So do understand your crashes, I would kindly ask you to open a SR with Microsoft Support (e.g Visual Studio Enterprise, or you could use Azure Dev/Test Support and using an Azure VM.
As soon you have the Ticket number come back to me and I will help you to manage to bring it to the ReFS people. They signalled interest into this. TYVM. Is this fine with you?

@Karl-WE
Copy link

Karl-WE commented Apr 5, 2024

Yes, it is mindbloggling and I catched that some time ago. https://techcommunity.microsoft.com/t5/windows-server-insiders/b25951-refs-3-10-volume-became-raw-refsutil-not-compatible-with/m-p/3947990/highlight/true#M3139
So do understand your crashes, I would kindly ask you to open a SR with Microsoft Support you have certainly access to support.
As soon you have the Ticket number come back to me and I will help you to manage to bring it to the ReFS people. They signalled.
If you can repro the crash on connection issue in a preview version of Windows 11, we might not require a SR.
I had this issue once but only for older versions of ReFS.
try to find out the ReFS version of the attached disk. It should survive a powershell > fsutil fsinfo refsinfo dl: command, hopefully

@CyberGonzo
Copy link

Hi @Karl-WE, I have never reported an issue with MS before and the methods you describe are all Latin to me.
The problem with my HDD and Images is that the moment I attach them to the system, the system crashes.
A while back I cloned an older ReFS version from an image to a partition (my software does that) and the moment Windows accessed the partition it blue-screen-crashed and would simply not boot anymore. I fixed it by booting the system with an older W10 boot DVD and by overwriting the partition with rubbish (using my software again).
I suppose I could boot the system again using that W10 DVD.

@Karl-WE
Copy link

Karl-WE commented Apr 5, 2024

Ok Gonzo, let us proceed offline. I got your DM!

@CyberGonzo
Copy link

CyberGonzo commented Apr 5, 2024

I booted in W10. I'm starting to feel confident W11 crashes because of ReFS 2.0. Turns out the HDD has 4 ReFS partitions. versions 1.2, 2.0 and 3.4. W10 mounted 1.2 and 3.4 correctly but failed to see 2.0. I think W11 takes it a step further and crashes on ReFS v 2.0.

Screenshot 2024-04-05 151647

@nblog
Copy link

nblog commented Apr 5, 2024

@CyberGonzo it is possible to try to analyze the DUMP file as ReFS 3.x (refs.sys) and ReFS 1-2 (refsv1.sys) are two different drivers,

@CyberGonzo
Copy link

It says both Could not read faulting driver name and
MODULE_NAME: ReFS IMAGE_NAME: ReFS.SYS FAILURE_BUCKET_ID: AV_R_INVALID_ReFS!CreateUnicodeStringDirectorKey
The latter suggesting the ReFS 3.x driver then

@CyberGonzo
Copy link

Just FYI and to complete my earlier findings. I cloned a small ReFS v 3.2 image to a HDD and W11 crashed immediately after mounting the disk. So it's not v 2.0 as I thought. It's also v 3.x. (PS. I used the publicly available image refs-v3_2.E01 found here: https://data.mendeley.com/datasets/9d7p6mbhpz/1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment