Bug in Cloud Filer / Placeholder / Reparse Point Disguising / OneDrive sync engine

onedrive-bug-report.md

Bug in Cloud Filer / Placeholder / Reparse Point Disguising / OneDrive sync engine

Starting from an unknown version of Windows 10, programs can no longer access dehydrated (online-only, placeholder) files with CreateFile(Ex) normally.

Consider the following excerpt:

// By default (if the program does not have a manifest, so it runs in Windows Vista mode),
// RtlQueryProcessPlaceholderCompatibilityMode returns PHCM_DISGUISE_PLACEHOLDER, and
// RtlQueryThreadPlaceholderCompatibilityMode returns PHCM_APPLICATION_DEFAULT.
FILE *fp = fopen("C:\\Users\\testuser\\OneDrive\\test.txt", "rb");
if (!fp)
{
  fputs("fopen failed.\n", stderr);
}
else
{
  if (fgetc(fp) == -1)
  {
    fputs("fgetc failed.\n", stderr);
  }
  else
  {
    fputs("Succeeded.\n", stderr);
  }
  fclose(fp);
}

And the following excerpt:

Get-Content -LiteralPath 'C:\Users\testuser\OneDrive\test.txt'

Environment for bug reproduction

• Windows 10 build 18363.1500, 64-bit, Education.
• OneDrive version 2021 build 21.062.0328.0001 (32 bit).
• We are connected to the Internet.
• The user syncs personal OneDrive at C:\Users\testuser\OneDrive, and has a file test.txt available at this root. This file is dehydrated (online-only, placeholder) before each case is tested.

Case 1

If the code excerpts are running as a non-administrator (including an administrator in admin-approval mode of UAC; unelevated):

• The C program fails with fgetc failed.
• The PowerShell script fails with permission denied.

Case 2

If the code excerpts are running as an administrator (elevated):

• The C program succeeds.
• The PowerShell script succeeds. (IMPORTANT: after running the C program, the file should be dehydrated before the PowerShell script is run; otherwise, the case for PowerShell will be in the wrong configuration that the file is already hydrated.)

Expected behavior

Both programs will just do fine. This held in some early version of Windows 10 (say version 1709).

Consequences of this bug

This makes a lot of applications fail to run when it tries to programmatically access files stored on OneDrive. Some typical scenarios:

• The user saves a document to OneDrive, and the application provides a list of recently opened documents, which when clicked causes the application to programmatically open the file. Most applications are not aware of placeholders and expect the files to be automatically hydrated whenever needed. If the user has dehydrated a file and tries to open it from the list of recently opened documents in the application, then the application will fail.
• The application asks the user to choose a folder, after which it processes files inside the folder. The access to files inside the picked folder is programmatic. If the folder resides in OneDrive and some of the files are dehydrated, the application will fail.
• Continuing the above example, the application operates a sparse subset of files inside the chosen folder (depending on the file name). When used on a OneDrive folder, it is expected that only the touched files are hydrated as necessary. This bug makes such operation impossible.

It seems that the shell handles this OK, so if the application asks the user to choose a file (not a folder) using common file dialog, it will run fine, as the common file dialog (shell) will hydrate the file upon selection.

References

• Cloud Filter API
• Compatibility with applications that use reparse points
• Evidence that CreateFile(Ex) worked in earlier versions of Windows 10:
  • Automatic file download notifications in Windows 10
  • arstechnica

@GeeLaw - Thanks a lot for bringing the issue to our attention.
I tried to reproduce the issue but couldn't succeed. Hence I have not been able to debug it so far. If you have one of these machines where you saw the issue, could you please confirm the following:

1. The error code.
2. If you see a reg key Computer\HKEY_CURRENT_USER\Software\Policies\Microsoft\CloudFiles\BlockedApps* on the machine ?

@abhijeet-gautam Thanks for your follow-up.

1. The error code is the generic "access denied" on ReadFile (not CreateFile), as reported by PowerShell.
2. The key exists, but it has no subkeys nor values.

For your reference, the following program can be used to show that it is reading the file that causes the error.

#include<cstdio>
#include<windows.h>

#define MY_FILE L"C:\\Users\\testuser\\OneDrive\\test.txt"

char buffer[128];

int main()
{
  /* Other possible arguments that do not affect reproduction:
  ** - dwDesiredAccess = GENERIC_READ | GENERIC_WRITE
  ** - dwShareMode = FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE
  ** - dwCreationDisposition = OPEN_EXISTING
  ** - dwFlagsAndAttributes = FILE_ATTRIBUTE_READONLY
  **/
  HANDLE hFile = CreateFileW(
    MY_FILE, /* lpFileName */
    GENERIC_READ, /* dwDesiredAccess */
    0, /* dwShareMode */
    NULL, /* lpSecurityAttributes */
    OPEN_EXISTING, /* dwCreationDisposition */
    FILE_ATTRIBUTE_NORMAL, /* dwFlagsAndAttributes */
    NULL /* hTemplateFile */
  );
  fprintf(stderr, "CreateFileW = %d\n", (int)GetLastError());
  if (hFile == INVALID_HANDLE_VALUE)
  {
    return 0;
  }
  DWORD bytesRead;
  if (!ReadFile(
    hFile, /* hFile */
    buffer, /* lpBuffer */
    1, /* nNumberOfBytesToRead */
    &bytesRead, /* lpNumberOfBytesRead */
    NULL /* lpOverlapped */
  ))
  {
    fprintf(stderr, "ReadFile = %d\n", (int)GetLastError());
  }
  CloseHandle(hFile);
  return 0;
}

Please pay special attention that the following conditions must be met:

• The file is dehydrated (online-only).
• The process accessing the file is unelevated (not running as administrator).

Thanks again @GeeLaw. I have a repro with another App only if the reg key exists. Could you please delete the regkey Computer\HKEY_CURRENT_USER\Software\Policies\Microsoft\CloudFiles and confirm if it resolves the issue ?

@abhijeet-gautam Thanks for the follow-up.

I tried:

1. Renaming HKCU\Software\Policies\Microsoft\CloudFiles\BlockedApps to BlockedApps1, and it does not fix the problem.
2. Renaming ...\CloudFiles to CloudFiles1, and it fixes the problem.

Now if I try accessing many dehydrated files, the popup of "automatic downloads" shows up, and offers me to block download and block app.

However, blocking app does not work properly. See below.

First, let me summarize the current behavior of ReadFile:

process → CloudFiles ↓	elevated	unelevated
exists	OK	Access to the path is denied
does not exist	OK	OK

Now, in OK scenarios, if a process triggers many automatic hydrations in a short time, Windows offers the user to cancel it.

If the user chooses "Cancel download", Windows offers two choices: "Cancel download" or "Block app".

• "Cancel download" is supposed to block the current operation, and the app subsequently can trigger more hydrations.
• "Block app" is supposed to block the application (EXE file) from future automatice hyration.

If the user chooses either of them, the current operation fails with "Access to the cloud file is denied", which is expected. However, if the user chooses "Block app", it will have no effect when the process is restarted, which is unexpected.

---

From what I experienced earlier, the expected behavior after "Block app" is that a key is created under HKCU\...\CloudFiles\BlockedApps, with values Enabled and ImagePath. It seems that Windows fails to create this key.

By using procmon, I think the cause of this problem is that the process triggering automatic hydration tries to open HKCU\...\CloudFiles with full access (instead of just read, which suffices for checking whether the automatic hyration should be allowed). This fails because HKCU\Software\Policies does not give the current user (if unelevated) full access, which is expected for enforcing the group policies.

For the same reason, explorer.exe fails to store the information in HKCU\...\CloudFiles\BlockedApps because it does not have sufficient permission to create subkeys.

Failed Workaround Even if I give the current user (even unelevated) full permission to HKCU\...\CloudFiles, the problem does go away, because the subkeys of BlockedApps have their own permissions (inheritance disabled). So if I block an app and unblock it using Settings, the apps will not be able to trigger hydration, because the triggering process tries to open the subkeys with full permission, which fails.

Suggested Fix 1 Move the information out of Policies. This is necessary to solve the problem for explorer.exe. Otherwise, the unelevated user has no way of storing this information. (Unless you want to use a service dedicated for this purpose, which can authenticate the requesting user and then write to the registry with permission from the service account.)

Suggested Fix 2 Do not open registry with excessive permissions (read is good for checking).

---

I should mention that having HKCU\...\CloudFiles is a natural condition --- it is caused by previous action in previous versions of Windows. Moreover, the permission setting on Policies is also a natural condition. So I consider this a bug, not an improper configuration caused by the user.

@GeeLaw - Thanks again for sharing the details. This is a known issue and I just wanted to make sure you are running into the same. The initial description didn't really confirm if it is the same. We are working on an apt resolution to this issue.

@GeeLaw - These details with the regkey etc. are very much appreciated. Thanks for your help on this :).