Skip to content

Instantly share code, notes, and snippets.

@EliahKagan
Last active June 26, 2024 01:51
Show Gist options
  • Save EliahKagan/513a4ad57654f417b47380d2e3ea4793 to your computer and use it in GitHub Desktop.
Save EliahKagan/513a4ad57654f417b47380d2e3ea4793 to your computer and use it in GitHub Desktop.
gitoxide bug report draft - symlinks to PermissionDenied targets
PermissionDenied checking Windows symlink target is misinterpreted as collision

Background

Checking out files into a working tree skips any individual files that already exist, reporting them correctly as collisions. This happens when overwriting is prohibited, as is the case in gix clone. For example, if a separate application creates files inside the working tree gix clone is checking out, and such a file would be overwritten in checked out, gix clone declines to check out that specific file and reports it as a collision. This behavior is correct and should not be changed.

Like other operating systems that support symbolic links, Windows permits a symbolic link to point to a location that is inaccessible to the user creating the symbolic link, including to a location where, due to permissions, traversal to the location will fail due to permissions. The user will not be able to deference that symbolic link other than to inspect the target path, not even to check it's target's existence and metadata, but the symlink itself is no problem. Unlike other operating systems that support symbolic links, Windows symbolic links come in two varieties: file and directory symbolic links. For this reason, gitoxide attempts to check the metadata of the symlink's target.

The problem

The problem is that, when attempting to check the metadata of the symlink's target fails with a PermissionDenied error, this is wrongly misinterpreted as a collision, even though there is no collision because the error happens in reading what the index specifies is the symlink's target, not when creating a new file. A wrong or unusual target of a symlink is never a real collision, because checkout does not create or mutate anything at the target.

This is what it looks like to use gix clone to clone a repository with such a symlink:

C:\Users\ek\src> gix clone git@github.com:EliahKagan/symlink-to-inaccessible.git
 20:53:39 indexing done 19.0 objects in 0.00s (60.9k objects/s)
 20:53:39 decompressing done 3.6KB in 0.00s (11.2MB/s)
 20:53:39     Resolving done 19.0 objects in 0.05s (377.0 objects/s)
 20:53:39      Decoding done 3.6KB in 0.05s (70.9KB/s)
 20:53:39 writing index file done 1.6KB in 0.00s (7.4MB/s)
 20:53:39  create index file done 19.0 objects in 0.06s (343.0 objects/s)
 20:53:39          read pack done 2.5KB in 0.09s (27.1KB/s)
 20:53:39           checkout done 3.0 files in 0.00s (2.4k files/s)
 20:53:39            writing done 1.4KB in 0.00s (1.1MB/s)
HEAD:refs/remotes/origin/HEAD (implicit)
        77d5afef8ef5ff6f3af10ab900cd1a08e18fc1cc HEAD symref-target:refs/heads/main -> refs/remotes/origin/HEAD [new]
+refs/heads/*:refs/remotes/origin/*
        77d5afef8ef5ff6f3af10ab900cd1a08e18fc1cc refs/heads/main -> refs/remotes/origin/main [new]
symlink: collision (PermissionDenied)
Error: One or more errors occurred - checkout is incomplete: encountered 1 collision(s)

The repository is created, but due to the symlink wrongly being detected as a collision, no symlink is created there:

C:\Users\ek\src> ls symlink-to-inaccessible

    Directory: C:\Users\ek\src\symlink-to-inaccessible

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d----           6/25/2024  8:53 PM                .git
-a---           6/25/2024  8:53 PM            617 COPYING
-a---           6/25/2024  8:53 PM            826 README.md

Relationship to #1420

This is related to #1420. As there, the error occurs when attempting to access metadata on the symlink's target, which is only done on Windows, so this bug does not affect non-Windows systems, and which is only done when actually attempting to create symbolic links, so this bug does not affect uses of gix clone where core.symlinks has been forced to false (though, as there, #1353 affects this).

Depending on how #1420 is fixed, the fix may automatically take care of this as well, by making it so no PermissionDenied error can ever be propagated upward from a position where it doesn't represent a real collision. I had originally planned to report this as part of #1420. However, on reflection I decided these are separate bugs, because:

  • Other ways of fixing that bug would not necessarily fix this, and vice versa.
  • This bug has the additional wrong behavior of reporting a non-collision condition as a collision.
  • This bug has the mitigating factor that it does not cause an entire clone to be canceled and the partially checked out working tree to be deleted (though the reason it doesn't do that is that it is wrongly treated as a collision).
  • This bug is less straightforward to write a regression test for in the test suite. It can definitely be done, though. Whether it needs to be done, and whether it needs to be done before a change that fixes the bug is accepted, might depend on how this bug is fixed. If it is fixed together with #1420 then the tests for that might be considered enough. Otherwise, it probably needs its own tests.

Symbolic links to locations to which the application cannot traverse, including when the error is PermissionDenied, should be created, unless a specific rationale is known for declining to do so. Such a rational must overcome the following factors:

  • Windows, like other operating systems, generally allows this.
  • Git creates them (as detailed below).
  • Dangling symlinks are created, and this is not appreciably different.
  • Dangling symlinks are created, and they can become this scenario if the target is created with restrictive permissions.
  • Non-dangling symlinks are created, and they can become this scenario if permissions are changed, or if the referred-to files are removed (making the symlinks dangle) and later replaced with files with restrictive permissions.
  • This is not fundamentally different from a symlink that sometimes dangles because the volume it is on is disconnected, not mounted, or otherwise unavailable.
  • Allowing this, as Git does, is considerably simpler than not allowing it.

As for dangling symlinks, these can be created as file symlinks. As in dangling symlinks, this is imperfect, but not inherently different from the situation of dangling symlinks where the target is created. In addition, there may be existing conditions where a file is hidden from the process rather than access being denied explicitly, in which case that is already treated as dangling.

If for some reason that is not to be done, another aspect of the expected behavior is that these are not collisions and should not be reported as such.

Git creates the symlinks. It creates them as file symlinks, even when the inaccessible targets are directories, since it cannot know they are directories and it falls back to creating file symlinks when the target type cannot be discerned:

C:\Users\ek\src> rm -r -fo symlink-to-inaccessible
C:\Users\ek\src> git clone git@github.com:EliahKagan/symlink-to-inaccessible.git
Cloning into 'symlink-to-inaccessible'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 19 (delta 0), reused 19 (delta 0), pack-reused 0
Receiving objects: 100% (19/19), done.
C:\Users\ek\src> cmd /c dir symlink-to-inaccessible
 Volume in drive C is OS
 Volume Serial Number is B203-10FB

 Directory of C:\Users\ek\src\symlink-to-inaccessible

06/25/2024  09:25 PM    <DIR>          .
06/25/2024  09:25 PM    <DIR>          ..
06/25/2024  09:25 PM               617 COPYING
06/25/2024  09:25 PM               826 README.md
06/25/2024  09:25 PM    <SYMLINK>      symlink [\Program Files\WindowsApps\MutableBackup]
               3 File(s)          1,443 bytes
               2 Dir(s)  84,863,729,664 bytes free

The referred-to MutableBackup entry is actually a directory, but the git process has no way of knowing that because it is not allowed to access metadata on that item or list the contents of its parent directory. So Git has created a file symlink.

This can be seen because the dir builtin in cmd.exe shows <SYMLINK>, whereas for a directory symlink it would show <SYMLINKD>.

Details on that particular path and why I used it for testing are included below.

1. Create a repository with a symlink into an inaccessible location

Whether such a location suitable for testing already exists or has to be created may vary by system.

On current desktop versions of Windows, the Microsoft Store keeps its Store applications in the WindowsApps subdirectory of the Program Files directory, which has restrictive permissions such that even administrators cannot list the directory's contents or access information on specific known subdirectory, at least if UAC is enabled and elevation has not been performed. (On server versions of Windows, the Microsoft Store is absent.)

The specific paths vary across systems. I decided to use the drive-relative path to a known subdirectory of WindowsApps. Although I actually performed more steps because I tried some other locations and ways of specifying the path--and although it is fairly straightforward to create such a repository in a native Windows system--I created the repository on an Ubuntu system (WSL is sufficient) in a manner similar to this, which is sufficient:

git init symlink-to-inaccessible
cd symlink-to-inaccessible
ln -s '\Program Files\WindowsApps\MutableBackup' symlink
git add .
git commit -m 'Initial commit'

This is a symlink to the MutableBackup subdirectory of WindowsApps, which users cannot typically access, if the repository is cloned on the system drive (or otherwise on the same drive that contains a Program Files directory that contains WindowsApps, in the rare case that this is different).

I made the test repository this way for convenience, so that the steps to verify this bug are simple in most cases, once the repository is already available. This can alternatively be tested by creating a new directory tree and setting restrictive ACL permissions.

The repository is available on GitHub. The following steps assume it is used and will have to be modified otherwise. The output shown reflects the presence of additional files in the repository, such as README.md, which are present mainly as documentation, though they also help verify that other files, besides the symlink, are still checked out in gix clone.

2. Optionally, clone the repo with git clone to verify Git behavior

The git clone behavior, including the specific command, is shown above in the "Git behavior" section.

The created symlink-to-inacessible directory should be deleted before testing again with gix clone (if it is to be run in the same location), which in PowerShell can be done with:

rm -r -fo symlink-to-inaccessible

3. Clone the repo with gix clone to observe the bug

This is shown above in "Current behavior," showing the error:

symlink: collision (PermissionDenied)
Error: One or more errors occurred - checkout is incomplete: encountered 1 collision(s)

After checking the output, and that the working tree exists with other files besides the symlink that is wrongly taken to be a collision, the created directory should be removed again before running further tests, to avoid clashes.

4. Optionally, clone with --trace for more output

This is mainly to help verify that nothing else unusual is happening. It can also be compared to the output in #1420.

C:\Users\ek\src> gix --trace clone git@github.com:EliahKagan/symlink-to-inaccessible.git
 20:54:22 indexing done 19.0 objects in 0.00s (55.1k objects/s)
 20:54:22 decompressing done 3.6KB in 0.00s (10.0MB/s)
 20:54:22     Resolving done 19.0 objects in 0.05s (375.0 objects/s)
 20:54:22      Decoding done 3.6KB in 0.05s (70.5KB/s)
 20:54:22 writing index file done 1.6KB in 0.00s (6.7MB/s)
 20:54:22  create index file done 19.0 objects in 0.05s (347.0 objects/s)
 20:54:22          read pack done 2.5KB in 0.06s (41.0KB/s)
 20:54:22           checkout done 3.0 files in 0.00s (2.5k files/s)
 20:54:22            writing done 1.4KB in 0.00s (1.2MB/s)
 20:54:22            tracing INFO     run [ 1.12s | 2.43% / 100.00% ]
 20:54:22            tracing INFO     ┝━ open_from_paths() [ 147ms | 8.21% / 13.11% ]
 20:54:22            tracing INFO     │  ┝━ gix_path::git::install_config_path() [ 54.4ms | 4.86% ]
 20:54:22            tracing DEBUG    │  │  ┕━ 🐛 [debug]: invoking git for installation config path | cmd: "git.exe" "config" "-l" "--show-origin"
 20:54:22            tracing DEBUG    │  ┝━ 🐛 [debug]: invoking git to get system prefix/exec path | cmd: "git.exe" "--exec-path"
 20:54:22            tracing INFO     │  ┕━ gix_odb::Store::at() [ 343µs | 0.03% ]
 20:54:22            tracing INFO     ┝━ remote::Connection::ref_map() [ 767ms | 0.00% / 68.59% ]
 20:54:22            tracing INFO     │  ┕━ remote::Connection::fetch_refs() [ 767ms | 0.01% / 68.58% ]
 20:54:22            tracing DEBUG    │     ┝━ gix_protocol::handshake() [ 678ms | 60.61% ] service: UploadPack | extra_parameters: []
 20:54:22            tracing DEBUG    │     │  ┕━ 🐛 [debug]: gix_transport::SpawnProcessOnDemand | command: "C:\\Windows\\System32\\OpenSSH\\ssh.exe" "-o" "SendEnv=GIT_PROTOCOL" "git@github.com" "git-upload-pack" "\'EliahKagan/symlink-to-inaccessible.git\'"
 20:54:22            tracing DEBUG    │     ┕━ gix_protocol::ls_refs() [ 89.1ms | 7.97% ] capabilities: Capabilities { data: "agent=git/github-74f58a100f14\nls-refs=unborn\nfetch=shallow wait-for-done filter\nserver-option\nobject-format=sha1", value_sep: 10 }
 20:54:22            tracing INFO     ┝━ fetch::Prepare::receive() [ 175ms | 0.30% / 15.62% ]
 20:54:22            tracing DEBUG    │  ┝━ negotiate [ 87.8ms | 0.07% / 7.85% ] protocol_version: 2
 20:54:22            tracing DEBUG    │  │  ┝━ mark_complete_and_common_ref [ 1.10ms | 0.07% / 0.10% ] mappings: 2
 20:54:22            tracing INFO     │  │  │  ┝━ mark_all_refs [ 328µs | 0.03% ]
 20:54:22            tracing DEBUG    │  │  │  ┝━ mark_alternate_refs [ 300ns | 0.00% ] num_odb: 0
 20:54:22            tracing INFO     │  │  │  ┝━ mark known_common [ 1.80µs | 0.00% ]
 20:54:22            tracing DEBUG    │  │  │  ┕━ mark tips [ 300ns | 0.00% ] num_tips: 0
 20:54:22            tracing DEBUG    │  │  ┕━ negotiate round [ 86.0ms | 7.69% ] round: 1
 20:54:22            tracing INFO     │  ┝━ gix_pack::Bundle::write_to_directory() [ 63.3ms | 5.66% ]
 20:54:22            tracing DEBUG    │  ┕━ update_refs() [ 20.3ms | 0.66% / 1.81% ] mappings: 2
 20:54:22            tracing DEBUG    │     ┕━ apply [ 12.9ms | 1.16% ] edits: 2
 20:54:22            tracing INFO     ┕━ gix::clone::PrepareCheckout::main_worktree() [ 2.84ms | 0.19% / 0.25% ]
 20:54:22            tracing INFO        ┝━ gix_index::State::from_tree() [ 14.7µs | 0.00% ]
 20:54:22            tracing ERROR       ┝━ 🚨 [error]: symlink: collided (PermissionDenied)
 20:54:22            tracing DEBUG       ┕━ gix_index::File::write() [ 720µs | 0.06% / 0.06% ] path: "symlink-to-inaccessible\\.git\\index"
 20:54:22            tracing DEBUG          ┕━ gix_index::File::write_to() [ 11.9µs | 0.00% / 0.00% ] skip_hash: false
 20:54:22            tracing INFO              ┕━ gix_index::State::write() [ 7.70µs | 0.00% ]
HEAD:refs/remotes/origin/HEAD (implicit)
        77d5afef8ef5ff6f3af10ab900cd1a08e18fc1cc HEAD symref-target:refs/heads/main -> refs/remotes/origin/HEAD [new]
+refs/heads/*:refs/remotes/origin/*
        77d5afef8ef5ff6f3af10ab900cd1a08e18fc1cc refs/heads/main -> refs/remotes/origin/main [new]
symlink: collision (PermissionDenied)
Error: One or more errors occurred - checkout is incomplete: encountered 1 collision(s)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment