Skip to content

Instantly share code, notes, and snippets.

@pjastam
Last active June 18, 2023 10:02
Show Gist options
  • Save pjastam/03f8b9eca4e97544f02bc55c464f8514 to your computer and use it in GitHub Desktop.
Save pjastam/03f8b9eca4e97544f02bc55c464f8514 to your computer and use it in GitHub Desktop.
Quarto line endings bug on Windows

Quarto line endings bug on Windows

For cross-platform projects, the recommended setting of Git on Windows is "Checkout Windows-style, commit Unix-style line endings". This means that Git will convert LF to CRLF when checking out text files from a remote repository to your local repository. When committing text files, CRLF will be converted to LF.

However, as freezing files in Quarto is done before the commit, the hashes of the freezed files in the _freeze folder in on Windows are based on the files with CRLF line endings and based on the files with LF line endings on Unix. The consequence is different hashes although the source files are identical. This bug is therefore called: same source different hashes in _freeze/ files from different OS machines #3599.

In my workflow I use RStudio as an IDE on Windows. Therefore, I suffered from this bug too, as I describe in the issue GitHub Actions fails b/o missing rmarkdown package #5870.

Per-repository solution

I found a way to tackle this problem in a per-repository way. This approach boils down to adding a .gitattibutes file to the repository and changing one project setting in RStudio. Note that this approach has the additional advantage that these adjustments only change the settings of the project repository, without altering any global settings on Windows (or other project repositories).

The easiest thing to do is to make the necessary adjustments in the remote repository, for example in your repo at GitHub.com. This is the way to go if you did not yet checkout the remote repo to your local Windows environment. The advantage of this starting point is that the number of adjustments to make is only two. I will describe this two-step-approach below first.

It takes some more time if the starting point for making the necessary adjustments is your local repository on Windows after checkout. In this case, you might prefer the easy two-step route nonetheless and delete your local repo, make the necessary adjustments in the remote repo and then checkout to your local Windows environment. But may be you have already made some nice updates to this local repo, that you do not want to ditch. In that case, it seems logical to start with making the necessary adjustments in your local repo and pushing them to the remote repo afterwards. I will describe this below second.

Make adjustments in remote repository

You decided to make the adjustments in your remote repository. The first step is to create a .gitattributes file with the following line of code (or add it to an already existing .gitattributes file):

*.qmd eol=lf

The second step is to add the following line of code to the .Rproj file:

LineEndingConversion: None

The second step prevents line conversion by RStudio when saving files in your project. You can find this setting in Rstudio -> Tools -> Project Options -> Code Editing -> Line ending conversion. This is set to None in this second step.

Note that this second step is a must if the global settings of Rstudio -> Tools -> Global Options -> Code -> Saving -> Line ending conversion are set to Platform Native, as is in my RStudio install. After all, this global setting means that when saving edited files the Git configuration variable core.autocrlf setting will be applied, which is set to true given the recommended "Checkout Windows-style, commit Unix-style line endings" setting of Git on Windows. As a consequence, line endings in a *.qmd file will be changed to the Windows CRLF after saving edited files, even if they contain Unix LF line endings before editing this file. This core.autocrlf setting is overruled by the line of code of the second step.

Now you can checkout this remote repository to your local environment and you are ready to go. No additional steps needed.

Make adjustments in local repository

You decided to make the adjustments in your local repository on Windows after checkout. The first step is to create a .gitattributes file by executing the following commands in your Rstudio Terminal pane:

# Create file .gitattributes
echo "*.qmd eol=lf"  > .gitattributes

# Commit to local repo & push to remote repo
git add .gitattributes
git commit -m "Added .gitattributes to repo"
git push

# Check nothing to commit (otherwise, first commit!)
git status

# Update line endings in local files
git rm --cached -r .
git reset --hard

The second step is to change the RStudio line ending setting to None by executing the following commands in your Rstudio Terminal pane (do nto forget to replace YOUR_RPROJ_FILENAME by the filename of your .Rproj file):

# Add this codeline to the .Rproj file
echo "LineEndingConversion: None"  >> YOUR_RPROJ_FILENAME.Rproj

# Commit to local repo & push to remote repo
git add .
git commit -m "Added .Rproj to repo"
git push

The third step is to delete the _freeze folder and rerender the Quarto files by executing the following commands in your Rstudio Terminal pane:

# Rerender Quarto files
rm -r _freeze
quarto render

# Commit to local repo & push to remote repo
git add .
git commit -m "Added _freeze to repo"
git push

Now you are all set and ready to go. No additional steps needed.

(see make-adjustments-in-local-repo.sh for the complete set of shell commands described in this section)

Warning

RStudio is used as an IDE in my use case. If you use another IDE, be sure to check the line ending settings and make changes in your IDE settings to overrule the core.autocrlf setting to true. Note that it is not necessary and not recommended to change the core.autocrlf setting itself for cross-platform projects.

Recommendation

It would help the average RStudio user on Windows to incorporate the adjustments described above in the Quarto project template that you can choose from when starting a new project in RStudio. In that case the adjustments described above are set by default. This recommendation is in line with this suggestion.

Credits

There is a general example of configuring git to handle line endings. The next section on this page describes a general procedure to refresh a repository after changing line endings. The Please Add .gitattributes To Your Git Repository post was very helpful for making this general example and procedure specific to my use case.

###############################
### ADD .GITATTRIBUTES FILE ###
###############################
# Create file .gitattributes (alternatively, create this file manually)
echo "*.qmd eol=lf" > .gitattributes
# Commit to local repo & push to remote repo
git add .gitattributes
git commit -m "Added .gitattributes to repo"
git push
# Check nothing to commit
git status
# Update line endings in local files
git rm --cached -r .
git reset --hard
###############################
### CHANGE .RPROJ FILE ###
###############################
# Add this codeline to the .Rproj file (alternatively, do this in RStudio menu:
# Tools -> Project Options -> Code Editing -> Line ending conversion: None)
echo "LineEndingConversion: None" >> YOUR_RPROJ_FILENAME.Rproj
# Commit to local repo & push to remote repo
git add .
git commit -m "Added .Rproj to repo"
git push
###############################
### REFREEZE .QMD FILES ###
###############################
# Rerender Quarto files
rm -r _freeze
quarto render
# Commit to local repo & push to remote repo
git add .
git commit -m "Added _freeze to repo"
git push
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment