Skip to content

Instantly share code, notes, and snippets.

@Ikalou
Last active October 1, 2024 13:49
Show Gist options
  • Save Ikalou/197c414d62f45a1193fd to your computer and use it in GitHub Desktop.
Save Ikalou/197c414d62f45a1193fd to your computer and use it in GitHub Desktop.
Git and Unity

EDIT: this is an old post and a lof the information in this document is outdated.

Using Git with Unity

Git logo

Git is a popular free and open source distributed version control system.

I am new to Unity, but as a long time git user, I wanted to use git for my Unity projects. The bottom line is... it doesn't work nearly as well as I would like it to.

This page contains all the information I could gather on the subject.

Configure Unity

Smart Merge

http://docs.unity3d.com/Manual/SmartMerge.html

Unity incorporates a tool called UnityYAMLMerge that can merge scene and prefab files in a semantically correct way. The tool can be accessed from the command line and is also available to third party version control software.

Unity 5 comes with a tool called UnityYAMLMerge that can automatically merge scenes and prefabs stored in YAML format. If there is a conflict, it will open a standard merge tool (such as KDiff3 or Meld) for a 3-way merge where BASE, LOCAL and REMOTE will all have been preprocessed.

Our objective is to configure git and Unity to take advantage of this tool.

Custom Merge Tool (optional)

You can configure UnityYAMLMerge to use a merge tool of your choice. Edit the file mergespecfile.txt (located in C:\Program Files\Unity\Editor\Data\Tools) accordingly. For instance, to enable KDiff3, change the following two lines...

unity use "%programs%/KDiff3/kdiff3.exe" "%b" "%l" "%r" -o "%d"
prefab use "%programs%/KDiff3/kdiff3.exe" "%b" "%l" "%r" -o "%d"

...and add the following line at the begining of the section 'Default fallbacks for unknown files'.

* use "%programs%/KDiff3/kdiff3.exe" "%b" "%l" "%r" -o "%d"

Edit the paths to match those on your environment.

Asset Serialization

http://docs.unity3d.com/Manual/class-EditorManager.html

To assist with version control merges, Unity can store scene files in a text-based format (see the text scene format pages for further details). If scene merges will not be performed then Unity can store scenes in a more space efficient binary format or allow both text and binary scene files to exist at the same time.

In order for the UnityYAMLMerge tool to work, assets must be stored in YAML format. Go into Edit > Project Settings > Editor, set the Asset Serialization option to Force Text.

Editor Settings

If you do this on a large existing codebase, it can take some time (and even crash).

Configure Git

Git mergetool

Now that everything is setup on the Unity side, we need to tell git to use the proper merge tool.

Add the following to your .git/config:

[merge]
	tool = unityyamlmerge

[mergetool "unityyamlmerge"]
	trustExitCode = false
	keepTemporaries = true
	keepBackup = false
	path = 'C:\\Program Files\\Unity\\Editor\\Data\\Tools\\UnityYAMLMerge.exe'
	cmd = 'C:\\Program Files\\Unity\\Editor\\Data\\Tools\\UnityYAMLMerge.exe' merge -p "$BASE" "$REMOTE" "$LOCAL" "$MERGED"

Edit the paths to match those on your environment.

Next, we want to prevent git from trying to merge some types of file on its own by declaring them as binary files.

To do this, put the following in your .gitattributes:

*.unity binary
*.prefab binary
*.asset binary

The good news is that UnityYAMLMerge will do all the merging for us. The bad news if that the file will be treated as on opaque blob as if it was still in binary format.

Human-redable diffs (optional)

We can use a simple textconv script to retain basic, human-readable diffs.

Add this to your .git/config:

[diff "unity"]
      textconv='C:\\Users\\petit_v\\unityYamlTextConv.py'

Edit the paths to match those on your environment.

Then, edit your .gitattributes to read (note the added diff=unity):

*.unity binary diff=unity
*.prefab binary diff=unity
*.asset binary diff=unity

An example of a textconv script, unityYamlTextConv.py, could be:

#!/usr/bin/env python

import sys
import yaml
import pprint

if len(sys.argv) < 2:
	sys.exit(-1)

def tag_unity3d_com_ctor(loader, tag_suffix, node):
	values = loader.construct_mapping(node, deep=True)
	if 'Prefab' in values:
		if 'm_Modification' in values['Prefab']:
			del values['Prefab']['m_Modification']
	return values

class UnityParser(yaml.parser.Parser):
    DEFAULT_TAGS = {u'!u!': u'tag:unity3d.com,2011'}
    DEFAULT_TAGS.update(yaml.parser.Parser.DEFAULT_TAGS)

class UnityLoader(yaml.reader.Reader, yaml.scanner.Scanner, UnityParser, yaml.composer.Composer, yaml.constructor.Constructor, yaml.resolver.Resolver):
    def __init__(self, stream):
        yaml.reader.Reader.__init__(self, stream)
        yaml.scanner.Scanner.__init__(self)
        UnityParser.__init__(self)
        yaml.composer.Composer.__init__(self)
        yaml.constructor.Constructor.__init__(self)
        yaml.resolver.Resolver.__init__(self)

UnityLoader.add_multi_constructor('tag:unity3d.com', tag_unity3d_com_ctor)
with open(sys.argv[1], 'r') as stream:
	docs = yaml.load_all(stream, Loader=UnityLoader)
	for doc in docs:
		pprint.pprint(doc, width=120, indent=1, depth=6)

Now it's possible to see what changed inside of a prefab, asset or scene:

Prefab Diff Imgur

There are a lot of things that could be done to improve the output of the script to make it more meaningfull to us humans.

[rant]
You may have noticed that the script above will not display the m_Modification field inside of a Prefab. Apparently, someone at Unity thought it was a good idea to store a complete list of all the modifications to a prefab inside of said prefab. Why?... I use a version constrol system whose sole purpose is to keep track of file changes, I don't need or want any of this.

If you make a change to a Prefab in the Editor and then Ctrl-Z, you can get yourself into a situation like this one:

$ git status
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   Assets/prefabs/Asteriod.prefab

no changes added to commit (use "git add" and/or "git commit -a")

$ git diff

$

This is because m_Modification has been altered, even though the prefab is still the same in every way. At this point, you should git chekout the file back, unless you want to introduce needless non fast-forward merges in your history. In fact, the UnityYAMLMerge tool seems to be ignoring this section entirely.

EDIT: It apprears this is a bug that will be fixed in Unity 5.4.

This is complete nonsense, this is madness, and it makes me sad.
[/rant]

Unfortunatly, PyYAML seems to choke on some Unity scenes. It's not celar to me weather PyYAML has a bug or if Unity is not following the spec.

yaml.scanner.ScannerError: mapping values are not allowed here

See:

.gitignore (optional)

You may use the following .gitignore file. Up-to-date definitions can be found on github.

Thanks to all the contributors for putting these files together.

#
# == Windows ==
#

# Windows image file caches
Thumbs.db
ehthumbs.db

# Folder config file
Desktop.ini

# Recycle Bin used on file shares
$RECYCLE.BIN/

# Windows Installer files
*.cab
*.msi
*.msm
*.msp

# Windows shortcuts
*.lnk

#
# == Unity ==
#

/[Ll]ibrary/
/[Tt]emp/
/[Oo]bj/
/[Bb]uild/
/[Bb]uilds/
/Assets/AssetStoreTools*

# Autogenerated VS/MD solution and project files
ExportedObj/
*.csproj
*.unityproj
*.sln
*.suo
*.tmp
*.user
*.userprefs
*.pidb
*.booproj
*.svd


# Unity3D generated meta files
*.pidb.meta

# Unity3D Generated File On Crash Reports
sysinfo.txt

# Builds
*.apk
*.unitypackage

Conclusion

I can't believe how hard it is to diff and merge some types of files in Unity, even after doing all that. How do Unity developers review their changes before committing them? How do they handle even trivial merge conflicts?

Git + Unity is full of spiders. And snakes. And they're on fire.

Sources:

@Nefari0uss
Copy link

Just want to say thanks for this guide, especially the text conversion script.

@benwrk
Copy link

benwrk commented Dec 2, 2017

Thanks a lot, that Python script is so useful.

PS. @jsgoller1 I am not sure if this is too late, but it seems to me that C:Program: command not found might be caused by unescaped backslash in your .gitconfig file. Check your .gitconfig file if all backslashes are escaped properly.

@Kimeiga
Copy link

Kimeiga commented Jun 21, 2018

This guide is fantastic! I'm going to put my current unity git files on gist in case anyone needs them, but I pretty much copied my .gitignore from your excellent one! (i wish I knew about these when I was working on a team on a Unity game and we had such a hard time with git!)

.gitattributes
.gitignore
.git/config

@hugabora
Copy link

Modified the unityYamlTextConv.py script to remove the ' stripped' part from the yaml files before parsing.
This fixes the invalid unity file parsing error yaml.scanner.ScannerError: mapping values are not allowed here

UnityLoader.add_multi_constructor('tag:unity3d.com', tag_unity3d_com_ctor)
with open(sys.argv[1], 'r') as ystream:
	data = ystream.read();
	data2 = re.sub('(---.*?) stripped', '\\1', data)
#	open('temp.txt', 'w').write(data2);
	stream = io.StringIO(data2)
	docs = yaml.load_all(stream, Loader=UnityLoader)
	for doc in docs:
		pprint.pprint(doc, width=120, indent=1, depth=6)

@Naphier
Copy link

Naphier commented Dec 14, 2018

We use the in-editor tool "UniMerge" which does visual diffing for scenes and prefabs. It's helped a lot, but it's still a manual process. I never had any luck with the smartmerge tool. Seemed to always have issues. UniMerge isn't perfect either, but at least you can understand what's happening.

Copy link

ghost commented Apr 10, 2019

We are a 2-man project, and after not being able to get the UnityYAML-merge thing to work (complaining about incorrect yaml which it generated itself!!) we finally chose to work on "feature"-scenes and then merge them manually on the master. Copy pasting GameObjects from one scene to another seems to work pretty well.

@doctorpangloss
Copy link

doctorpangloss commented Apr 26, 2020

CORRECT MACOS & WINDOWS GIT UNITYYAMLMERGE INSTRUCTIONS

Here's the actual, correct instructions for getting UnityYamlMerge to work:

  1. Create or update your .gitattributes file at the root of your git repository containing your project to specify that Unity text-based assets should be merged using a new merge command named unityyamlmerge:
* text=auto

# Unity files
*.meta -text merge=unityyamlmerge diff
*.unity -text merge=unityyamlmerge diff
# warning: this will interact with terrains poorly
*.asset -text merge=unityyamlmerge diff
# uncomment the following line and store your terrains in a separate directory
# Assets/Terrains/*.asset binary
*.prefab -text merge=unityyamlmerge diff
*.mat -text merge=unityyamlmerge diff
*.anim -text merge=unityyamlmerge diff
*.controller -text merge=unityyamlmerge diff
*.overrideController -text merge=unityyamlmerge diff
*.physicMaterial -text merge=unityyamlmerge diff
*.physicsMaterial2D -text merge=unityyamlmerge diff
*.playable -text merge=unityyamlmerge diff
*.mask -text merge=unityyamlmerge diff
*.brush -text merge=unityyamlmerge diff
*.flare -text merge=unityyamlmerge diff
*.fontsettings -text merge=unityyamlmerge diff
*.guiskin -text merge=unityyamlmerge diff
*.giparams -text merge=unityyamlmerge diff
*.renderTexture -text merge=unityyamlmerge diff
*.spriteatlas -text merge=unityyamlmerge diff
*.terrainlayer -text merge=unityyamlmerge diff
*.mixer -text merge=unityyamlmerge diff
*.shadervariants -text merge=unityyamlmerge diff
  1. Modify the .git/config file in your repository by adding the unityyamlmerge command specified below. Observe the use of driver, which makes this actually work. You will need to use the correct path for your platform.

For macOS with a 2021.3.10f1 editor installed through the hub, the following command is correct:

[merge "unityyamlmerge"]
	driver = '/Applications/Unity/Hub/Editor/2021.3.10f1/Unity.app/Contents/Tools/UnityYAMLMerge' merge -h -p --force %O %B %A %A
	name = Unity SmartMerge (UnityYamlMerge)
	recursive = binary

On Windows:

[merge "unityyamlmerge"]
	driver = 'C:/Program Files/Unity/Hub/Editor/2021.3.10f1/Editor/Data/Tools/UnityYamlMerge.exe' merge -h -p --force %O %B %A %A
	name = Unity SmartMerge (UnityYamlMerge)
	recursive = binary

Observe you can use forward slashes in paths on Windows, you can omit the "trustExitCode" configuration, you do not declare these files as binary, etc. The configurations above, even from Unity itself, were flawed, and these work on both platforms consistently.

What about the other solutions?

This ranks really highly on Google (#2 result). Nobody above this line has had merging working. UniMerge also does not work on macOS because its merge driver script is written incorrectly. Don't bother with either the GitHub-for-Unity or official (preview) Unity git packages because they have show-stopping bugs on macOS.

How should I resolve the merges?

Use Rider's merge functionality in Git > Resolve Conflicts...

I am not sure what the equivalent is for VS Code.

You can also run a git-supporting merge tool on the directory after you have run

Feature branch to development branch

This workflow shows rebasing, which you should do when you are the only person working on a feature / task.

On Windows & macOS

git checkout feature
git rebase development

At this stage, you will be given an opportunity to resolve the conflicts.

hint: Resolve all conflicts manually, mark them as resolved with
hint: "git add/rm <conflicted_files>", then run "git rebase --continue".
hint: You can instead skip this commit: run "git rebase --skip".
hint: To abort and get back to the state before "git rebase", run "git rebase --abort".

This means you should open Rider, navigate to Git > Resolve Conflicts... and start working on the conflicts.

Once you have finished resolving all the conflicts, use the following commands:

# review your changes
git status
# add them all
git add -A
# review for safety one more time
git status
# "commit" by continuing the rebase
git rebase --continue --no-edit

On Windows, git rebase may show vi as your editor in the command line to modify the commit message. --no-edit skips this, but in case you want to edit it, you can edit in vi (out of scope here) or use a different text editor (see Google).

Sharing a branch with others

git checkout develop
git pull develop
# resolve conflicts in Rider
git commit -m "Merge"

Troubleshooting

Error parsing file '.merge_file_a11288': File is not a valid text serialized YAML file. Make sure that Asset Serialization is set to 'Force text' in Editor Settings.

Updated
This means you have an invalid, binary asset in your project in a Force Text Serialization project. It is a real error.

Most of the time, this means you are trying to merge a terrain .asset file, which is actually mostly binary data. Uncomment the line about terrains in the .gitattributes snippet above.

@elitegoliath
Copy link

elitegoliath commented Jul 18, 2020

@doctorpangloss Your instruction set ALMOST works for me. I'm testing this out by causing an intentional merge conflict in a .meta file. I get the following error though: Error parsing file '.merge_file_a11288': File is not a valid text serialized YAML file. Make sure that Asset Serialization is set to 'Force text' in Editor Settings.

I've tried every example I could find all over the web and absolutely NONE of them work. Not a single one of them except for the one you mentioned.

@elitegoliath
Copy link

On Windows 10, I got this to work (with Meld):

mergespecfile.txt

unity use "%programs%\Meld\meld.exe" "%b" "%r" "%l" -o "%d" --auto-merge
prefab use "%programs%\Meld\meld.exe" "%b" "%r" "%l" -o "%d" --auto-merge
* use "%programs%\Meld\meld.exe" "%b" "%r" "%l" -o "%d" --auto-merge

.gitattributes

# Macro for Unity YAML-based asset files.
[attr]unityyaml -text merge=unityyamlmerge diff

# Unity files
*.meta unityyaml
*.unity unityyaml
*.asset unityyaml
*.prefab unityyaml

.git/config

[merge "unityyamlmerge"]
    driver = 'C:/Program Files/Unity/Hub/Editor/2019.3.14f1/Editor/Data/Tools/UnityYAMLMerge.exe' merge -p \"$BASE\" \"$REMOTE\" \"$LOCAL\" \"$MERGED\"
    name = Unity SmartMerge (UnityYamlMerge)
    recursive = binary
[diff]
    tool = meld
[difftool "meld"]
    path = C:/Program Files (x86)/Meld/meld.exe
[merge]
    tool = meld
[mergetool "meld"]
    path = C:/Program Files (x86)/Meld/meld.exe
    prompt = false

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment