Skip to content

Instantly share code, notes, and snippets.

@LotteMakesStuff
Last active January 5, 2023 20:54
Show Gist options
  • Save LotteMakesStuff/6e02e0ea303030517a071a1c81eb016e to your computer and use it in GitHub Desktop.
Save LotteMakesStuff/6e02e0ea303030517a071a1c81eb016e to your computer and use it in GitHub Desktop.
UPM: How to make a custom package

UPM: How to make a custom package So, Unity has this shiny new package manager, and you have code you want to share between projects - wouldn't it be great if we could bundle up our shared code and plug it into all the projects that need it? Let's figure out how to make our own Package!


Todo

  • Modify the project manifest
  • Make a package manifest
  • Package the manifest up with some test code
  • Try it out in Unity!

Project Manifest

Unity uses a 'Manifest' file to tell it what packages a project cares about. It's essentially a list of the packages we want and what versions we care about. we can also flag up packages we don't want included. Let's open up our manifest and in a custom dependency on a local package.

Note: where to find the manifest

  • Depending on what beta version of Unity 2018.1 you are using, manifest.json might be in a diffent folder as its location changed in b9
    • Unity 2018.1b9 >: ProjectRoot/Packages
    • Unity 2018.1b1-b8: ProjectRoot/UnityPackageManager
{
"dependencies": {
"com.unity.package-manager-ui": "1.8.0",
"com.unity.analytics": "exclude",
"com.lms.boundingvolumes": "file:../../CustomPackageTest/lmsBoundingVolumes"
}
}

This is an example of what a projects manifest file might look like. It's easy to see we have a dependency on the Package Manager UI (explicitly version 1.8.0). We have also excluded the Analytics package. There could be any number of other packages included or excluded, depending on your projects precise setup and requirements.

More interestingly, we have added in our custom package, com.lms.boundingvolumes. As we can see, all package names start with com. The packages Unity hosts on its repository then all have .unity.. This clearly identifies them as packages published by Unity. For our custom package, I've replaced that with .lms., short for LotteMakesStuff. Finally, we have the name of our actual package! In this example were packing up code that implements a Bounding Sphere structure, so we picked the name boundingvolumes as it nicely reflects the contents of our package. (obviously, our package could contain any code - I just picked bounding volumes as I was working on them recently!).

Unlike with the package we are getting from Unity's package repository, instead of specifying a version number we want UPM to resolve, instead, we are providing a local, relative file path to our package. File locations have to be relative. this might cause issues with code stored in version control, as we have to make sure the packages we depend on are in the same relative position on each development system. In this example, we go back two levels to the folder that contains our projects root folder and then going into a folder containing just our custom package. In a source control context, it might make more sense to host the shared content in a Unity project of its own and then have the package in a subfolder of the host projects Assets folder. this provides an easy way to test and develop our shared components and a nice way to ensure the relative paths work, all a developer has to do is ensure the shared code project is checked out side by side with the project that consumes it.

Todo

  • Modify the project manifest
  • Make a package manifest
  • Package the manifest up with some test code
  • Try it out in Unity!

Package Manifest

The package manifest sits in the root of the folder our packages contents is stored in and provides a whole bunch of metadata to Unity describing what our package actually is. Most of this is currently used by the Package Manager UI to control how it draws our packages information, or by the Package Repository to resolve references to our package. Unfortunately, as we currently cant publish our own packages to the repository - most of it does nothing but it's still useful to make sure its set so it shows in the UI correctly. The tags that Unity currently uses are defined in the package manager documentation

{
"name": "com.lms.boundingvolumes",
"displayName": "Bounding Volumes",
"version": "0.0.1",
"unity": "2018.1",
"description": "Custom Bounding Volume structs, for situations where we cant use the ones provided by Unity.",
"keywords": [
"maths",
"unity"
],
"category": "Unity"
}
  • name - this is the name of the package and has to match the name we use in the project manifest exactly. Its even case sensitive.
  • displayName - The Package Manager UI window will show the value from name by default, which isn't particularly human readable. We can provide a more readable string for it in this tag
  • version - All UPM packages should provide a version, using the Semantic Versioning system by convention. As our packages are only referenced locally and never resolved by the package manager, this is less important but still a best practice.
  • unity - the minimum version of unity this package needs to work, which for now is pretty much always 2018.1
  • description - this is a basic description that is displayed in the Package Manager UI when our package is selected
  • keywords & category - this dont really apply too much to us right now, keyword is used for repository search and category doesn't seem to be used anywhere right now
  • dependencies (not shown) - we can include a list of packages we depend on, in a similar way that the project manifest does.

Todo

  • Modify the project manifest
  • Make a package manifest
  • Package the manifest up with some test code
  • Try it out in Unity!

The Test Package

Lets build up a really simple test package. All it will contain is the Package Manifest we wrote, one small code file, and an assembly definition. Assembly Definition files are a realtivly new unity feature which lets us partition code off into sepearte files when it gets compiled. This will help keep unitys compilation times down, especally when pacakges contain a lot of code. You can read more about them here

This is what our entire looks like.

Todo

  • Modify the project manifest
  • Make a package manifest
  • Package the manifest up with some test code
  • Try it out in Unity!

How does it look in the Package Manager UI?

Todo

  • Modify the project manifest
  • Make a package manifest
  • Package the manifest up with some test code
  • Try it out in Unity!

Notes

  • The package manager seems to be based on NPM. NPM supports URL links to packages and links to packages in git repositories, unfortunately, I've not been able to get these to work with UPM, only local, relative file references.
    • let me know if you get this working!
  • Currently, the Unity Package Manager UI window seems to ignore the author: tag in package manifests, and displays every package as being written by Unity.
  • You can find a Zip with our test Package below! extract it somewhere on your system and set up a relative file dependency just like we did in this tutorial to try it for yourself! you should then be able to use the LotteMakesStuff.Mathmatics.BoundingSphere class in your test project!

Got Questions? Im @LotteMakesStuff on Twitter, My DMs are open!

Buy Me a Coffee at ko-fi.com Become a Patron!

@okcompute
Copy link

Interesting investigation report! :)

  • Git URL will be supported at some point in the future. I cannot give an ETA though :)
  • Author support in the UI will come soon.

We are super excited to bring this new system to all of you. Though, this is not an easy task. We are injecting a package manager in a foreign ecosystem. This requires time to make sure we don't break everything.

Regards,

Pascal
Unity Package Manager team lead

@LotteMakesStuff
Copy link
Author

hi @okcompute, glad you found it interesting!! Im loving the feature so far and have had so much fun trying to figure out how it works and just generally poking and prodding and breaking it over the past week - its an amazing remarkable achievement! feels like a whole new unity~

As for Git URL/Remote custom packages.... i might have a cheeky project trying to address that in a very buggy state ^^ Please give my thanks to all your team!

@twilson90
Copy link

Thanks so much for writing this.
I've been using this method for importing shared assets recently and found it helped with my ever-increasing compile time on a rather large project I'm working on.

I now have a separate project called 'Shared Assets' which contains the packages and their package.json.
I've even written an editor utility for easily switching out packages without having to edit the project's json in a text editor (overkill, I know)
The only problem with this whole approach is opening my project now takes upto a minute longer, as it recompiles each individual local package.
I've noticed the builtin packages (in C:\ProgramData\Unity\cache\packages) are precompiled as dlls and don't need recompiling.
I suppose I could write an editor script in the Shared Assets project that creates neat little precompiled versions of themselves.

Anyway, thanks again, this has been a big help in reducing compile times.

@twilson90
Copy link

I've just noticed something - if you put local URIs in the package manager json (file:../../etc.), then upon changing any scripts within the project, Unity compiles project scripts in a slightly different way.
Usually, you get a little spinning wheel in the bottom right for a few seconds, at which point you can maybe query if EditorApplication.isCompiling is true, you can interact with the editor until it begins loading the new assemblies.
However, if you've done the trick that Lotte has outlined, then you are immediately confronted with a dialog 'Compiling Scripts...' as soon as you focus on the UnityEditor window after modifying a project script. The whole application freezes. No interaction, and no chance to query the state of EditorApplication.isCompiling.
Kind of annoying as I rely on EditorApplication.isCompiling for a few things.

@andybak
Copy link

andybak commented May 9, 2018

@okcompute Git url support would be amazing. However many Unity projects on Github consist of the whole project. It would be amazing to be able to specify a subdirectory.

As git only allows checkouts of whole repos (unless you do subtree trickery which doesn't help keep the size of the repo down) it would be useful to be able to specify which subdirectory should be regarding as the package root. I imagine that Unity would check out the whole repo but ignore code above a certain specified directory.

Does that make sense?

@LotteMakesStuff I'd love to see the cheeky, buggy project you refer to! EDIT - is it this? https://gist.github.com/LotteMakesStuff/2247e8a7743a1ceb52331f8e2f5531f7

@Feddas
Copy link

Feddas commented May 9, 2018

@LotteMakesStuff This is great. I can't help but wonder how you figured out manifest.json version numbers could be replaced with file paths. Did you randomly try stuff making guesses based off the error messages or did you do digging through https://github.com/Unity-Technologies/UnityCsReference to help figure it out.

@zeh
Copy link

zeh commented May 25, 2018

Great instructions; just got it working on a test package thanks to it. Super excited about UPM. Can't wait to see people using packages for code (instead of polluting a game's code), and of course, Git support. So, thank you so much, and thanks Unity.

I'd add a few more details to make things smoother:

  1. On step 5, you do need to create an "Assembly Definition" by doing Assets > Create > Assembly Definition (on any project) and then copying that file to the package folder. This is implied in the text but wasn't clear to me until I visited the page and learned how. Also, the file name doesn't seem to matter much - I followed a different naming syntax as the screenshot and it still worked.
  2. On step 6, the meta files don't seem to be needed. In fact, Unity 2018.1 is removing them at start if I had them there.
  3. On windows, you can also use absolute paths for the package location, e.g.:
"com.me.mypackage": "file:d:/whatever/upm/mypackage"

If the package location is incorrect (e.g. doesn't exist), Unity will tell you when starting the editor and loading the project.

@yacuzo
Copy link

yacuzo commented Jun 4, 2018

Great stuff. I did however hit a crippling snag: If I want to use assembly definitions in my project (which depends on these local packages) then none of the code in the package is found, and I am unable to reference the asmdef (BoundingVolumes.asmdef) from my own asmdef.

Do you know a way around this?

@saviorextr
Copy link

This was extremely useful for me, much Thanks.
@yacuzo: I was able to get around this by manually adding the asmdef reference into the asmdef of my own project. Was counting on it to not work, but it totally does.

@starikcetin
Copy link

Thanks for compiling this, super useful.

@Caiuse
Copy link

Caiuse commented Jul 10, 2018

With 2018.2 being released anyone got any updates on using git repos instead of directory paths?

@mattyoung-improbable
Copy link

mattyoung-improbable commented Jul 12, 2018

Great Post. I'm getting an error using 2018.2.0b10:

NotSupportedException: Visual Studio Tools: Project file has no matching compilation unit for <AssemblyName xmlns="http://schemas.microsoft.com/developer/msbuild/2003">LMS.BoundingVolumes</AssemblyName> SyntaxTree.VisualStudio.Unity.Bridge.ProjectFilePostprocessor.OnGeneratedCSProject (System.String path, System.String content) System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:222) Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation. System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:232) System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MethodBase.cs:115) UnityEditor.AssetPostprocessingInternal.CallOnGeneratedCSProject (System.String path, System.String content) (at C:/buildslave/unity/build/Editor/Mono/AssetPostprocessor.cs:178) UnityEditor.VisualStudioIntegration.SolutionSynchronizer.SyncProjectFileIfNotChanged (System.String path, System.String newContents) (at C:/buildslave/unity/build/Editor/Mono/VisualStudioIntegration/SolutionSynchronizer.cs:319) UnityEditor.VisualStudioIntegration.SolutionSynchronizer.SyncProject (MonoIsland island, System.Collections.Generic.Dictionary2 allAssetsProjectParts, UnityEditor.Scripting.Compilers.ResponseFileData responseFileData, System.Collections.Generic.List1 allProjectIslands) (at C:/buildslave/unity/build/Editor/Mono/VisualStudioIntegration/SolutionSynchronizer.cs:312) UnityEditor.VisualStudioIntegration.SolutionSynchronizer.GenerateAndWriteSolutionAndProjects (ScriptEditor scriptEditor) (at C:/buildslave/unity/build/Editor/Mono/VisualStudioIntegration/SolutionSynchronizer.cs:244) UnityEditor.VisualStudioIntegration.SolutionSynchronizer.Sync () (at C:/buildslave/unity/build/Editor/Mono/VisualStudioIntegration/SolutionSynchronizer.cs:216) UnityEditor.VisualStudioIntegration.SolutionSynchronizer.SyncIfNeeded (IEnumerable1 affectedFiles, IEnumerable1 reimportedFiles) (at C:/buildslave/unity/build/Editor/Mono/VisualStudioIntegration/SolutionSynchronizer.cs:190) UnityEditor.SyncVS.PostprocessSyncProject (System.String[] importedAssets, System.String[] addedAssets, System.String[] deletedAssets, System.String[] movedAssets, System.String[] movedFromAssetPaths) (at C:/buildslave/unity/build/Editor/Mono/SyncProject.cs:175) UnityEditor.AssetPostprocessingInternal.PostprocessAllAssets (System.String[] importedAssets, System.String[] addedAssets, System.String[] deletedAssets, System.String[] movedAssets, System.String[] movedFromPathAssets) (at C:/buildslave/unity/build/Editor/Mono/AssetPostprocessor.cs:134)

After having a dig around, it looks like it's getting confused about whether to create a .csproj for the assembly definition in the package, and fails if it's not within the unity project folder. There must be some logic that decided not to do it for the built in unity packages, that's not working for 'file:' ones.

Edit: This turned out to be an issue with outdated Visual Studio Tools for Unity. Found updates to the packages here: https://forum.unity.com/threads/local-package-outside-of-project-folder.540170/#post-3563292

@Thaina
Copy link

Thaina commented Jul 15, 2018

I have create custom package and it easily work with some reimport/restart unity

Now I would like to know how to create and maintain dependency of packages. How can I reference another package in the package?

Edit: Now is OK I just need to make a reference to asmdef and try to force the whole thing to rebuild with some random effort and it works now. Thanks for you kickstart guide that now I know the UPM system

@LotteMakesStuff
Copy link
Author

This is great. I can't help but wonder how you figured out manifest.json version numbers could be replaced with file paths.

@Feddas great question, i actually saw it in an early version of the ECS and then just played around with it until i figured it all out

@LotteMakesStuff
Copy link
Author

@andybak git endpoints is on the roadmap the disclosed at unite berlin! yay!
Oh, and my sneaky hack was to implement a proxy server that sat between Unity and UPM. it could echo all the calls to the UPM service and modify any return results i was interested in to inject my own packages. It was fun, but ultimatly didnt work very well

@UCh
Copy link

UCh commented Oct 4, 2018

Apparently is possible to use your own NPM registry with packages, including transitive dependency resolution https://forum.unity.com/threads/other-registries-than-unitys-own-already-work-nice.533691/

@shrikky
Copy link

shrikky commented Dec 19, 2018

Thank you very much! I am getting this error :

error CS0009: Metadata file `C:/Users/xxx/projects/SystemsUnityTools/SystemsUnityTools/Packages/Moq.4.10.1/lib/net45/Moq.dll' does not contain valid metadata

Please help

@keenanwoodall
Copy link

keenanwoodall commented Feb 2, 2019

@LotteMakesStuff What is the best way for a package to get/create files?

Example 1:
If a tool has some scriptable object that stores settings, it shouldn't be stored alongside the tool in the Packages folder because any changes to it will get overridden when the package files are updated/synced with the source files, right?

Example 2:
A package has a custom texture it needs for a custom editor. How can we get this file when the actual package's file path could be many things?


AFAIK, if the package source is on disk, the package is just referenced (not copied over). However if the package is created by Unity (or hosted on a repo), the files are copied under YourProjectName/Library/PackageCache/name@version. And if the package source is a repo, the packages folder name ends up being name@somereallylongcommithash which makes it even harder to predict any file paths.

All of these different variables has prevented me from being able to load assets from within the package. Is there a way to get the package's file path?

[Edit] So it looks like AssetDatabase.FindAssets and AssetDatabase.LoadAssetAtPath will let you load from "Packages/com.example/example" regardless of the package's real location. Makes sense as the package is in the database.

@MoutonSanglant
Copy link

MoutonSanglant commented Feb 13, 2019

Hello, just to inform that git support is possible since 2018.3b.

Note that (quoting okcompute_unity):

As stated in the release notes, this is an experimental feature. We are targeting production ready quality for 2019.1. Super glad that you are trying this out though. Please, let us know if you find any issues.

In order to enable using a git remote, edit your manifest.json like so:

  "dependencies": {
    ...
    "com.mycompany.package": "https://your.remote-git.com/path/to/repository.git",
    ...
}

You can find more information on this topic: https://forum.unity.com/threads/git-support-on-package-manager.573673/

@Morgan-6Freedom
Copy link

Morgan-6Freedom commented Oct 26, 2019

Hello, I succeed to create a package on my github, with the manifest json.
However, how do I add it in my project ? The only option available is "Add Package from disk"

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment