Unity hotswapping notes

Unity-hotswapping-notes.md

Unity hotswapping notes

Unity has built-in support for hotswapping, which is a huge productivity booster. This feature works not only with graphics assets like bitmaps and meshes, but also with code: if you edit the source and save it, the editor will save the state of the running game, compile and load the new code, then load the saved state and continue where it left off. Unfortunately, this feature is very easy to break, and most available 3rd party plugins have little regard for it.

It looks like there’s a lot of confusion about hotswapping in Unity, and many developers are not even aware of its existence – which is no wonder if their only experience is seeing lots of errors on the console when they forget to stop the game before recompiling... This document is an attempt to clear up some of this confusion.

Nota bene, I’m not a Unity developer, so everything below is based on blog posts and experimentation. Corrections are most welcome!

The basic flow of hotswapping

When you recompile while the game is playing in the editor, the following series of events happens:

1. Active scripts are disabled.
2. All the live objects are serialised (invoking custom callbacks where applicable).
3. The assembly is unloaded.
4. The changed code is compiled.
5. The new assembly is loaded.
6. The previously saved state is reloaded (objects are instantiated as needed, fields populated, custom callbacks executed).
7. All previously active scripts and components are enabled.

And this is where your console is usually filled with hundreds of null pointer exceptions...

Hotswapping vs. persistence

It’s important to understand that hotswapping and persistence (storing scene, prefabs and scriptable object assets) use slightly different flavours of the serialisation system. The biggest difference is that during hotswapping private fields and auto-properties (i.e. those with a backing field) are also saved and restored when the new code is loaded, while these fields are not persisted in assets by default.

What exactly is preserved during hotswapping?

While remembering the rules is useful, there’s a simple trick one can use: turn on the debug mode of the inspector view. In this mode, you can see the raw fields, including private ones and property backing fields in read-only mode. If you don’t see a field here, it means that it has a type not supported by the serialisation system or it’s explicitly excluded (annotated with the NonSerialized attribute).

According to the documentation, the following types are supported at the moment:

• classes inheriting from UnityEngine.Object (this includes everything that inherits from MonoBehaviour, ScriptableObject, Component and EditorWindow, for instance)
• certain primitive types (bool, int, float, string etc.) but not necessarily all of them (e.g. uint and long only got support recently, so double check before use)
• enums
• classes and structs annotated with the Serializable attribute (however, these are treated with value semantics, see below)
• arrays and Lists of all the above (but not when nested, e.g. List<List<int>> won’t work!)

Also, fields cannot be static, const, or readonly, otherwise they will be invisible to the serialisation system.

Here’s a non exhaustive list of what’s not supported:

• interface types
• generic types
• standard library types
• delegates of any kind (including Action and Func)
• inheritance for types that aren’t derived from UnityEngine.Object
• coroutines (they are stopped during hotswapping and never reinstated)

Some of these limitations can be worked around, as we’ll see.

Value vs. reference types

C# has a clear distinction between value and reference types: structs and primitive types have value semantics and are stored inline, while classes are reference types that live on the heap. The serialisation system has such a distinction too, but the line is drawn elsewhere. If a type is a descendant of UnityEngine.Object, it is serialised as a reference. In every other case (value type or custom class marked as Serializable) the type has value semantics in the context of serialisation. This has some consequences:

• the class is serialised inline: even if there’s one physical object, its data is going to be copied for each reference to it
• references are lost: every reference to an object becomes a reference to a separate object (with identical contents) after hotswapping
• null cannot be represented: it is replaced by an instance whose fields are initialised with default values
• infamously, if a type marked as Serializable is recursive, it would normally induce infinite inlining, so Unity has an internal limit of 7 nesting levels when storing such data (in my opinion it should completely disallow such usage)

One notable gotcha to remember: null strings turn into empty strings during hotswapping. Generally speaking, you should never check values of these types for null if you want to support hotswapping. Methods like string.IsNullOrEmpty come in handy in many situations.

Customising the (de)serialisation step

Unity introduced the ISerializationCallbackReceiver interface, which provides a hook into the serialisation process. The purpose of this hook is to convert unsupported types into supported ones and back, thereby increasing the expressiveness of the system. For instance, Dictionary is not supported by default, but it can be converted into a list of keys and values (which the system already knows how to deal with), then it can be reconstructed from those lists after reloading.

Read the documentation carefully, because this interface comes with a lot of warnings. It’s especially important to keep in mind that it might be executed on a different thread, so even innocent looking things like checking Application.isPlaying might lead to an error. In the end, however, it’s a powerful tool that lets us work around some of the above mentioned limitations.

Supporting static fields

The easiest way to deal with static fields is to wrap them in properties with logic to retrieve or reconstruct the instances on demand:

private static Stuff _instance;

public static Stuff Instance
{
    get
    {
        if (_instance == null)
        {
            _instance = ...; // new / FindObjectOfType / LoadResource etc. as applicable
        }
        return _instance;
    }
}

For instance, if this object lives on the scene, it will be reconstructed by the serialisation system, then the reference will be retrieved on demand the first time it is referenced through the getter.

Should you find the need to save something static out of band or perform some other operation before hotswapping, one rarely mentioned weapon is the AppDomain.CurrentDomain.DomainUnload event. Once I needed to use it to be able to cleanly hotswap a 3rd party library, but it might be useful for other purposes as well. Use your imagination.

Supporting generic types

The serialisation system simply ignores generic types other than List<T>. However, there’s a simple trick to make them visible: convert them into a concrete type with inheritance. For example, if you created some pooled list type called PooledList<T> while optimising your memory usage, you can use it in the following manner:

[Serializable] public class PooledGameObjects : PooledList<GameObject> { }

public PooledGameObjects Pool;

Of course, this assumes that PooledList supports hotswapping, i.e. either it uses supported data structures and types internally, or it implements the serialisation callback interface to massage its data into the right shape.

If the class doesn’t have a default constructor, you need to include it in the definition, e.g. if you have to pass a capacity to the list:

[Serializable] public class PooledGameObjects : PooledList<GameObject>
{
    public PooledGameObjects(int capacity) : base(capacity) { }
}

The alternative to introducing a custom type for each usage is to add extra logic through the serialisation callback. In my opinion, the above trick is a lot more lightweight (just one line), and the new type allows the code to be a bit more self-documenting. One thing to keep in mind is that marking the original generic class as Serializable is useless, because this attribute is not inherited.

You can also create hotswap-friendly wrappers around otherwise unsupported generic data structures like Dictionary and use this trick to avoid writing a callback for every class that needs to use it.

Finally, this trick also works with the otherwise unsupported case of nested lists, just define a type for the inner list:

[Serializable] public class MaterialList : List<Material> { }

public List<MaterialList> MaterialLists;

Dealing with UI callbacks

There are two ways to subscribe to UI events: manage listeners in code or wire up methods in the scene. The latter option automatically supports hotswapping, because it explicitly stores the reference of the target object and the name of the method. However, I tend to avoid this feature, because it’s hard to manage these references and they silently break as soon as I rename the method in question. Sadly, listeners registered in code cannot be persisted, since they are delegate types.

One solution in this case is to register listeners in OnEnable and remove them in OnDisable, both of which are called during hotswap. If this is not applicable to your use case (say, you want your listener to be persistent, even when the object is not active), then you can use a custom serialisation callback to set a flag that tells your program to reconstruct the listeners. I try to use this as a last resort simply because OnEnable and OnDisable are readily available and guaranteed to run in the main thread where I can access engine functionality immediately. Note that Awake and Start are not called after hotswapping.

Coroutines

Hotswapping causes all coroutines to stop, and their state is not saved. There’s really no easy solution here; just keep this limitation in mind. Usually you don’t need hotswapping to cover all areas of your application, just enough to be able to iterate on certain elements without having to restart everything, so this is not necessarily a problem unless you rely heavily on coroutines in your gameplay logic.

Supporting interfaces and readonly fields

My recommendation is to avoid these features for fields that you want to preserve during hotswapping. If you still want to use them, then the serialisation callback is the only sensible way to do so, unless you know they are repopulated through a different mechanism (e.g. constant initialiser or constructor). This is a clear trade-off: you can use more features, but you pay by having to write more code.

References and further reading

• Unity Manual: Serialization
• Unity Reference: ISerializationCallbackReceiver interface
• Unity Reference: SerializeField attribute
• Unity dev blog: introduction to serialization (2012)
• Unity dev blog: a fresher look on serialization (2014) (This is a bit longer version of the manual entry.)

Hello Gergely,

thank you so, so much for writing up your article! And also for writing it in such a short time. I can't imagine how much effort you put into trial-and-error and experimentation to collect all these information. Why is there no official documentation regarding hotswapping anyway? I hope you advertise your article a lot.

Following are some thoughts that came to mind when first reading your article:

everything below is based on blog posts

If you have some useful links to more information please provide them.
Is there an official documentation other than the one about "script serialization"?

auto-properties (i.e. those with a backing field)

I know auto-properties: public int foo { get; set; }
But what are "backing fields"?

Also, fields cannot be static, const, or readonly, otherwise they will be invisible to the serialisation system.

What does invisible exactly mean? It makes sense not to serialize const and readonly but do they keep their original values or are the set to their default values too?
What about static constructors?

references are lost: every reference to an object becomes a reference to a separate object (with identical contents) after hotswapping

Do you have a canonical code snippet on how to implement manual reference handling? I imagine a naive implemention is not to difficult but it probably has some not so obvious edge cases that someone else already thought of (circular references, null values etc.).

You can also create hotswap-friendly wrappers around otherwise unsupported generic data structures like Dictionary and use this trick to avoid writing a callback for every class that needs to use it.

Interesting, so I can avoid writing serializers for Dictionaries if I just define concrete types for every combination, like so:

[Serializable] public class DictionaryStringInt    : Dictionary<string, int   > { }
[Serializable] public class DictionaryStringFloat  : Dictionary<string, float > { }
[Serializable] public class DictionaryStringString : Dictionary<string, string> { }

Thanks again! I will try to adopt our code base accordingly.
Cheers, Gerold.

If you have some useful links to more information please provide them.

To be honest, I can only remember the Unity dev blog as a source. It’s a strangely underappreciated topic. In any case, it’s a good idea, and I’ll dig up the links to the relevant posts.

Is there an official documentation other than the one about "script serialization"?

There might be; I’ll take a second look.

I know auto-properties: public int foo { get; set; }
But what are "backing fields"?

Backing fields are just the private fields (with some mangled names) generated internally for such properties where the data is actually stored.

What does invisible exactly mean? It makes sense not to serialize const and readonly but do they keep their original values or are the set to their default values too?

Invisible as in completely ignored. Those fields might as well not exist at all as far as the serialiser is concerned. Both const and readonly fields will assume whatever values are specified in the newly compiled code.

What about static constructors?

They are executed after hotswap on demand as usual. Really, everything is just a direct consequence of how the whole process works on the high level (as outlined in the ‘basic flow’ section): the program’s state is saved, then a new program starts and the previously persisted state is loaded into it.

Do you have a canonical code snippet on how to implement manual reference handling? I imagine a naive implemention is not to difficult but it probably has some not so obvious edge cases that someone else already thought of (circular references, null values etc.).

It’s really up to you. You might want a specialised or a general solution based on your use case. Unity’s solution relies on assigning an integer instance id to each live object, then using these ids to reconstruct the reference graph with new instances. I personally never implemented reference serialisation on top of Unity’s system.

Interesting, so I can avoid writing serializers for Dictionaries if I just define concrete types for every combination

Well, apart from the fact that Dictionary is not understood by the serialiser, yes. The documentation itself suggests a way to implement a hotswap friendly dictionary as two separate lists for key and values (you can keep an actual dictionary at runtime if you convert to this other representation during serialisation).

Thanks again! I will try to adopt our code base accordingly.

You’re welcome! It’s not really possible to provide 100% support for everything, but so far we’ve managed to keep it close enough so we get a lot of mileage out of hotswapping.

Well, apart from the fact that Dictionary is not understood by the serialiser, yes.

Ah. Apparently we have to define concrete types AND implement dictionary serialization.
What I did now is to define SerializableDictionarys and implement them like so:

[Serializable] public class DictionaryStringInt : SerializableDictionary<string, int> {}

arrays and Lists of all the above (but not when nested, e.g. List<List> won’t work!)

This of course also applies to nested dictionaries. Thus Dictionary<string, List<int>> can be used like so:

[Serializable] public class ListInt : List<int> {}
[Serializable] public class Dictionary<string, List<int>> : DictionaryStringListInt : SerializeableDictionary<string, ListInt> {}

Some pitfalls I stumbled upon when implementing hot-swapping:

• longs don't get serialized(!).
  This is especially interesting when working with DateTimes which are [Serializable] in .NET but not in Unity(?). My solution was to stringify-and-parse them using the ISerializationCallbackReceiver. Any ideas on how to better support DateTime without writing a lot of support code are highly appreciated!

    using System;

    public class MyData : ISerializationCallbackReceiver
    {
        public DateTime datetime = DateTime.UtcNow;

        void Awake
        {
            Debug.Log( datetime.ToString() );
        }

        // hot swapping support
    #if UNITY_EDITOR
        [SerializeField] private string _datetime;

        public void OnBeforeSerialize()
        {
            _datetime = datetime.ToString(); // WARNING: timezone information gets lost here so only rely on UTC or serialize "Kind" manually too!
        }

        public void OnAfterDeserialize()
        {
            datetime = DateTime.Parse( _datetime );
        }
    #endif
    }

• ScriptableObject have to be in their own file
  I kept a lot of classes together in one file. Instead of implementing my own reference serialization mechanism I thought "just convert it to a ScriptableObject and be done with it". Which works, except that the class has to be in it's own file(?). Otherwise you will get null when calling ScriptableObject.CreateInstance<T>().

• Delegates are not supported

  Here’s a non exhaustive list of what’s not supported: delegates of any kind (including Action and Func)

This sucks! Our whole architecture relies on passing delegates down to child objects which only allow them to change things in the parent object selectivly. Using ISerializationCallbackReceiver did not work because apparently OnAfterDeserialize is also called when starting up the engine the first time and it wouldn't allow calls like:

// THIS DOESN'T WORK!
public void OnAfterDeserialize()
{
    if( substate != null ) { substate.InitDelegates( OnChangeFoo, OnChangeBar ); }
}

My solution was to just put it into OnEnable of the parent object.

longs don't get serialized(!).

Hmm, I had the impression that long started to be supported recently. But if this is not the case, you could just serialise it as two ints instead of going through a string representation.

Any ideas on how to better support DateTime without writing a lot of support code are highly appreciated!

You can always wrap it in a struct that handles the hotswapping for you, and use that everywhere. It’s almost like a Haskell newtype...

ScriptableObject have to be in their own file

Yeah, if a type is derived from Unity’s Object, it should be in a separate file for the sake of peace. But supporting classes/structs can go in the same files without causing issues.

Using ISerializationCallbackReceiver did not work because apparently OnAfterDeserialize is also called when starting up the engine the first time

This is pointed out in their documentation: these callbacks are invoked whenever (de)serialisation happens, which includes instantiating a prefab that contains them. It’s a really basic and ubiquitous mechanism in Unity.

just convert it to a ScriptableObject and be done with it

We just ran into a problem with ScriptableObject on Windows Phone and our JSON Serializer. Apparently the Windows Phone runtime works a little differently (Reflections probably) and Unity doesn't like it when ScriptableObjects are instantiate with new instead of CreateInstance<>. Putting a #if UNITY_EDITOR around : ScriptableObject worked around the problem.

Yeah, that’s how you have to instantiate ScriptableObjects, otherwise strange things might happen.

Multi-dimensional and jagged arrays of primitive types are not supported. These might be considered "nested" as well.
int[][] jaggedArr;
int[,] multidimArr;

Is it possible to add serialization functionality to a sealed class? I'm using a collection library which has all classes sealed and find myself adding a ISerializationCallback to every class I use one of these collections. It works but it's a lot of boilerplate code.

I'd just wrap that class in my own, pretty much like any other unsupported class like Dictionary.

Looks like 2017.1 introduced DidReloadScripts which should be a simpler solution if you don't need to do work before reloading.

differences to keep in mind about DidReloadScripts compared to implementing the ISerializationCallbackReceiver interface:
-the method marked by the attribute has to be static or it doesn't work (no error, just does nothing), which is annoying because.. well it is static, so you can't get non-static class fields not even with a singleton thing, because.. well the singleton isn't (re)instanced yet! since all statics break right after serialize!!
-it does not work in generic classes (I tried to use it for a Manager<T> thing but the error clearly says it is not possible to use DidReloadScripts inside a generic class itself.)
-It is in UnityEditor, so if you want to use it in normal scripts you have to remember to put it between #if UNITY_EDITOR (...) #endif or it is not gonna let you build your game.

All those bad things are with DidReloadScripts, implementing ISerializationCallbackReceiver instead does fix all those problems though!

Thanks for all the info! As previously mentioned, Unity should really consider unifying the documentation on this topic. You've done a great service of doing as much as you have 😄

I'm wondering though... Is there some way we can leverage the hotswap system for a fully dynamic save/load game system? Let's say in a Unity project, I took care to ensure the game is as "hotswappable" as possible. Then could I add a save and quit feature, where everything's serialised as they would be during hotswap and saved somewhere? Later in the future the user could then load and the hotswap process would pop everything back in place?

I'd probably have some version control in place to watch out for broken states, but would this be possible at all? I've been thinking for months about how to go about "true" saving and loading, to no avail...

My guess is that it won't be possible because I assume all the hotswap functionality lives in UnityEditor land. Hopefully I'm wrong!

Thank you, this is still helping people in 2021.

I want to know about static classes/functions without state.
Will these be hot reloaded? It seems like yes for Unity? since it reloads the assemblies?
But I know in other systems (like Eclipse/Java) this isn't always the case.

Unity's hot swapping functionality can be useful for small projects, but it's impractical for larger projects due to all the existing limitations and constraints. To improve iteration times in our own project, we developed a C# Hot Reload extension for Unity which we recently made public.

It works similar to Unity's default hot swap functionality but with faster compiling and no domain reload, allowing for iteration times of < 1 second rather than the 2 minutes it would normally take (for projects as large as ours).

Basically we made a C# compiler which only compiles the specific method that changed (which is very fast, milliseconds), and then we swap just that function in playmode/editmode.

If you're curious, the extension can be found on our website hotreload.net and on our Unity Forum post.