george-hawkins/serialization.md

## serialization.md

      
    Raw
  

              serialization.md
            
          
    Serialization and loops in Kotlin

First we found there were loops in the things we were trying to serialize and so you have to tell things that are going to walk these structures (like Jackson or even plain old equals) how to handle these to avoid recursing infinitely. Fair enough.
For Jackson you do this by telling it has to generate references like so:
@JsonIdentityInfo(generator=ObjectIdGenerators.UUIDGenerator::class, property="@id")
class Type ...

This fixes the problem entirely for serialization. However it doesn't solve the problem for deserialization. E.g. if you've got:
class Foo(var bar: Bar?)
class Bar(val foo: Foo)

val foo = Foo(null)
val bar = Bar(foo)

foo.bar = bar

You've created a loop here - you can tell Jackson how to break the loops with references but it sure as hell going to work out how to reconstruct the things through a combination of constructor calls and setter calls.
So instead you have give Jackson a way to construct the objects without any constructor arguments and then build them up field by field so it doesn't get stuck in a loop wondering whether it should make a Foo first and pass it to the constructor of a Bar or vice-versa.
You can either do this yourself by mandating that your primary constructors never take any arguments and all properties are var and nullable (or have default values):
class Foo {
    var bar: Bar?
}
class Bar {
    var foo: Foo?
}

You then have to set all the properties explicitly after construction (or provide secondary constructors that set the properties explicitly).
And if you make things nullable (as in the example above) rather than using defaults then Kotlin will obviously force you into null checking that you know you don't need.
An alternative is the Kotlin no-arg plugin that generates constructors without arguments behind the scenes, that can be used by Jackson but don't mess up your world.
You should be able to write things like:
@NoArg
class Bar(val foo: Foo)

But due to a bug (logged here) you have to explicitly tell Jackson to ignore the primary constructor:
@NoArg
class Bar @JsonCreator(mode = JsonCreator.Mode.DISABLED) constructor(val foo: Foo)

Not great - but OK, that's how it is.
However there are a number of other issues with using the constructors generated by the no-arg plugin (that are unrelated to loop related issues).
First you should avoid open properties, e.g.:
open class Foo(open val name: String)
class Bar(override val name: String): Foo(name)

This is stupid (but we actually have examples of this in our code) but anyway you can construct slightly more sensible cases (in our case we make name a val in Type and override it with a var in some subclasses).
When you do this kind of thing you end up with two completely separate private variables in Foo and Bar that are both called name - usually though you'll never notice this.
However if Jackson reconstructs an instance of Bar using an auto-generated no-args constructor both Foo.name and Bar.name will start off as null, then Jackson will build things up a property at a time. However it will only set Bar.name - in JSON you can't have something like { "super.name": "xyz", "name": "xyz" } - Jackson just sets the lowest instance of name and doesn't go looking for shadowed versions higher up.
So even after construction Foo.name is null despite it appearing to be a non-nullable value - and this (if you press the right buttons) results in unexpected behavior later.
It's fairly hard to provoke that issue but an easier one to hit one is seen here:
class Foo(val name: String) {
    @JsonIgnore val foo = "foo $name"
}

Well clearly foo will end up with an unexpected value if you construct Foo with no-args and only set foo later. You might think the following could be a valid solution:
class Foo(val name: String) {
    @JsonIgnore val foo by lazy { "foo $name" }
}

I.e. only construct foo when someone really needs it (which will be well after Jackson or whatever has properly setup name).
But actually this shows up yet another problem with no-arg constuctors - if we look at what this translates to in Java we can see the problem:
public class Foo {
    private String name;
    private SynchronizedLazyImpl foo$delegate;

    public Foo(String name) {
        this.foo$delegate = new SynchronizedLazyImpl(() -> "foo " + getName(); });
        this.name = name;
    }

    public getName() { return name; }
    public getFoo() { return foo$delegate.getValue(); }
}

This all looks fine - however the problem is the construction of foo$delegate within our constructor with arguments. What kind of additional constructor does the no-args plugin generate? Something like this:
    public Foo() { }

I.e. not only is name not constructed but neither is our foo$delegate - if we later try to access foo we'll get an NPE as behind the scenes getValue() is called on a null foo$delegate.
You can side step this and avoid lazy by accepting the cost of constucting foo every time it's needed (as we do currently with some of our vtable values):
class Foo(val name: String) {
    val foo get() = "foo $name"
}

I thought this was a narrow issue to do with lazy and logged KT-27902 but actually it's bigger and there are less easily solved situations.
All these properties have the same problem:
class Foo(val name: String) {
    private val alpha by lazy { "alpha $name" }
    private val beta = mutableListOf<String>()
    private val gamma = 42
}

The problem is that even though you can do the same in Java too it's really just syntactic sugar for:
public class Foo {
    public Foo(String name) {
        this.foo$delegate = ...;
        this.beta = new ArrayList<String>();
        this.gamma = 42;
        this.name = name
    }
}

I.e. everything really happens with in the constructor. Things would be rosy if Java split things into a no-args constructor that setup things like alpha, beta and gamma and another one that invoked this and also setup name but it doesn't (and there are fairly obvious reasons not to do this - reflection would turn up more constructors than one expected for one thing).
The no-arg plugin runs after the bytecode is created (I believe) and just bangs in a completely no-work constructor, i.e. public Foo() { }, it's not about to deconstruct the bytecode for the existing constructor and see which bits are dependent on its input arguments and which are not (and therefore can be pulled out into a no-args constructor).
Note that the above properties are private and have no public getters, if they were public and marked with @JsonIgnore (as we do in various places) you'd see the same problem.
So in short one should avoid:

overriding properties
avoid lazy
avoid properties that are derived from constructor arguments, e.g. "foo $name" above.
avoid properties that require some construction but which are excluded from the serialized form.

So in the end such classes should just be super simple basic DAO classes - that makes one ask if the serialized type classes should be specifically for this purpose - like ClientValue etc. - that aren't meant to be worked with directly but are expected to be translated into workable counterparts after deserialization.