Skip to content

Instantly share code, notes, and snippets.

@BerkeleyTrue
Last active May 23, 2018 07:23
Show Gist options
  • Save BerkeleyTrue/f960946ba72814006651 to your computer and use it in GitHub Desktop.
Save BerkeleyTrue/f960946ba72814006651 to your computer and use it in GitHub Desktop.
Loopback Replication/Synchronization

Loopback Sync

Sync in Loopback requires Loopback in the client as well the server.

Loopback attaches a replicate method to each Model Class. Lets say I define a model called 'RemoteMat' in my loopback server. In my client loopback app, I define a local varient of this model called 'LocalMat'.

My app would use Browserify to package Loopback for the client.

On first app load, I would call a Bi-direction replication (or syncronization) method. This is currently not implament in Loopback but it is easy to create. I will go into more detials later.

This Bi-direction (or BiDi) first pulls down changes from the server, checks for any conflicting data. If no conflicts are detected, then a push replication is initiated, which pushes up any changes to the local storage that the client currently does not have. On completion, The data in local storage should mirror what is on the server.

Pull Replication

In the client:

loopback.models.RemoteMat.replicate(loopback.models.LocalMat, function (err, conflicts) {
  if (err) { /* handle errors and end function */ }

  if (conflicts) { /* handle conflicts */ }
  // At this point there should be no conflicts or conflicts should have been resolved.
});

You can see that replication takes two arguments, A loopack model class, and a node style callback.

Push Replication

loopback.models.LocalMat.replicate(loopback.models.RemoteMat, function (err, conflicts) {
  if (err) { /* handle errors and end function */ }

  if (conflicts) { /* handle conflicts */ }
  // At this point there should be no conflicts or conflicts should have been resolved.
});

You can see that both push and pull have the same structure. The model class that is used to call the replicate method is called the source model, the model class that is pulled in is called the target.

Bi-Directional Replication or Sync.

Bi directional replication involves calling both pull and push in sequence and resolving any conflicts that may arise in between.

loopback.models.RemoteMat.replicate(loopback.models.LocalMat, function (err, conflicts) {
  if (err) { /* handle errors and end function */ }

  if (conflicts) { /* handle conflicts */ }
  // At this point there should be no conflicts or conflicts should have been resolved.
  loopback.models.LocalMat.replicate(loopback.models.RemoteMat, function (err, conflicts) {
    if (err) { /* handle errors and end function */ }

    if (conflicts) { /* handle conflicts */ }
    // At this point there should be no conflicts or conflicts should have been resolved.
    // We are done!!
  });

});

Replication Algorithm

In order to work with replication, you must have model change tracking enabled. This in turn will create a new collection for each model with tracking. Going back to the above, the mat model in the server will have a corresponding collection Mat-Change. This is a full loopback model class.

This model class has certain properties, the interesting ones are prev and rev. These properties are hash values generated from the model class the Change model is representing. As you might have guessed, the rev property stands for Revision, while prev is the hash of the previous revision.

When a new instance of a model class is created, a Change model is created with the rev hash genereate and the Change type, another property of interest, set as create. The other possible Change types are updated, deleted, and unknown.

The replication algorithm has five main steps

  • getSourceChanges
  • getDiffFromTarget
  • createSourceUpdates
  • bulkUpdate
  • checkpoint

Lets take a look at this from the point of view of the client pushing up local changes (push replication) to the server db. Lets again use the models RemoteMat and LocalMat;

Get Source Changes

In this case, the source is the model class stored in the browsers localStorage, the LocalMat model. This method returns all the model Change instances.

Get Diff From Target

Now loopback will hand off those Change model instance data up to the server. Once there Those changes are compared to the change models on the server.

If the two change models are the same, nothing is done. However, if two change models are not the next step is to check if the two models are sequential changes, that is if the remote change from the server has its prev hash equal to client change model rev hash. If not, then the change is marked as a conflict.

If there is not conflict between the changes, the change is marked as a delta.

These conflicts and deltas are returned to the client.

Create Update From Target

This method takes all the deltas sent back from the server and creates an array of updates, which are objects that contain the update type, which is taken from change type, the change instance, and the current model data (in the case of a 'delete' type, no data is added).

Bulk Update

That array of updates created in the previous step is sent to server. There, the server applies the appropriate methods to the data.

Checkpoint

On completion of the buld updates on the server, the client will create a checkpoint.

//TODO: Explain Checkpoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment