Last active
January 6, 2023 10:00
-
-
Save nandilugio/0f1113880afe8074e5745e576b4ff32d to your computer and use it in GitHub Desktop.
Sync proposal
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## Sync framework ############################################################ | |
# To be added to models that can be part of a synced group of entities | |
# AR implementation. Other could be used if API is preserved. | |
module Syncable | |
attr_accessor :sync_group_id, :syncer | |
after_update do | |
SyncGroup.find(sync_group_id).touch! if changed.include?(@_ssynced_attrs) | |
end | |
def find_by_sync_group_id(id) | |
end | |
def self.synced_with(syncer_class) | |
syncer = syncer_class | |
end | |
def self.synced_attrs(attrs) | |
@_synced_attrs = attrs | |
end | |
# Registers the synced group of local instances mapping to a NS entity instance | |
# AR implementation. Other could be used if API is preserved. | |
class SyncGroup < ActiveRecord::Base | |
attr_accessor :sync_group_id, :syncer, :touched_at, :synced_at | |
def touch! | |
touched_at = Time.now | |
end | |
def mark_synced! | |
synced_at = Time.now | |
end | |
end | |
# Base for integrations | |
class Syncer | |
# "From now on, sync these instances as a X on the remote" | |
def sync(*attrs) | |
thing = find(*attrs) | |
group = group_dependencies(thing) | |
ns_thing = translate(thing) | |
create!(ns_thing, group) | |
end | |
# Sync all instances that have been updated since last sync | |
def update_all_pending! | |
group_ids = SyncGroup | |
.where(syncer: self.class) | |
.where("touched_at > synced_at") | |
.pluck(:id) | |
group_ids.each do |group_id| | |
update_group!(group_id) | |
end | |
end | |
private | |
def group_dependencies(thing) | |
group = new_group | |
sync_group_members(thing).each do |syncable_member| | |
syncable_member.sync_group_id = group.id | |
end | |
group | |
end | |
def new_group | |
SyncGroup.create!(syncer: self.class) | |
end | |
def update_group!(sync_group_id) | |
thing = find_by_sync_group_id(sync_group_id) | |
ns_thing = translate(thing) | |
update!(ns_thing) | |
end | |
# The following operations (CRUD) can be refactored to be done asynchronously: | |
# - Sidekiq: doesn't ensure order: pass thing.id and translate to ns_thing in | |
# worker to ensure last version is synced? probably not enough if we need to | |
# keep order of events. | |
# - Implement a queue: low throughput, postgres is enough. A sidekiq worker | |
# (unique? self-enqueuing?) can consume it to avoid a single-point-of-failure. | |
def create!(ns_thing) | |
create_remote(ns_thing) | |
group.mark_synced! | |
end | |
def update!(ns_thing) | |
update_remote(ns_thing) | |
group.mark_synced! | |
end | |
end | |
## Local model ################################################################# | |
class Thing | |
has_one :thing_dep | |
include Syncable | |
synced_with NetSuiteSomethingSyncer # See below (*) | |
synced_attrs :x, :y | |
end | |
class ThingDep | |
belongs_to :thing | |
include Syncable | |
synced_with NetSuiteSomethingSyncer # See below (*) | |
synced_attrs :x, :y | |
end | |
# (*) Thing and its deps map to a Something entity in NS | |
## Netsuite integration ######################################################## | |
# The following encapsulate NetSuite specific logic: | |
# - Network communication | |
# - Finding info on local model | |
# - Translation of that info to NetSuite's corresponding model | |
# Only needed if CRUD operations are done the same way for all NS objects | |
class NetSuiteSyncer < Syncer | |
def create_remote(ns_thing) | |
ns_thing.add | |
end | |
def update_remote(ns_thing) | |
ns_thing.update | |
end | |
end | |
# Thing and its deps map to a Something entity in NS | |
class NetSuiteSomethingSyncer < NetSuiteSyncer | |
def find(thing) | |
thing.includes(ThingDep) | |
end | |
def find_by_sync_group_id(id) | |
Thing.find_by_sync_group_id(id) | |
end | |
def translate(thing) | |
# Could use a NetSuiteSomethingTraslator | |
NetSuiteGem::Entity::Something.new({x: thing.x, y: thing.y + 1}) | |
end | |
def sync_group_members(thing) | |
[thing, thing.thing_dep] # Syncables | |
end | |
end | |
## Business logic (controllers,etc.) ########################################### | |
class ThingCreationService | |
def create(*attrs) | |
thing = Thing.new(*attrs) | |
thing.thing_dep = ThingDep.new(*attrs) | |
thing.save! | |
mail_user(thing) | |
NetSuiteSomethingSyncer.new.sync(thing) | |
end | |
end | |
# NetSuiteSomethingSyncer#sync registers instance for syncing and does NS create | |
# We could easily modify the proposal to defer the NS create | |
thing = ThingCreationService.create(...) | |
# Marks group as dirty | |
thing.thing_dep.update_attributes!(x: 345) | |
# This could be a recurrent worker, a self-enqueuing one, we could enqueue | |
# on-update, etc. | |
class NetSuiteSyncWorker | |
def perform | |
# Performs NS update | |
NetSuiteSomethingSyncer.new.update_all_pending! | |
end | |
end |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Yes, the transformation or
#translate
function will do exactly that, but in order to get thetouched_at
timestamp updated when thatBillingInfo
dependency changes, we need to keep track of those.In fact there is something I don't 100% like there but see no magic we can do to solve cleanly: The developer has the responsibility to be explicit about the dependencies that can change the output of the transformation function so there are 2 things that have to be manually synchronized while developing: the dependencies declared in
#sync_group_members
and the ones used in#translate
.I've also just noticed that the inclusion of
Syncable
already offers the info#sync_group_members
returns, so specifying that by hand is probably redundant. Maybe some magic there is not that difficult to implement but need to think about this a bit. Anyway, these are implementation details only. We'll solve these if we agree on developing this.Yes, in fact @Laura is bringing it back from the dead 👻 It's named
FiscalCustomer
now :p. See #4403.Nah man, this solution transcends NetSuite ;p heheh. In fact, it could be used for SalesForce or whatever.
Looking forward to see it!