Skip to content

Instantly share code, notes, and snippets.

@zsfelfoldi
Created April 21, 2020 23:32
Show Gist options
  • Save zsfelfoldi/95a221d96a3ac89f15dca6f56551e779 to your computer and use it in GitHub Desktop.
Save zsfelfoldi/95a221d96a3ac89f15dca6f56551e779 to your computer and use it in GitHub Desktop.

Proposed NodeStateMachine specs

Persistence rules

  • a subset of the node state flags and fields can be persisted in a database
    • a flag is persistent if NodeStateFlag.persistent is set
    • a field is persistent if encode and decode are not nil
    • persistent flags and fields are created with a separate constructor
  • stored states are loaded during startup
  • in-memory states that have been changed since loading the stored version or newly created ones with no stored version are considered "dirty" until they are saved (again)
  • any node with no flags set is discarded from both the memory and the db immediately
    • fields (even persistent fields) are also discarded if all flags are reset
  • saving the state of a node is possible at any time, dirty nodes are saved during shutdown
    • the saved version contains persistent flags and fields only (in-memory state is not affected)
      • flags with an active timeout are not persisted
    • if the saved version does not have any flags set then it is not saved, previous saved entry is removed

Subscription rules

  • state subscription callbacks are called when the relevant flags of a node are changed
    • relevant flags are specified with a bit mask, the callback receives the relevant bits of the old and new state
  • field subscription callbacks are called when the relevant field of a node is changed
    • each subscription is sensitive to a single field
    • the callback also receives the state at the moment of the field change
  • callbacks may perform further state and field changes
  • state and field updates are guaranteed to only return after all cascading effects of the change have been finished
    • infinite loops, deadlocks and logic hazards should be avoided by design
  • nodes loaded during startup and saved/discarded at shutdown can also trigger callbacks
    • during startup nodes are assumed to go from a special state (OfflineState) to their persisted state
      • all nodes are loaded first
      • then all resulting callbacks are called
      • finally Start() returns
    • during shutdown they are assumed to go back to OfflineState
      • all callbacks are called (based on the node states before calling Stop())
        • changing node states and fields are not possible anymore
      • dirty nodes are all persisted
      • finally Stop() returns
    • fields are assumed to be nil before startup and after shutdown
      • field subscriptions are called with the state parameter being OfflineState

Timeout rules

  • timeout can be applied to any state flag of any individual node either when setting the flag or later
  • the flag is reset after the given period if it has not been reset before
    • resetting a flag removes the timeout
    • callbacks are called regardless of the reason of the flag being reset
  • adding a shorter timeout to a flag overwrites an existing longer one, adding a longer one has no effect
  • setting a flag again removes any previous timeouts
  • flags with an active timeout are not persisted

Exported functions and types

Static setup

  • func NewFlag(name string) *NodeStateFlag
  • func NewPersistentFlag(name string) *NodeStateFlag
  • func NewField(name string, ftype reflect.Type) *NodeStateField
  • func NewPersistentField(name string, ftype reflect.Type, encode func(interface{}) ([]byte, error), decode func([]byte) (interface{}, error)) *NodeStateField
type NodeStateSetup struct {
	Flags []*NodeStateFlag
	Fields []*NodeStateField
}

Initialization

  • func NewNodeStateMachine(db ethdb.KeyValueStore, dbKey []byte, clock mclock.Clock, setup *NodeStateSetup) *NodeStateMachine
  • func (ns *NodeStateMachine) AddStateSub(mask NodeStateBitMask, callback NodeStateCallback) error
  • func (ns *NodeStateMachine) AddFieldSub(field int, callback NodeFieldCallback) error
  • func (ns *NodeStateMachine) StateMask(flag *NodeStateFlag) NodeStateBitMask
  • func (ns *NodeStateMachine) StatesMask(flags []*NodeStateFlag) NodeStateBitMask
  • func (ns *NodeStateMachine) FieldIndex(field *NodeStateField) int
  • NodeStateCallback func(node *enode.Node, oldState, newState NodeStateBitMask)
  • NodeFieldCallback func(node *enode.Node, state NodeStateBitMask, oldValue, newValue interface{})

Operation

  • func (ns *NodeStateMachine) Start()
  • func (ns *NodeStateMachine) Stop()
  • func (ns *NodeStateMachine) SetState(node *enode.Node, set, reset NodeStateBitMask, timeout time.Duration)
  • func (ns *NodeStateMachine) AddTimeout(node *enode.Node, mask NodeStateBitMask, timeout time.Duration)
  • func (ns *NodeStateMachine) GetField(node *enode.Node, fieldId int) interface{}
  • func (ns *NodeStateMachine) SetField(node *enode.Node, fieldId int, value interface{}) error
  • func (ns *NodeStateMachine) Persist(node *enode.Node) error
  • func (ns *NodeStateMachine) Node(id enode.ID) *enode.Node
  • func (ns *NodeStateMachine) ForEach(require, disable NodeStateBitMask, cb func(node *enode.Node, state NodeStateBitMask))
@rjl493456442
Copy link

flags with an active timeout are not persisted

It's a very good point. Since we can always assume that after the specified time, the flags will be removed.
But a open question here: if the "timeout" specified is very large(e.g. 12hours, to be honest I'm not sure we have this usecase), how should we handle it?

Subscription rules

  1. Let's add an additional option for subscription: nodeID. Now we can only create the subscription for the whole set. However for the general usage, it's quite common to subscribe the specific node. And it's quite trivial to implement it.

  2. Should all subscription callbacks be called before "Start" and after "Stop"? E.g. caller might create lots of subs and it's annoying to receive the same "notification".

Maybe we can let caller specify whether the "OfflineState" is subscribed or not. If not then don't bother the subs.

APIs

Let's rename the AddStateSub and AddFieldSub to SubscribeState and SubscribeField

@zsfelfoldi
Copy link
Author

About long timeouts: I indeed do have a use case where long timeouts can happen and I would not want to discard the flag instantly. This is why I added timeout persistence, but now I think it is a special use case and it can easily be realized externally. What I want to do in this case is:

  • store the absolute time when the timer will expire (put it in one of the fields)
  • let the flag be dropped when persisting the node
  • check for the persistent timeout field in a startup callback, if it is set and it is still in the future then set the flag again with the proper timeout

@zsfelfoldi
Copy link
Author

About subs:

  • I think calling them during startup and shutdown is fine, this whole mechanism will not handle millions of nodes (if it will ever do that, it needs to be improved anyway). We iterate through persisted nodes anyway, checking it in a few subs whether it is interesting or not is negligible overhead right now.
  • subscribing to individual nodes would be possible but I am not sure we should do it. It would be another runtime thing that would complicate the state of individual nodes that it is supposed to simplify. It would allow different usage patterns but this component is deliberately limited in functionality. Now the subs are static and the nodes just have flags, fields and maybe timeouts. The main goal is to make possible interactions between other components easy to understand and test. I don't want to start using it in tricky ways for special cases because that would beat its purpose.
  • the name changes are fine, I'll do that :)

@rjl493456442
Copy link

Yes sure, for the individual node subscription it's unnecessary to implement now. Unless we do need the functionality.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment