Data structure for the MS Transferor document

transferor_format.json

# OPTION A:
{"wf_A": {"timestamp": 0000
          "primary": ["list of transfer ids"],
          "secondary": ["list of transfer ids"]},
 "wf_B": {"timestamp": 0000
          "primary": [],
          "secondary": []},
}

# OPTION B:
{"wf_A": {"timestamp": 0000
          "primary": {"dset_1": ["list of transfer ids"]},
          "secondary": {"PU_dset_1": ["list of transfer ids"]},
 "wf_B": {"timestamp": 0000
          "primary": {"dset_1": ["list of transfer ids"],
                      "parent_dset_1": ["list of transfer ids"]},
          "secondary": {"PU_dset_1": ["list of transfer ids"],
                        "PU_dset_2": ["list of transfer ids"]},
 "wf_C": {"timestamp": 0000
          "primary": {},
          "secondary": {},
}

# OPTION C (the chosen one!) - it assumes we store all the transfer information within the same Couch document:
{"wf_A": [{"timestamp":000, "dataset":"/a/b/c", "dataType": "primary", "transferIDs": [1,2,3]},
           "timestamp":000, "dataset":"/a/b/c", "dataType": "secondary", "transferIDs": [4]}],
 "wf_B": [{"timestamp":000, "dataset":"/a/b/c", "dataType": "primary", "transferIDs": [1,2,3]},
           "timestamp":000, "dataset":"/a/b/c", "dataType": "parent", "transferIDs": [4,5,6]}],
 "wf_C": [],
}

# OPTION D - it assumes a new document is created for every request:
{"workflowName": "blah,
 "lastUpdate": 000,  # just as timestamp above
 "transfers": [{"dataset":"/a/b/c", "dataType": "primary", "transferIDs": [1,2,3], "campaignName": "blah2017", "completion": [0.0]},
               {"dataset":"/a/b/c", "dataType": "secondary", "transferIDs": [4], "campaignName": "blah2018", "completion": [0.0]},
               {"dataset":"/a/b/c", "dataType": "parent", "transferIDs": [4,5,6], "campaignName": "blah2017", "completion": [0.0]}]
}

Ok, let's hope nothing else changes. Let's proceed with option D then, one record/document per workflow.

@vkuznet Valentin, I added a completion parameter to the option D, such that we can persist the transfer completion every time it gets calculated (and persist it).