You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There is a piece of work particularly challenging in the Archivematica I18N project that this document tries to break down into smaller units of work. Archivematica's workflow data is currently kept in a relational database managed by the Dashboard. It includes information about the microservices, including human-readable strings (labels) that are shown in the user interface. These strings need to be translated into other languages other than English.
The workflow data is currently considered static data not editable by the user. Keeping it in the database makes it hard to maintain (e.g. changes require database migrations) and hard to translate. This document suggests pulling this workflow data our of the database and store it in a simple JSON file - simplest way I can think of but with the goal of making much easier to introduce new way to describe our workflow rules in the future. This is considered an implementation detail specific to MCPServer that we want to encapsulate - in other words, we want to hide this mechanism from the consumers: MCPClient and Dashboard. MCPServer will expose a new API based on the gRPC protocol.
gRPC's Python implementation offers us the possibility to build scalable API client and servers without extra gateway interfaces like gunicorn or Apache/mod_wsgi. gRPC implements its own C shared library used in many of their platform-specific libraries like C++, Ruby, Python, PHP or NodeJS.
chain-links.json (generated with devtools) includes the data from all the models listed above. It's a proof of concept.
Workflow models used in MCPServer
Once the workflow data is encoded in JSON, we'll need a thin layer to read from it. The following is the list of cases in the MCPServer where workflow data is accessed. Understanding how the current models are used will help to shape the API of the module replacing it.
archivematicaMCP.py
WatchedDirectory: simple all()
jobChain.py
MicroServiceChain: simple get() without FKL
jobChainLink.py
MicroServiceChainLink: simple get(id=?) with FKL(currenttask=>TaskConfig)
They are all models frequently used from Dashboard and/or MCPClient.
UserProfile
UnitVariable
File
Job
SIP
Task
Transfer
Database functions databaseFunctions.{logTaskCreatedSQL,logTaskCompletedSql,lobJobCreatedSQL,createSIP} (imported from archivematicaCommon) used in the MCPServer interact with the models above.
The Job processing model keeps copies of seome of these values in Job.jobType and Job.microserviceGroup. We'll look into that later. The Job should likely instead reference to the corresponding chain link.
Usage of workflow models in other microservices
Notice that we're ignoring cases where processing models are used.
This is where we generate the processing configuration form, which involves mostly the MicroServiceChoiceReplacementDic and MicroserviceChainChoice models.
Solution: use new RPCs to populate forms. I'll build a RPC to retrieve details from any chain link (MSCL) regardless its type and configuration. The RPC should be able to serialize embedding different message types.
Also, this is the place where we use MicroServiceChoiceReplacementDic to store the configuration of DIP upload (AT, ATK). StandardTaskConfig is used to store AtoM's configuration.
Solution: use DashboardSetting.objects.{get,set}_dict
Dashboard: components/api/views.py
The MicroServiceChainChoice model is used to extract the available choices for a specific transfer that needs to be approved. Once the right choice is determined, this view asks MCPServer to proceed with the choice via MCPClient.
Solution: use new RPC to approve transfers.
Dashboard: components/ingest/views_as.py
The MicroServiceChoiceReplacementDic model is used to extract the configuration parameters to set up the AS client.
Solution: use DashboardSetting.objects.{get,set}_dict
Dashboard: components/ingest/views_atk.py
The MicroServiceChoiceReplacementDic model is used to extract the configuration parameters to set up the ATK client.
Solution: use DashboardSetting.objects.{get,set}_dict
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters