React Native Upgrade

gistfile1.txt

Merging
-------
- Remove REACT_NATIVE_VERSION and configuration.all from android/build.gradle and keep the solution is newer version of RN
- There are patches in master which should be applied only if it is required. remaining patches should be removed

More notes
----------
- ListViewHelper
- spread is broken in realm and spread cannot be used on model classes
- Radio/Checkbox value cannot be array as native-base doesn't support it correctly (it gets confused)

Key Changes already discussed which remain from the first spike
-------------------------------------------------------------------
- auto linking of packages feature in react native (MainApplication.java)
- some components packages have been moved to deprecated packages. we are referring to those deprecated packages right now.
- some packages we depend on require patching because they have not been kept uptodate by the library maintainers. these are very minor changes, as one can see in the patches folder. for this we are using (https://www.npmjs.com/package/patch-package).
- node is upgrade to 16 in avni-client, latest LTS version in. model and web app are in 10 still.
- the packager is called metro. you can reload app from metro, and launch RN pop up menu. the logs are also available here.
- the debugger still doesn't work for me as I get some errors
- metro logs a lot of warnings which come with stack trace making the app sluggish on emulator
- i could not get flipper to work. it recognises only the emulator but not the RN app. also flipper extension for realm is the recommended way for viewing realm db.
- NetInfo api has changed, have adopted code accordingly

Realm and Our Model
-------------------
- Upgraded to realm 11-rc-1 as 10x crashes for errors for which realm 11 throws meaningful exceptions
- the previous realm changes that we discussed using last approach, have been removed.
- Schema object is registered with Realm and not the class. Schema name should be passed to db.objects and not the class (earlier we could pass both).
- now we have proxies for Realm, RealmResults, RealmList, RealmObject (our model classes). the implementation of proxies can be found here - https://github.com/avniproject/avni-models/tree/d54b6fcd2c8503ba5d5b55b871d93cbcaa8e7bb5/src/framework. The corresponding interfaces of realm are available here - https://www.mongodb.com/docs/realm-sdks/js/latest/Realm.html
- RealmListsProxy is further proxied by JavaScript Proxy which handles length and indexor [] operator. it can be seen here - https://github.com/avniproject/avni-models/blob/1e7d8372a33cd04b029a32291b8b7831077fc164/src/framework/RealmResultsProxyHandler.js
- Some of the methods from realm interface are not used by us, so they may not have been implemented in our proxy yet, though have mostly tried to implement everything.
- Our models look like this - https://github.com/avniproject/avni-models/blob/455c981d95c667af5720227f2129af916ffa04cd/src/Individual.js#L54 (check primitive, list, and entity handling)
- We had to implement custom toJSON as realm toJSON doesn't work anymore. They have a fix it seems which works only in hermes version of JS engine. Code can be seen here - https://github.com/avniproject/avni-models/blob/d54b6fcd2c8503ba5d5b55b871d93cbcaa8e7bb5/src/PersistedObject.js#L41. toJSON is required sometimes when we post the models objects to the server.
- lodash binds to Array.prototype functions which mutates array ('pop', 'push', 'shift', 'sort', 'splice', 'unshift'). this implies that if lodash is used on RealmListProxy which is an array as it extends from it, our overridden methods will not be called. hence we cannot use these methods to mutate. we usually don't use lodash to mutate realm lists, but if it is done then we can simply use array methods available to us which have been overridden.
- ListView.DataSource errors out when we give it JS Proxy object instead of array as it uses typeof probably. Hence in such cases we have to use asArray method in RealmListProxy. If we encounter prop types of array giving this warning then we can do the same.
- We should not declare properties in JS class like anymore - https://github.com/avniproject/avni-models/blob/30cdfffa23eb961055875cadd454a133040cfc4f/src/SubjectMigration.ts as these will come from getters and setters
- identifier related, subject migration, news, videoTelemetric, a few comment related classes, entityApprovalStatus, extension, ruleFailureTelemetry, DraftEncounter, MediaQueue models do not have the new approach. I have left this intentionally so that others get a change to define them. You can add live templates (at end of this file) in Intellij to do this fast and error free.

JavaScript issues to be aware of
--------------------------------
- When importing classes from within the module (say importing ProgramEnrolment in Individual) one should use export from the local source and not from index.js which is meant for exporting to the outside world). Node 16 gives error by getting stuck in a loop which ends up getting created. I saw mostly this in tests. Outside tests when they happen you may get unclear error like - "super is undefined".
- Avoid importing transactional entities from non-transactional entities. Or usually in lower functional layers from the higher layers. This also can cause above issue.
- Type script lets not use it in models, life is complicated as it is. I have a removed a few instances, but not all. Feel free to remove.
- Lastly, in case you come across, individual.affiliatedGroups. Note that this is done wrong. affiliatedGroups is not present in db but is part of state of Individual. This is the only place I have seen this.

Native base
-------------
- How NB handles some objects have changed e.g. Checkbox and Radio. The code has been updated in some places not everywhere.
- CHSContent and CHSContainer has been changed to not use NB too much.
- We are still quite dependent on NB so have not ventured to remove it as mostly we can live with it. NB has changed a lot and maybe worth continuing with it as NB, or RNPaper, or some other library is usually used.

React Native
------------
- You may find that some components have been removed so we may have to do the needful (e.g. DatePickerAndroid has been removed)
- I am not sure about this, but you may need to upgrade to react native cli to v8

What I have tested
------------------
- new model against avni-web and rules-engine unit tests only
- Avni client (full download sync without the entities for which models haven't been changed, basic search subject test, create subject without a form, post subject to server, entity sync status). Some other screens may be partially working.

To Do (new stories to be created)
---------------------------------
[MUST]
- Search subjects, all functional scenarios
- Subject registration with multiple form pages (functional only, though you may have fix bare minimum UI issues)
- Update remaining model classes and get the unit tests for model, avni-client and rules-config to pass.
- Create/edit other entity types - encounter, enrolment, group/family
- View reports and dashboard, run against an organisation that has them (functional, though you may have fix bare minimum UI issues)
- Functionally check and fix checklists/graph screens
- Functionally fix all remaining screens under more
- UI styling fixes across the app
- Merge avni-client and avni-model to master (squashing in between commits will be better than merging each interim commit)
- Restart.restart not working
- All functionality in web app (note there is no UI library change here hence only one story)

[SHOULD]
- upgrade rules-server to latest model
- Remove warnings that come with stack trace in metro logs
- Figure out how to view realm database using flipper or some other option

[COULD]
- Switch to hermes engine (upgrade realm to rc2 which is hermes release candidate)
- Image Picker Patch
- Upgrade to final version of realm-11
- Remove patches if the library maintainer have fixed the issue.
- Lodash proxy

To go in documentation on readme
--------------------------------
- Diagram of proxy structure (there are some nuances that are not obvious from the code. our proxies are a mixture of proxy and gateway. gateway because they acts as boundary between the application layer and realm objects).
- all the details above in realm section



LIVE TEMPLATES
-------------
for entity collection properties
--
get $prop$() {
    return this.toEntityList("$prop$", $entity$);
}

set $prop$(x) {
    this.that.$prop$ = this.fromEntityList(x);
}


for sub-entity properties
--
get $prop$() {
    return this.toEntity("$prop$", $entity$);
}

set $prop$(x) {
    this.that.$prop$ = this.fromObject(x);
}


for primitive properties
--
get $prop$() {
    return this.that.$prop$;
}

set $prop$(x) {
    this.that.$prop$ = x;
}