Skip to content

Instantly share code, notes, and snippets.

@ShivanshRakesh

ShivanshRakesh/gsoc-20-progress.md

Last active October 29, 2020 14:53
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ShivanshRakesh/eff0a92eae189b64eb4a0b20f06bfbcf to your computer and use it in GitHub Desktop.
Save ShivanshRakesh/eff0a92eae189b64eb4a0b20f06bfbcf to your computer and use it in GitHub Desktop.
Download ZIP
GSoC'20 Progress
Raw
gsoc-20-progress.md

GSoC'20 with OpenMRS

GSoC at OpenMRS

Project Title: Add support for FHIR Narratives
Mentors: Ian Bacher, Herbert Yiga, Suthagar Kailayapathy
Student: Shivansh Rakesh
Project Link: Wiki Page Link

Progress Reports

Final-Report

Overview:

FHIR (or Fast Healthcare Interoperability Resources) is a standard for exchanging electronic health records. It describes elements (called resources) and an API (or application programming interface) for implementation of the same. FHIR Narratives are human-readable representations of resources, in the form of XHTML fragments. The project aims at adding support for FHIR Narratives to the FHIR2 Module of OpenMRS.

As a part of this project, we have successfully implemented support for FHIR narratives, along with addition of other useful features like localisation of narratives and custom narrative support.

Objectives:

Following were the objectives of the project:

  • Create FHIR narratives for all resources defined by OpenMRS FHIR Module (COMPLETED)
  • Develop a framework to support implementation-driven overrides for FHIR narratives (COMPLETED)
  • Add support for localization of the generated narratives (COMPLETED)

Brief Implementation Details:

Our implementation uses Thymeleaf Template Engine for rendering of narratives. The narrative templates are basically XHTML fragments using Thymeleaf Standard Dialect. The narrative basically gets associated to the resource as the text element. The implementer can then use it in the desired way, using the getText() method.

Narrative templates for all the supported resources: AllergyIntolerance, Condition, DiagnosticReport, Encounter, Location, Medication, MedicationRequest, Observation, Patient, Person, Practitioner, RelatedPerson, ServiceRequest, Task were created. Also, narrative template for the PractitionerRole resource was created, considering its future scope of addition.

Support for narrative override or custom narratives was added through an option of specifying a global property narrativesOverridePropertyFile for custom property file location which would contain locations of narrative overrides for specific resources. Since it is a global property, no changes are required in the code of FHIR Module for adding custom narratives.

The localisation support was added by implementing OpenmrsMessageResolver and using it to populate the narratives with localised text through keys stored in message files (messages_*.properties).

Here is an example of how a narrative looks like: image Example used for the above narrative: https://www.hl7.org/fhir/allergyintolerance-example.html

Contributions:

All contributions during this project were done in the OpenMRS FHIR2 Module, the details of which are given below:

GitHub repository:

https://github.com/openmrs/openmrs-module-fhir2

Public Talk thread for project related discussions:

https://talk.openmrs.org/t/gsoc20-add-support-for-fhir-narratives

Issues and Pull Requests:

  • FM2–193: Generate default narratives provided by HAPI FHIR (link)
    Status: FIXED
    Pull Requests: PR#194 [MERGED]
  • FM2–194: Create default narratives for resources (AllergyIntolerance, MedicationRequest, Condition) (link)
    Status: FIXED
    Pull Request: PR#198 [MERGED]
  • FM2–207: Create default narratives for resources (Encounter, Observation, Location) (link)
    Status: FIXED
    Pull Request: PR#207 [MERGED]
  • FM2–208: Create default narratives for resources (Person, Practitioner, ServiceRequest) (link)
    Status: FIXED
    Pull Request: PR#219 [MERGED]
  • FM2–230: Create default narratives for resources (RelatedPerson, Task, PractitionerRole) (link)
    Status: FIXED
    Pull Request: PR#232 [MERGED]
  • FM2–243: Fhir2 build errors in windows environment (link)
    Status: FIXED
    Pull Request: PR#279 [MERGED]
  • FM2–249: Improve content and layout of narratives (link)
    Status: FIXED
    Pull Request: PR#238 [MERGED]
  • FM2–251: Add support for overriding default narratives (link)
    Status: FIXED
    Pull Request: PR#257 [MERGED]
  • FM2–252: Add support for relative path with prefix ‘openmrs:’ for narrative templates (link)
    Status: FIXED
    Pull Request: PR#255 [MERGED]
  • FM2–280: Add support for localisation of narratives (link)
    Status: FIXED
    Pull Request: PR#278 [MERGED]
  • FM2–281: Add checks for empty properties in narrative templates (link)
    Status: FIXED
    Pull Request: PR#273 [MERGED]
  • FM2–295: Narrative generation failing for MedicationRequest resource (link)
    Status: FIXED
    Pull Requests: PR#275 [MERGED], PR#288 [MERGED]
  • FM2–297: Create default narratives for resources (DiagnosticReport, Medication, Patient) (link)
    Status: FIXED
    Pull Request: PR#287 [MERGED]
  • FM2–299: Add necessary Javadoc to methods and classes (link)
    Status: FIXED
    Pull Request: PR#291 [MERGED]

Resources:

Following are some of the resources for reference:

Future Works:

  • An additional planned task for the project was implementation of search feature for FHIR resources using _text parameter. This would be one of the future tasks related to this project.
  • Another nice addition could be implementation of a web-based client for viewing the FHIR narratives of resources.

Project Overview and Demo:

https://www.youtube.com/watch?v=mZw0qKcNK9I

Thoughts on GSoC:

My experience in working on this GSoC’20 project with OpenMRS was simply awesome. It was one of the best experiences I’ve ever had. Not only I got a chance of contributing to such a big real-world project, but also I got to learn a lot throughout the program.

I would really like to thank Ian Bacher who always supported me and guided me through the project and helped me a lot in resolving the blockers I faced. I would also like to thank Herbert Yiga and Suthagar Kailayapathy who also mentored me and were always there for support throughout the project.

Also, I would like to thank the entire OpenMRS community which has always been so supportive. Be it solving issues, suggesting improvements or even finding issues with the implementation, the OpenMRS community has always been there for help. I would definitely say that it is one of the most active and supportive opensource communities, and that I take pride in being its member.

Finally, I would thank Google for conducting GSoC and providing this great opportunity for working in such a great opensource community on such a big, real-world project.

Week-1

Summary:

This week, being the very first one of the GSoC Coding Phase, marked the start of development (officially). Overall, this first week went good. There were few blockers, some of which got resolved with a bit of look-up, and others got resolved with the help of my project mentors. There was a slight change of approach, after discussion with my mentors. The week basically involved work on generating HAPI Default Narratives and creating default narrative templates for few resources.

Progress - Week 1

The ticket FM2–193 has been completed. Currently working on FM2–194. Default Narrative Templates for the following have been created:

  • AllergyIntolerance
  • MedicationRequest
  • Reference, ContactPoint, Boolean, Duration Also, an example object for each has been created for testing.

Tickets:

  • [Completed] FM2–193: Generate default narratives provided by HAPI FHIR (link)
  • [In Progress] FM2–194: Create default narratives for resources (AllergyIntolerance, MedicationRequest, Condition) (link)

Pull Requests:

  • [Merged] FM2: PR#194 (link)
  • [Under review] FM2: PR#198 (link)

Change of Approach

Earlier, the plan was to create a method which would take in a resource and return a narrative for that. This method would be called in the appropriate translators in order to generate the narratives and set them as the text element of the resources. But to make things simpler, we are now setting Thymeleaf Narrative Generator for the FhirContext using the setNarrativeGenerator() method while initialising the FhirRestServlet (as suggested by Ian Bacher). This way, we’re leaving it on HAPI to do the job of creating those narratives for the resources. Our job is to create templates for the narratives. Issue(s) Faced

  1. When to set the DeaultThymeleafNarrativeGenerator and when to set the CustomThymeleafNarrativeGenerator? There are some resources for which HAPI FHIR provides the Default Narrative Templates. We definitely want to use these narratives. That is when we use the DefaultThymeleafNarrativeGenerator. But, not all resources have narrative templates provided by HAPI FHIR. For those, we need to make our own default narrative templates and use the CustomThymeleafNarrativeGenerator to generate the narratives using those templates. But the problem was, how to decide when to set which narrative generator, since our plan was to set the narrative generator while initialising the REST Servlet.

Solution: To solve this, we decided to just use the CustomThymeleafNarrativeGenerator for generating both HAPI Default Narratives and OpenMRS Default Narratives, by including property files (containing narrative template paths for resources and datatypes) for both.

  1. Displaying Reference Elements in the Narratives Many resources have one or more elements of type Reference class. The problem was how to display them in the narratives.

Solution: We’re displaying the elements as their reference values with links in the form of http://localhost:8080/openmrs/ws/fhir2/R4/<reference>. In case they have a display element, we use that for containing the link. Here is how the narrative for a reference looks:

<th:block th:if="${resource.display}">
  <a th:text="${resource.display}" th:href="'http://localhost:8080/openmrs/ws/fhir2/R4/' + ${resource.reference}"></a>
</th:block>
<th:block th:unless="${resource.display}">
  <a th:text="${resource.reference}" th:href="'http://localhost:8080/openmrs/ws/fhir2/R4/' + ${resource.reference}"></a>
</th:block>

Week-2

Progress - Week 2

The second week of the GSoC period ended. This week went as it was planned. Default narratives for the following resources were created:

  • Encounter
  • Condition
  • Observation
  • Location
  • Generic datatypes used in the above (eg. SampledData, etc.) Also, the tests were restructured and class organisation was improvised.

Tickets worked on:

  • [Under final review] FM2–194: Create default narratives for resources (AllergyIntolerance, MedicationRequest, Condition) (link)
  • [Under review] FM2–207: Create default narratives for resources (Encounter, Observation, Location) (link)

Pull Requests:

  • [Final Review Pending] FM2: PR#198 (link)
  • [Draft Under Review] FM2: PR#207 (link)

Issue Faced:

Inconsistency in timezone during tests.

In the tests for Narrative Generation, we had to assert that the generated narrative and the expected narrative matched. In some narratives, there are elements which depend on the timezone used. By default, the tests considered the local timezone because of which they failed in such assertions.

Solution: The issue was solved by using the following code in BaseFhirNarrativeTest class to keep the timezone consistent during tests (OpenMRS Talk discussion link).

protected static TimeZone defaultTimeZone;
@BeforeClass public static void setupTimeZone() {
    defaultTimeZone = TimeZone.getDefault();
    TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
}
@AfterClass public static void resetTimeZone() {
    TimeZone.setDefault(defaultTimeZone); 
}

Week-3

Progress — Week 3

The third week of the GSoC period ended. This week went as it was planned. No major issues were faced. Default narratives for the following resources were created:

  • Person
  • Practitioner
  • ServiceRequest
  • Generic datatypes used in the above

Tickets:

  • [Under Final Review] FM2–207: Create default narratives for resources (Encounter, Observation, Location) (link)
  • [Under Review] FM2–208: Create default narratives for resources (Person, Practitioner, ServiceRequest) (link)

Pull Requests:

  • [Merged] FM2: PR#198 (link) [FM2–194 closed]
  • [Under Final Review] FM2: PR#207 (link)
  • [Under Review] FM2: PR#219 (link)

Issue(s) Faced

This week had no major issues to deal with. There was just a minor issue with the narrative properties file. Switching its name to narratives.properties ended up with the tests using just HAPI’s narrative file when the resource location was set as classpath:narratives.properties. This was solved by a bit of reorganisation of the resource files. The narrative resources were moved to org/openmrs/module/fhir2/ and the properties file location was set to classpath:org/openmrs/module/fhir2/narratives.properties, thereby solving the issue.

Week-4

Progress — Week 4

The fourth week of the GSoC period ended. This week went as it was planned. No major issues were faced. Default narratives for the following resources were created:

  • RelatedPerson
  • Task
  • PractitionerRole
  • Generic datatypes used in the above

Tickets:

[Under Review] FM2–230: Create default narratives for resources (RelatedPerson, Task, PractitionerRole) (link)

Active Pull Requests:

  • [Under Review] FM2: PR#207 (link)
  • [Under Review] FM2: PR#219 (link)
  • [Under Review] FM2: PR#232 (link)

Issue(s) Faced

This week had no major issues to deal with. Though, there were some changes suggested by Ian (discussion link):

Change in property table layout: Currently, there is one main hapiPropertyTable which contains most of the properties in a narrative. But, in cases where there are multiple sub-properties in a property with multiple entries, there are separate subPropertyTable for those. In order to maintain the order of the property listing, currently the main table has to be broken to give space for these sub-tables. This creates an inconsistency in the alignment. Thus, the suggestion is to make these sub-tables reside in the rows of the main-table itself.

Addition of details to properties of type Coding: Currently, for the properties of type Coding, only the ‘Value’ attribute is displayed in the narratives. The suggestion is to display the ‘Detail’ attribute associated with the value too, to make it more descriptive.

These changes shall be covered in the coming week, on top of the commits in the above listed PRs once they get approved, as these changes would also require changes in the expected narrative of all resources.

Week-5

This week had the evaluation for Coding Phase-1. I’ve passed the evaluation and I’m all set to give my best in the rest of the GSoC period too.

Apart from the evaluation, this week I wrapped up all tasks of Coding Phase-1. As such no challenges were faced this week, and it went out pretty smoothly.

Ticket worked on:

  • FM2-249 : Improve content and layout of narratives. (link)

Pull Request:

  • [Under Review] PR#238 (link)

Week-6

Progress — Week 6

This was the first week of coding phase-2 according to the GSoC timeline. This week involved discussions with my mentors regarding the objectives for coding phase-2 and the approach that is to be followed. There has also been progress towards the same.

Objectives for Coding Phase-2:

Following are the major objectives for Coding Phase-2:

  • Add support for custom narratives.
  • Support relative paths prefixed with openmrs: for location of narratives and properties file. For example, the relative path openmrs:configuration/Patient.html would point to the location <OPENMRS_APPLICATION_DATA_DIRECTORY>/configuration/Patient.html.

Approach to be followed:

Support for custom narratives could be provided by giving the implementer an option to specify a properties file containing the location of narrative templates which he/she wants to override. The FHIR Module would give a higher priority to these overrides to serve the intended purpose. In order to support relative paths prefixed with openmrs:, we would need to implement our own narrative generator with the required changes in order to support openmrs: prefixed relative paths.

Tickets:

  • [PR Under Review] FM2–251: Add support for overriding default narratives (link)
  • [In Progress] FM2–252: Add support for relative path with prefix openmrs: for narrative templates (link)

Pull Requests:

  • [Under Review] PR#252 (link) [for FM2–251]

Challenges Faced:

One of the main concerns was that for specifying custom narratives, the implementer should not be needing to fork the FHIR repository, make changes to it and rebuild it. Instead, there should be an option to do it without modifying the code of FHIR Module. What we decided upon for solving this was to have a Global Property for the location of the narrative overrides property file. The FHIR Module would check this global property and give the properties file a higher priority while setting the narrative generator.

For supporting the relative paths, the basic change that is needed is an additional check for openmrs: prefix in the static method NarrativeTemplateManifest.loadResource(). Once this is changed, we would be requiring the Narrative Generator to use this modified method. But, there are many private variables and methods which cannot be overridden, because of which some classes cannot be simply inherited. Hence, parts of these have to be re-implemented.

Week-7

Progress — Week 7

This week went well and in line with the objectives which were planned.

  • A basic working version of a modified narrative generator which supports file paths prefixed with openmrs: was implemented.
  • Changes were made in PRs for FM2–249 & FM2–251 based on the code reviews.

Tickets:

  • [PR Under Review] FM2–251: Add support for overriding default narratives (link)
  • [Draft PR Under Review] FM2–252: Add support for relative path with prefix openmrs: for narrative templates (link)

Pull Requests:

  • [Under Review] PR#252 (link) [for FM2–251]
  • [Draft Under Review] PR#255 (link) [for FM2–252]

Challenges Faced:

Most of this week was focused on implementation of the modified narrative generator to support the openmrs: prefix and simplification of the same. The basic version taking HAPI’s narrative generator as the base was implemented successfully. After that, the focus was on the simplification part. The main issue which is being faced is that the class NarrativeTemplateManifiest has a private constructor because of which it cannot be inherited. Also, different methods in NarrativeTemplateManifest call many non-public methods of NarrativeTemplate like setTemplateName(), addAppliesToProfile(), etc. which makes it a bit difficult to simplify the NarrativeTemplate class.

Week-8

Progress — Week 8

This week went quite well and in line with the objectives which were planned.

  • The implementation of the modified narrative generator was finalised and the required changes were made. Tests were added for ensuring coverage of the same.
  • PRs for FM2–249 & FM2–251 got merged.

Tickets:

  • [Closed] FM2–249: Improve content and layout of narratives (link)
  • [Closed] FM2–251: Add support for overriding default narratives (link)
  • [PR Under Review] FM2–252: Add support for relative path with prefix openmrs: for narrative templates (link)

Pull Requests:

  • [Merged] PR#238 (link) [for FM2–249]
  • [Merged] PR#252 (link) [for FM2–251]
  • [Under Review] PR#255 (link) [for FM2–252]

Issues Faced:

Simplification of the modified narrative generator was the main topic of discussion. But it turned out that since NarrativeTemplateManifest has a private constructor, it can’t be inherited. Thus making the simplification task difficult.

Solution: The solution we settled upon was to get the necessary parts of the code from HAPI FHIR and add a license notification for the same. The necessary changes were made to them to fulfil our requirements. Also, the classes were dressed to resemble the OpenMRS conventions and unused methods were removed.

Week-9

This week had the evaluation for Coding Phase-2. I've passed the evaluation and I'm all set to give my best in the next phase too. Apart from the evaluation, this week all the tasks of Coding Phase-2 got wrapped up. As such no challenges were faced this week, and it passed pretty smoothly.

Ticket worked on:

  • [Closed] FM2–252 : Add support for relative path with prefix 'openmrs:' for narrative templates (link)

Pull Request:

  • [Merged] PR#255 [for FM2–252] (link)

Objectives for Coding Phase-3

The initial plan for Coding Phase-3 includes the following objectives:

  • Working on localisation of default narratives.
  • Making the templates reference to a narrative.css file.
  • Improvements in the default narrative templates.
  • Implementing search for FHIR resources using _text parameter.
  • Standardisation of class names for the narrative template (which more or less has been done).
  • Writing Javadocs for the methods and refactoring the code.

Week-10

This week went well with the following progress:

  • Made improvements in the default narrative templates.
  • Updated objectives for Coding Phase-3.
  • Worked on an issue related to narratives.
  • Worked on localisation of narratives.

Tickets:

  • [In Progress] FM2–280: Add support for localisation of narratives (link)
  • [PR Under Review] FM2–281: Add checks for empty properties in narrative templates (link)
  • [PR Under Review] FM2–295: Narrative generation failing for MedicationRequest resource (link) [Thanks to Varun Gupta for reporting this issue]

Pull Requests:

  • [Under Review] PR#273 (link) [for FM2–281]
  • [Under Review] PR#275 (link) [for FM2–295]

Updates in the Objectives:

  1. One of our objectives was to make the narratives reference to a css file. But, for this the css styles had to be made inline. Since in the narratives which get generated, only the div and status elements get associated to resource, and the head part is ignored. Thus, we thought of dropping this objective. Thus, discussing it with one of my mentors, Ian Bacher, we thought of dropping this objective.

  2. Another objective was to avoid rendering of the empty property table in the narratives in case of an empty resource. This was done by checking for the empty properties and the resource itself using the .empty() method. But, for the resources like Patient, the default narrative is provided by HAPI itself, so empty property tables couldn't be avoided if we use those. So, we thought of creating our own default templates for even the resources for which HAPI provides one.

Issues Faced:

  1. One of the issues I faced is that since we are autowiring messageSource in our message resolver, on running tests, messageSource remains null. For that, we might use @Mock annotation for the messageSource but then the messages won't be populated in the narrative templates. 

  2. Also, I faced some issues in fetching the messages from the messages.properties file. Specifically, on doing something like this:

OpenMRSMessageResolver resolver = new OpenMRSMessageResolver(null);
AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(resolver.getClass());
OpenMRSMessageResolver app = context.getBean(resolver.getClass());
log.info(app.getMessageSource().getMessage("fhir2.title", new Object[] {}, Locale.getDefault()));

I got the error org.springframework.context.NoSuchMessageException: No message found under code 'fhir2.title' for locale 'en_GB', even if the property existed in the messages.properties file.

Week-11

This week was quite productive with the following progress:

  • Worked on localisation of narratives.
  • Created narrative templates for Patient, Medication and DispenseRequest resources.
  • Worked on an issues related to narrative generation.

Tickets:

  • [Draft PR created] FM2–280: Add support for localisation of narratives (link)
  • [PR Under Review] FM2–297: Create default narratives for DiagnosticReport, Medication, Patient (link)
  • [PR Under Review] FM2–295: Narrative generation failing for MedicationRequest resource (link)
  • [PR Merged] FM2–281: Add checks for empty properties in narrative templates (link)
  • [PR Merged] FM2–243: Fix test errors on Windows (link)

Pull Requests:

  • [Draft PR under review] PR#278 (link) [for FM2–280]
  • [Under Review] PR#288 (link) [for FM2–295]
  • [Under Review] PR#287 (link) [for FM2–297]

Issues Faced:

The issues from last week got resolved. There was an issue with the messageSource in tests (which would be autowired in the running instance) because of which messages were not getting fetched from the messages.properties file. Solution: In tests, we are now using ReloadableResourceBundleMessageSource configured as per our needs.

The issue which came up this week is that the message source is not fetching the messages from the existing api/src/main/resources/messages.properties file. Instead, it is fetching only from omod/src/main/resources/messages.properties file.

Week-12

This week was the last week of the Coding Phase. This week was a good one. All our main project objectives got almost wrapped up and the changes got approved.

The following progress happened this week:

  • Added Javadocs to the methods and classes.
  • Added localisation support for narratives of Patient, Medication and DispenseRequest resources.
  • Wrapped up localisation of resources.
  • Fixed issues related to narrative generation.

Tickets:

  • [PR Under Review] FM2–280: Add support for localisation of narratives (link)
  • [PR Under Review] FM2–299: Add necessary Javadoc to methods and classes (link)
  • [PR Under Review] FM2–295: Narrative generation failing for MedicationRequest resource (link)
  • [PR Merged] FM2–297: Create default narratives for DiagnosticReport, Medication, Patient (link)

Pull Requests:

  • [Changes Approved] PR#278 (link) [for FM2–280]
  • [Changes Approved] PR#288 (link) [for FM2–295]
  • [Under Review] PR#291 (link) [for FM2–299]

Issues Faced:

The issues from last week got resolved. This week went pretty smooth, with no major issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment