commit 1b3276d2271518ef0e5bc5d5083fa029b101fb14
Before this refactor, the Organization model had a method, get_transfer_org which hardcoded the transfer capability between organizations.
# in user_accounts/models/organization.py
def get_transfer_org(self):
"""If this organization is allowed to transfer to another
organization, this shoud return the other organization they are
allowed to transfer submissions to.
"""
if self.slug == constants.Organizations.ALAMEDA_PUBDEF:
return self.__class__.objects.get(
slug=constants.Organizations.EBCLC)
elif self.slug == constants.Organizations.EBCLC:
return self.__class__.objects.get(
slug=constants.Organizations.ALAMEDA_PUBDEF)
return NoneAs we add more organizations, we may wish to set up more relationships. First I want to simplify this code by adding a transfer relationship between organizations.
class Organization(models.Model):
...
can_transfer_applications = models.BooleanField(default=False)
transfer_partners = models.ManyToManyField(
'self', symmetrical=True, blank=True)The can_transfer_applications flag allows us to check whether to perform additional queries that would be necessary for orgs that transfer applications.
Now, in our organizations.json fixture, we can modify EBCLC and Alameda Pub Def to set this transfer relationship.
{
"model": "user_accounts.organization",
"fields": {
"name": "East Bay Community Law Center",
"can_transfer_applications": true
}
},
{
"model": "user_accounts.organization",
"fields": {
"name": "Alameda County Public Defender's Office",
"can_transfer_applications": true,
"transfer_partners": [
["East Bay Community Law Center"]
]
}
},Finally, we can now query transfer capability and specific partners in a cleaner, more flexible way:
# boolean flag
if org.can_transfer_applications:
# additional query
transfer_options = org.transfer_partners.all()commit 8a4ea09b827f636d8fcd5bc7e20f3809716d2bf2
This new Transferred status type should not appear in the status types listed when an org user is sending a new status update. Currently, the is_active boolean flag on status types will control whether or not they show up in the list of possible status types. But semantically, it means something different, it indicates a status type that is no longer used. Therefore, we will add a new flag that specifically excludes status types from the options, called is_a_status_update_choice. The cost of adding a boolean flag is minimal, so adding such a specific flag is not a concern.
# template_option.py
class TemplateOption(models.Model):
...
is_a_status_update_choice = models.BooleanField(default=True)Now we can add a fixture for the Transferred status type:
{
"model": "intake.statustype",
"pk": 8,
"fields": {
"label": "Transferred",
"display_name": "Transferred",
"template": "Your {{county}} application is now being handled by {{to_organization_name}}. {{to_organization_next_step}}",
"help_text": "",
"slug": "transferred",
"is_a_status_update_choice": false
}
},We can also enumerate our hardcoded status types in our code as a series of constants.
# in intake/models/status_type.py
# all of these are defined in the `template_options.json` fixture
CANT_PROCEED = 1
NO_CONVICTIONS = 2
NOT_ELIGIBLE = 3
ELIGIBLE = 4
COURT_DATE = 5
OUTCOME_GRANTED = 6
OUTCOME_DENIED = 7
TRANSFERRED = 8Currently, during a transfer, we actually delete the old application. What if, instead, we created a new application and flagged the old one, adding a new transfer model that links between the transfer status update and the newly created application?
Using was_transferred_out, we can clearly mark old apps as having been transferred. The _out portion should clarify which org was responsible for the transfer (the one tied to this application).
class Application(models.Model):
...
was_transferred_out = models.BooleanField(default=False)It's likely that in the long run, we will want to give additional flags or properties to the relationship between an applicant and the org handling their application. For example, if we allow orgs to collaborate, or if we explicitly mark orgs as having abandoned an application, or if an applicant does not wish to work with an org further. In this picture, it would make sense to have flags on an Application object.
Our application transfers can be a simple link between the status update that created them (which has the transferred status type), and the new application to the target organization.
class ApplicationTransfer(models.Model):
"""A record that links a 'Transferred' status update to the new application
created from the transfer.
"""
status_update = models.OneToOneField(
'intake.StatusUpdate', related_name='transfer',
on_delete=models.CASCADE)
new_application = models.ForeignKey(
'intake.Application', related_name='incoming_transfers',
on_delete=models.PROTECT)
reason = models.TextField(blank=True)
def __str__(self):
return '{} because "{}"'.format(
self.status_update.__str__(),
self.reason)When we perform the transfer, it's important that we no longer delete the old application, but instead set the boolean flag. The full transfer process is 4 queries total, saving 3 new objects (status update, application, application_transfer) and updating the existing application object.
def transfer_application(author, application, to_organization, reason):
"""Transfers an application from one organization to another
"""
transfer_status_update = models.StatusUpdate(
status_type=models.status_type.TRANSFERRED,
author=author,
application=application
)
transfer_status_update.save()
new_application = models.Application(
form_submission=application.form_submission,
organization=to_organization)
new_application.save()
transfer = models.ApplicationTransfer(
status_update=transfer_status_update,
new_application=new_application,
reason=reason)
transfer.save()
application.was_transferred_out = True
application.save()Adding these additional queries to the transfer step means that the cost is placed on a step that occurs rarely, rather than at the time of display or retrieval, which occurs more often.
The dilemmas of querying for the application index drove much of the solutions above.
Pagination is an important factor here, because it means that we want to return one ordered query, that is then paginated, as opposed to two queries and then sorting them into one combined list.
The receiving organization's appp index query should look essentially the same:
def get_submissions_for_org_user(user):
return models.FormSubmission.objects.filter(
organizations=user.profile.organization
).prefetch_related(
'applications',
'applications__organization',
'applications__status_updates',
'applications__status_updates__status_type',
).distinct()For displaying the latest status, we have a serializer post-processing step that preselects the latest status from all the status updates for an application:
class ApplicationFollowupListSerializer(serializers.ModelSerializer):
organization = serializers.SlugRelatedField(
read_only=True, slug_field='slug')
status_updates = MinimalStatusUpdateSerializer(many=True)
class Meta:
model = models.Application
fields = [
'organization',
'status_updates'
]
def to_representation(self, *args, **kwargs):
data = super().to_representation(*args, **kwargs)
sorted_status_updates = sorted(
data.get('status_updates', []), key=lambda d: d['updated'],
reverse=True)
latest_status = \
sorted_status_updates[0] if sorted_status_updates else None
data.update(latest_status=latest_status)
return dataWe would want some similar step for the receiving organization that would flag incoming applications as having been recently transferred.
class ApplicationIndexSerializerWithTansfers(ApplicationFollowupListSerializer):
organization = serializers.SlugRelatedField(
read_only=True, slug_field='slug')
status_updates = MinimalStatusUpdateSerializer(many=True)
incoming_transfers = TransferSerializer(many=True)
class Meta:
model = models.Application
fields = [
'organization',
'status_updates',
'incoming_transfers'
]Assuming a TransferSerializer that includes organization_name and local_date, we can now clearly flag incoming transfers:
<td>
{# Status #}
{% if submission.latest_status %}
{%- elif submission.incoming_transfers -%}
Transferred from {{
submission.incoming_transfers[0].organization_name
}} {{
humanize.naturaltime(submission.incoming_transfers[0].local_date)
}}
{%- else %}
New
}
{% endif %}
</td>Here's what the template now looks like in the app index for the org who sent the transfer.
{%- for submission in submissions %}
{%- if submission.was_transferred_out %}
{%- include "submission-transfer-display.jinja" %}
{%- else %}
{%- include "submission-display.jinja" %}
{%- endif %}
{%- endfor %}In the app index, we paginate form submissions, based on their date_received. This means that if an application is transferred after a month in one org's hands, it will appear in the receiving org's index, but may appear far down the list.
This would add a separate application index where transfers would specifically appear. Perhaps we would have separate incoming and outgoing transfer tabs. This implicitly creates a separate queue based on the type of incoming or outgoing status.
This would be a complex query, that would have to aggregate across tables and compare. Feels like an overly complex solution for the sake of avoiding refactoring.
This, coupled with a data migration
Adding some sort of explicit queue feels like a better long term solution.