Skip to content

Instantly share code, notes, and snippets.

@chdorner
Last active March 23, 2017 15:56
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save chdorner/cdda116d013d939107ba8327f0533bb2 to your computer and use it in GitHub Desktop.
Save chdorner/cdda116d013d939107ba8327f0533bb2 to your computer and use it in GitHub Desktop.
Nipsa annotation storage

Nipsa annotation storage

We currently have a nipsa boolean field on the user table. But now we build the functionality to nipsa individual annotations, instead of users. There are two different options of how we can store this data. The first one is to just add a nipsa column on the annotation table. The second option is to go back to how nipsa used to be stored, as a separate nipsa table with a user_id, and now with another annotation_id.

Another point that came out of a nipsa API discussion is that in the future we should probably limit the user nipsa flag to specific groups, thus allowing group moderators to flag a user, but only in their group and not across the whole platform (including across authorities).

I will now list pro/cons of these two options.

NIPSA column on the annotation table

I'm thinking of the following changes:

  • We add a migration and model property for the new nipsa boolean column on the annotation table.
  • The search index presenter will set the "nipsa": true/false item in the JSON payload that we send to ElasticSearch based on that field.
  • We move the nipsa filtering out of the nipsa package as it is now, and move it into the memex (soon h) search package.

Basically, this would mean that we're making memex (soon the h annotation storage) aware of what nipsa is and how it works, instead of the current situation where nipsa is a separate component which hooks itself into the search code.

Pros:

  • Easier to implement based on the needs for nipsa v2 / moderation.
  • No additional work to unify the user nipsa flag storage (caveat; see next item).
  • We could get rid of the hooks in the search code which allows current nipsa to hook itself in.

Cons:

  • Harder to add additional information why an object was nipsa'd (which user did it, a free-form text field for a reason).
  • Separation of concerns - the search will know about yet another concept that it currently doesn't.
  • If we need to limit user nipsa to specific groups, we would need to awkwardly store that on the user table. As described above, we might have to limit the user nipsa to specific groups and not across all authorities and groups like now.

Separate NIPSA table

I'm thinking of the following changes:

  • We add a new migration and model for a nipsa table, with an annotation_id column.
  • The nipsa package will hook another AnnotationTransformEvent subscriber to add the "nipsa": true/false item into the JSON payload that we send to ElasticSearch.

We can keep the part of nipsa that filters out nipsa'd annotations in the nipsa package as these changes only affect the storing part, not the searching part.

In the future, we can unify the user and annotation nipsa parts by:

  • Add a new migration and model property for the new user_id column on the nipsa table.
  • Change the AnnotationTransformEvent subscriber to check in the table, rather than the user.nipsa column.
  • Remove the user.nipsa column.

Note: by adding two separate user_id and annotation_id we can make full use of foreign keys which delete entries when an annotation or a user gets deleted, and we can also add a database constraint which ensures each nipsa row has either a user_id or an annotation_id but never both of none.

Pros:

  • We can easily extend nipsa to include more information like the user who nipsa'd the user or annotation, or a free-form text field for a reason.
  • nipsa will stay it's own mostly separate component without leaking complexity into other parts of the code base.
  • Limiting user nipsa to specific groups in the future could result in adding another column on the nipsa table, and then have multiple nipsa entries (one per group)

Cons:

  • More effort to implement.
@nickstenning
Copy link

Thanks for the write-up here. I think that when it's put like this it's fairly clear that we want to have this data stored in a separate table or tables.

We probably will want both:

  1. The ability for site admins (i.e. our staff) to shadowban users site-wide.
  2. The ability for group administrators/moderators to shadowban users within groups they administer/moderate.

As such, I'm not sure we necessarily want to try and fit that all into one table. I'm very happy to look at a proposed schema which might put it all in one table, but it certainly sounds like we might want to consider keeping "hiding annotation" and "shadowbanning users" as separate tables in the database, even if the end result (setting "hidden": true in the search index) is the same.

@dhwthompson
Copy link

Yup: I agree with everything @nickstenning just said.

Thanks, @chdorner, for taking the time to write this up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment