So I ran into a lovely issue while porting MiqGroup#filters (text blob, serialized hash) to Entitlement filter columns (Postgres Array columns, divided): Postgres multidimensional arrays must be of equal length.
The problem with this is that we currently store filters like this:
[["/managed/environment/dev"],
["/managed/location/chicago", "/managed/location/london"],
["/managed/prov_max_cpu/1"],
["/managed/service_level/gold"]]
I was originally going to leave the format as is because it affects too much to change it.
But now I realize I can't do that (at least, not without just porting over a serialized text blob).
The current format is pretty fragile, plus with multiple columns now the prefixes (i.e. /managed/
) are pointless.
Maybe we should port this to a JSON column, an excellent tool for 'tagging'. Then...
{ "environment": ["dev"],
"location": ["chicago", "london"],
"service_level": ["gold"] }
Problems:
- These are just filters, not the actual tags themselves...so I'm guessing tags elsewhere will have to be changed. That could be a very, very large undertaking.
- My example is just for
tag_filters
("managed" filters); I can't really deduce what the format is for 'belongsto' yet. I'm guessing it could work similarly.
Ultimately, we must do tagging in a better way now IMO, as the next step is for ✨ Multiple entitlements ✨ which need to be able to make sense of 'merging' filters. (Right?)
That's a good point, though I hesitate and wonder what things make use of the format of the filters here (and how we'd have to go query off to that table, then). I also note (though this is my dev data, so might just be me...) that the taggings table seems to be only "managed" filters.
Your thought on the intermediate arrays makes sense to me, I don't understand why it was aggregated like that in the first place.
@lpichler was kind enough to show more data on the format of these things:
Talking with @jrafanie yesterday, we can remove a whole lot of complexity and parsing by doing JSON as I mentioned; it will also make future merging considerations easier with multiple entitlements. I can also provide a utility class to allow filters to be converted back to 'the old format' as needed so that this can be gradually moved over.