| // Every network maintainer gets an API key to authenticate POST, PATCH, DELETE requests on sensors and observations in their network | |
| // What a network maintainer would POST to /sensor-observation | |
| { | |
| "data": { | |
| "type": "sensor-observation", | |
| "api-key": "abc123mykey", // Assigned to maintainer of sensor network | |
| "attributes": { | |
| "time": "2016-02-29T18:39:30.519207", // The one attribute that we mandate. | |
| "observation": { | |
| // The maintainer of the sensor network is responsible | |
| // for making sure that these fields are consistent. | |
| // Not all need to be present in each observation, | |
| // but "temperature" in one reading needs to be comparable to "temperature" in another. | |
| "temperature": 12, | |
| "no2": 334 | |
| } | |
| } | |
| }, | |
| "relationships": { | |
| // Client needs to inform server which sensor this belongs to | |
| "reported-by": { | |
| "type": "sensor" | |
| "id": 3096 | |
| } | |
| } | |
| } | |
| // Let's say that POST created a sensor-observation object with ID 4567345 | |
| // Here's what GET sensor-observations/4567345 would look like. | |
| { | |
| { | |
| "data": { | |
| "type": "sensor-observation", | |
| "id": "4567345", // Globally unique ID generated by server | |
| "attributes": { | |
| "time": "2016-02-29T18:39:30.519207", | |
| "observation": { | |
| "temperature": 12, | |
| "no2": 334, | |
| ... | |
| } | |
| } | |
| "relationships": { | |
| "reported-by": { | |
| "type": "sensor" | |
| "id": 3096 | |
| } | |
| } | |
| }, | |
| "included": [{ | |
| "type": "sensor", | |
| "id": "3096", | |
| "attributes": { | |
| "location": [-87.9137261060048, 41.6462579944754], // Required for every sensor | |
| "description": "Anything special about this sensor?" | |
| "observation-attribute-metadata": { | |
| // Details particular to this sensor. | |
| "temperature": { | |
| "unit": "degrees Fahrenheit", | |
| "description": "Uses Acme Corp. chip with +- .5 degree accuracy." | |
| }, | |
| ... | |
| } | |
| "sensor-attributes": { | |
| // What else do clients need to know about this particular sensor? | |
| "height": 2.5, | |
| "direction": "NE", | |
| "installed-date": "" | |
| ... | |
| } | |
| } | |
| "relationships": { | |
| "network": { | |
| "type": "sensor-network", | |
| "id": 3 | |
| } | |
| } | |
| }, { | |
| "type": "sensor-network", | |
| "id": 3, | |
| "attributes": { | |
| "name": "Array of Things", | |
| "maintainer-email" "data.engineer@uchicago.edu", | |
| "description": "Brief description of this sensor network." | |
| "info-link": "https://arrayofthings.github.io/developers" | |
| // ... Whatever other mandated metadata we require ... | |
| "sensor-attribute-metadata": { | |
| "height": { | |
| "unit": "meters", | |
| "description": "How far off the ground is this sensor positioned on the CDOT light post on which it is mounted?" | |
| }, | |
| "direction": { | |
| "unit": "compass direction", | |
| "description": "Which direction is this node facing off of the light post? Options are N, NE, E, SE, S, SW, W, NW" | |
| }, | |
| ... | |
| } | |
| } | |
| }] | |
| } |
Internal storage, presented in pseudo-Rails ORM notation. DDL designates schema-upfront. JSON designates document storage.
@levyj sorry I missed this comment for so long. Thank you for the detailed feedback!
It would be more along the lines of wanting all observations, filtered by some combination of sensor, time, and observation type. Do you have any thoughts what that would look like?
That was a deliberate dodge on my part. So no, not yet, but stay tuned. I wanted to start by thinking through metadata, so that's why I went with the trivial example of requesting by ID, even though nobody will ever really query that way.
Being duly sensitive to bikeshedding, could there be anything to separating data from metadata?
First off, you're not bikeshedding at all. This is precisely the sort of feedback I was hoping for. As for the trouble with repetition with sending all of the sensor and sensor network metadata all the time, I think a good default behavior would be to leave out the included section of the response (so line 50 down, basically) and just include the relationship attribute that would let you query for metadata based on ID. I like the way you put the distinction between an API for the data and an API for the metadata. I was trying to go that way by splitting out the sensor-observations, sensor-nodes, and sensor-networks resources. You can think of the sensor-observations as the data and the other two as the metadata.
This, of course, requires knowing when to call the metadata API. As a thought, what if the data API had either a flag to indicate this was the first data observation since a metadata change or the identifier (observation ID if sequential or timestamp) of the first data observation after a metadata change? That could be a trigger to the consumer to call the metadata API. Of these two, I prefer the latter approach. The former has the risk of missing the flagged data observation and then being out of sync for an extended period of time.
I like that idea a lot. I hadn't considered how to signal to the client that the metadata has changed. As for whether it should be a flag or an automatic trigger to include the new metadata in the response, I also lean to the second. I imagine it's equally easy for a client program to neglect to check a flag as it is to not notice that updated metadata has been included in the response. But at least including it in the response automatically means that much less work for the developer.
In either case of the above, one obviously could get more complicated by flagging different types of metadata changes instead of a simple "something changed" signal. I do not have a strong opinion on that at this point. I could cook up use cases for making this really fancy but complexity has all sorts of cost -- not least, increasing risk of error as multiple entities try to stay in sync -- and there may be something to the idea of just being told to hit the metadata API for that sensor and either reload everything or check the elements I care about and deal with any changes found.
Interesting. That would be useful. By default I would go with the simpler approach but I'd be open to hearing arguments on why it's useful enough to justify the complexity.
As usual, a thought-provoking exchange. Thanks, Jon!
I think my main interest would be from the GET side but probably without knowing an observation ID. It would be more along the lines of wanting all observations, filtered by some combination of sensor, time, and observation type. Do you have any thoughts what that would look like?
Being duly sensitive to bikeshedding, could there be anything to separating data from metadata? It seems as if including both in every reply could involve a lot of repetition and not necessarily make it easy for the consumer (me) to detect changes in metadata. What if there were two APIs -- one to deliver data (used frequently) and one to deliver metadata (used infrequently)? This, of course, requires knowing when to call the metadata API. As a thought, what if the data API had either a flag to indicate this was the first data observation since a metadata change or the identifier (observation ID if sequential or timestamp) of the first data observation after a metadata change? That could be a trigger to the consumer to call the metadata API. Of these two, I prefer the latter approach. The former has the risk of missing the flagged data observation and then being out of sync for an extended period of time.
In either case of the above, one obviously could get more complicated by flagging different types of metadata changes instead of a simple "something changed" signal. I do not have a strong opinion on that at this point. I could cook up use cases for making this really fancy but complexity has all sorts of cost -- not least, increasing risk of error as multiple entities try to stay in sync -- and there may be something to the idea of just being told to hit the metadata API for that sensor and either reload everything or check the elements I care about and deal with any changes found.