Skip to content

Instantly share code, notes, and snippets.

@calvinmetcalf calvinmetcalf/census.md
Last active Dec 18, 2015

Embed
What would you like to do?

First off, this API is awsome and as someone who has had to slog through the assemply this data usually requires THIS IS AWSOME I mean you even have CORS set up already, I mean I've talked to people in government who think JSON is a security risk. Now as requested from national day of civic hacking, here are some ways you can make it even better.

Almost all of the issues I've had with this API relate to it breaking my expectations about how things work while still behaving in a technically correct manner. A governmental API should not be clever, most of these things by themselves are probably fine but taken together make for a frustrating experience developing, in no particular order:

  • Using a non-standard jsonp parameter. Most sites use 'callback' as the parameter for jsonp, you should probably not use a different one exclusively (there is no reason you can't use multiple ones)

  • It's a JSON API, the field metadata should probobly be in JSON (no reason it can't be in both XML and JSON).

  • By default the API should be permissive in what it accepts and ignore parameters it doesn't understand, pedantic parsing rules can be helpful in development but it's probobly a good idea to explicitly require it to be set.

  • The API should give an error when it doesn't have data instead of giving a giant stack of null values.

  • The data returned should be more idiomatic JSON, what we get back is an array of arrays with the first item being an array of field names,[[header1,header2],[value,value]..] this just feels like CSV turned into JSON without thought.

    • A more helpful version might have {headers:[header1,header2],rows:[[value,value]..]} aka the headers and rows separated into different arrays.
    • You could also add more details into the headers so instead of [header1,header2] you could have [{name:header1,type:int|str,descriptiveName:descriptive}..]
    • Or since the data is being gziped when it's being sent to the client it wouldn't be that much bigger to do headers in every row for true JSON objects. (last resort the second one is probobly best) data isn't being gziped.
  • Unlike CSV, JSON allows nesting, much of your data just screams for a nested representation, instead of

      [total,female,female18-35,female35-65, Male...]
    

    you could nest this stuff

      {
        total:x,
        bySex:{
          female:{
            total:x,
            18-35:x,
            35-65:x
          },
          male:{
            total:x,
            ...
          }
        }
      }
    
  • Your server is not setup in a very friendly way, this stuff needs to be gziped when it's sent (a 70k file is now 18k) and the etag header should be set as well so that data requested more then once doesn't have to be downloaded multiple times.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.