routes.19.json

{
  "code": null,
  "name": "Inner Loop",
  "short_name": "19",
  "color": null,
  "associated_objects": {
    "Shark::Station": []
  },
  "itinerary": null,
  "path": null,
  "identifier": "routes.19",
  "description": "Inner Loop"
}

stations.BUS100W.json

{
  "identifier": "stations.BUS100W",
  "associated_objects": {}
}

vehicles.1708.json

{
  "saturation": 23,
  "code": null,
  "capacity": 60,
  "name": "1708",
  "route": "routes.5B",
  "associated_objects": {},
  "longitude": null,
  "heading": 0,
  "last_station": "stations.BUS624E",
  "latitude": null,
  "schedule_delta": null,
  "onboard": 14,
  "identifier": "vehicles.1708",
  "speed": 0,
  "next_station": "stations.BUS100E"
}

Route 19 currently doesn't exist on either CityBus or DoubleMap, which is odd. Maybe they're replacing it with something else? In any case, it probably went away from DoubleMap first (where path, itinerary, and color come from), but still present on CityBus for a little while (the reason it's still present.

As of this commit, the schema definition for Shark::Route looks like this:

class Shark::Route
  ...
  attribute :color,           type: String, nilable: true
  attribute :path,            type: Array[[Float, Float]],  default: []
  attribute :itinerary,       type: Array[String],          default: []
  ...
end

So, even if path and itinerary were nil coming from a source, they should have been substituted with blank arrays. color is nilable, so it would come through as null.

For stations, it looks like you've found a persistent record from when I was testing (still don't have a local instance. This is why I'm writing unit tests). There was an issue where config.serialized_attributes wasn't getting properly filled out by Schema#transform, but I am 95% sure that the issue is completely resolved. That was a few days though, so I'm a little confused with why that data is still there, since stations get updated daily.

For vehicles, it looks like there is an issue where schemas are passing validations for objects with nils for attributes that do not have a default. latitude and longitude should be required, since a default value doesn't make sense, and nil is useless. I'll get on that soon.

tl;dr: are you on the latest commit? I might need to fix vehicles, but everything else should have been fixed already.

From a local instance, with the lastest commit (a238997), with a fresh crossbar started up right before shark:

{
  "identifier": "routes.14",
  "associated_objects": {
    "Shark::Station": []
  }
}

{
  "identifier": "stations.BUS241",
  "associated_objects": {}
}

{
  "saturation": 3,
  "code": null,
  "capacity": 60,
  "name": "4008",
  "route": "routes.1B",
  "associated_objects": {},
  "longitude": null,
  "heading": 334.3599853515625,
  "last_station": "stations.BUS287",
  "latitude": null,
  "schedule_delta": null,
  "onboard": 2,
  "identifier": "vehicles.4008",
  "speed": 0,
  "next_station": "stations.BUS122"
}

Are you seeing anything like this?

btw, I wrote a script to make it easy to query meta.last_event on an identifier

I'm just now starting to write tests for Serializable, which should reveal any issues that are happening here.

If nothing comes out of that, then it is probably a configuration issue, though I can't see anything wrong with the current (a238997) configuration.

1) Shark::Serializable with a default configuration includes all attributes
     Failure/Error: expect(serialized).to include(:name, :list, :hash)

       expected {:identifier => "dummyclasss.Main", :associated_objects => {Shark::DummyClass => ["dummyclasss.Main.1", "dummyclasss.Main.2"]}} to include :name, :list, and :hash
       Diff:
       @@ -1,2 +1,3 @@
       -[:name, :list, :hash]
       +:associated_objects => {Shark::DummyClass=>["dummyclasss.Main.1", "dummyclasss.Main.2"]},
       +:identifier => "dummyclasss.Main",

There is indeed an error. I'm really not sure how this happened, since it was working before, and I don't remember much of anything changing. Regardless, I'll get on fixing this shortly.