Berlin/Hamburg GTFS-Realtime talk

talk.txt

realtime data in public transport
	cancellations
	delays
	positions
	notices/warnings
	why?
		without it, public transport doesn't feel reliable enough
		long-term: frustration accumulates, people stop using it (if they have the choice)
		why not?! part of service: A->B, on time, reliably, comfortably
existing (legacy) APIs
	not very oriented on travellers' needs:
		focused on routing
		focused on "ahead-of-time" inquiries
		not suited for travel/commute companions
	prevent "new" use cases
		custom routing
			more accessible
			(more) intermodal
			meet me in the middle
		dashboards
			reliability, operating status (see DC Metro Hero)
			check if public funding is wasted
	not open
		technically
			proprietary APIs/formats
			lacking documentation
		legally
			signing contracts not always an option
		non-local actors won't even bother
	often only request-based, no streaming access
		rate limits
		waste of resources
		(latency)
[GTFS-Realtime](https://gtfs.org/reference/realtime/v2/)
	applies [GTFS-static](https://gtfs.org/reference/static) concepts to realtime data
	GTFS-static
		when is a bus supposed to stop where?
		.zip archive of CSV files
	GTFS-realtime
		bus on time, cancelled?
		notices/warnings, e.g. "today no step-free access"
		where is the bus?
		(congestion)
		binary, [Protocol Buffers (pbf)](https://developers.google.com/protocol-buffers/docs/reference/proto3-spec)
		mostly same naming as GTFS-static
	how to consume feed?
		common variant: "dump" mode
			all current feed msgs in one file, periodically fetched
		differential mode
			[draft](https://github.com/google/transit/issues/84), old but relevant
			individual messages with `DIFFERENTIAL` flag
			any streaming transport, e.g. WebSockets, MQTT, AMQP
HAFAS
	public transport backend & mobile app system
	many customers -> many instances
		e.g. VBB & BVG, HVV, DB Navigator, Nah.SH, SaarVV, VVS
		[transport-apis](https://github.com/public-transport/transport-apis)
	[hafas-client](https://github.com/public-transport/hafas-client)
		built-in support for many endpoins
	[many other clients](https://github.com/public-transport/hafas-client#related)
	mobile "mgate.exe" API
		abbreviated field names
		built for mobile apps
	"rest.exe" API
		intended for the public
[transport.rest](https://transport.rest)
	public, truly open
		no authentication
		rate limit
	target group:
		quick experiments, scripting, hackathons
			e.g. `curl 'https://v5.hvv.transport.rest/stops/6237/departures' | jq '.[0].when'`
		less technical/experienced users:
			beginners, students
			no-code environments?
		unified API across data sources
			eventually deprecated by another standard
	wrapper APIs around archaic, hard-to-use, closed APIs
		no duplicate reverse engineering & guesswork
[hafas-gtfs-rt-feed](https://github.com/derhuerst/hafas-gtfs-rt-feed)
	thin wrappers:
		[berlin-gtfs-rt-server](https://github.com/derhuerst/berlin-gtfs-rt-server): VBB HAFAS + hafas-gtfs-rt-feed
		[hamburg-gtfs-rt-server](https://github.com/derhuerst/hamburg-gtfs-rt-server): HVV HAFAS + hafas-gtfs-rt-feed
	[demo using gtfs-rt-inspector](https://public-transport.github.io/gtfs-rt-inspector/?feedUrl=https%3A%2F%2Fv0.hamburg-gtfs-rt.transport.rest%2Ffeed&view=inspector)
	1. poll HAFAS to get all vehicles/"runs"
		[hafas-monitor-trips](https://github.com/derhuerst/hafas-monitor-trips)
		all vehicles in bounding box, every 3min
		refresh vehicles/"runs" every 60s
	2. match runs against GTFS-Static data
		[match-gtfs-rt-to-gtfs](https://github.com/derhuerst/match-gtfs-rt-to-gtfs)
		PostgreSQL DB via [gtfs-via-postgres](https://github.com/derhuerst/gtfs-via-postgres)
		fuzzy matching:
			`S+U Warschauer Str.` -> `warschauer strasse`
			`Warschauer Straße (Bln)` -> `warschauer strasse`
		-> realtime data with GTFS-Static IDs
	3. serve as dump
		newer msg for same run replaces older
		encode all as one pbf "dump"
		serve via HTTP, with caching headers
next steps:
	more regions!
		built with configurable endpoint in mind
		server capacity?
		other APIs than HAFAS
			[trias-client](https://github.com/andaryjo/trias-client)
			[kpublic-transport](https://github.com/KDE/kpublictransport)
			[public-transport-enabler](https://github.com/schildbach/public-transport-enabler)
	official feeds?
		more efficient conversion
		more visibility -> greater impact -> indirect effects
		(fewer layers of complexity)
	federation of data sources?
		stops/lines/routes with stable IDs
			generate IDs from properties, e.g. normalized names & locations
				[pan-european-public-transport](https://github.com/derhuerst/pan-european-public-transport)
			shared catalogs of IDs
			[Linked Connections](https://linkedconnections.org)
		aggregation of differential msgs