mfournier/riemann irc log for @faxmodem

## riemann irc log for @faxmodem
2016-04-20 10:12:04	mfournier	FWIW, I've updated a couple of production riemann's to master about a week ago, so far so good !
2016-04-20 10:13:07	mfournier	only noticeable change: when querying the index returns an unreasonable amount of results, the server seems to interrupt the query and logs this error:
2016-04-20 10:13:08	mfournier	riemann.transport.tcp - forcefully closing
2016-04-20 10:13:08	mfournier	 connection from  #object[java.net.InetSocketAddress 0x35977b02 /192.168.200.10:35946] . Client might be
2016-04-20 10:13:09	mfournier	not reading acks fast enough or network is broken
2016-04-20 10:13:17	mfournier	s/error/warning/
2016-04-20 10:14:46	mfournier	so this seems the right thing to do from the server perspective, but clients (tried the ruby lib & riemann-c-client) don't expect this and don't provide useful error messages (yet).
2016-04-20 12:01:34	insitu	oh, I am the one responsible for that
2016-04-20 12:02:21	insitu	mfournier: I had some OOME errors in riemann caused by misbehaving clients
2016-04-20 12:03:47	mfournier	insitu: great, thanks ! Better being defensive against misbehaving clients :)
2016-04-20 12:04:38	insitu	not sure how to provide more informations to client though
2016-04-20 12:04:58	insitu	I thought you were having troubles with this "feature" :-)
2016-04-20 18:28:14	aphyr	calston: I think we've got a changelog (thanks y'all!) and I just need to confirm that the tests still run. Last time I ran em there were errors.
2016-04-20 18:29:48	aphyr	We could augment the client state machines to include out-of-band warnings
2016-04-20 18:29:54	aphyr	but it's also tricky to figure out how to surface those
2016-04-20 18:30:16	aphyr	and it won't be backwards compatible with Riemann's current protocol, which is req/rsp
2016-04-20 18:30:22	aphyr	poor design planning on my part.
2016-04-20 18:30:53	aphyr	The forward compat path would probably be to add an introductory feature negotiation message
2016-04-20 18:31:00	aphyr	where the client advises it can receive out-of-band messages
2016-04-20 18:31:11	aphyr	and then Riemann can send it warnings about this sort of thing
2016-04-20 18:31:34	aphyr	Then we augment the normal req/rsp messages with a req-id field, so you can match up requests to responses.
2016-04-20 18:31:52	aphyr	That'd also clean up a lot of concurrent state machine garbage in threaded clients
	2016-04-20 10:12:04 mfournier FWIW, I've updated a couple of production riemann's to master about a week ago, so far so good !
	2016-04-20 10:13:07 mfournier only noticeable change: when querying the index returns an unreasonable amount of results, the server seems to interrupt the query and logs this error:
	2016-04-20 10:13:08 mfournier riemann.transport.tcp - forcefully closing
	2016-04-20 10:13:08 mfournier connection from #object[java.net.InetSocketAddress 0x35977b02 /192.168.200.10:35946] . Client might be
	2016-04-20 10:13:09 mfournier not reading acks fast enough or network is broken
	2016-04-20 10:13:17 mfournier s/error/warning/
	2016-04-20 10:14:46 mfournier so this seems the right thing to do from the server perspective, but clients (tried the ruby lib & riemann-c-client) don't expect this and don't provide useful error messages (yet).
	2016-04-20 12:01:34 insitu oh, I am the one responsible for that
	2016-04-20 12:02:21 insitu mfournier: I had some OOME errors in riemann caused by misbehaving clients
	2016-04-20 12:03:47 mfournier insitu: great, thanks ! Better being defensive against misbehaving clients :)
	2016-04-20 12:04:38 insitu not sure how to provide more informations to client though
	2016-04-20 12:04:58 insitu I thought you were having troubles with this "feature" :-)
	2016-04-20 18:28:14 aphyr calston: I think we've got a changelog (thanks y'all!) and I just need to confirm that the tests still run. Last time I ran em there were errors.
	2016-04-20 18:29:48 aphyr We could augment the client state machines to include out-of-band warnings
	2016-04-20 18:29:54 aphyr but it's also tricky to figure out how to surface those
	2016-04-20 18:30:16 aphyr and it won't be backwards compatible with Riemann's current protocol, which is req/rsp
	2016-04-20 18:30:22 aphyr poor design planning on my part.
	2016-04-20 18:30:53 aphyr The forward compat path would probably be to add an introductory feature negotiation message
	2016-04-20 18:31:00 aphyr where the client advises it can receive out-of-band messages
	2016-04-20 18:31:11 aphyr and then Riemann can send it warnings about this sort of thing
	2016-04-20 18:31:34 aphyr Then we augment the normal req/rsp messages with a req-id field, so you can match up requests to responses.
	2016-04-20 18:31:52 aphyr That'd also clean up a lot of concurrent state machine garbage in threaded clients