Skip to content

Instantly share code, notes, and snippets.

@soldair
Last active December 19, 2015 23:19
Show Gist options
  • Save soldair/6034011 to your computer and use it in GitHub Desktop.
Save soldair/6034011 to your computer and use it in GitHub Desktop.
multilevel ending does not error active streams

I wanted to open the discussion around a case i think is an important bug using multilevel/leveldb over the network.

When you are working with multilevel client and the connection to the server ends in any way all of the "query" streams (no matter their state) simply emit end and close

i have seen this in production with tcp sockets etc. I address it in a clunky way in multilevel-reconnected but i want feedback on what the pattern should be and how i should modularize this.

In the scope where you create the read stream you no idea it did not get all of the data unless you check if the connection has disconnected since the stream started.

any app that has ownership of consistency for a segment of keys will not produce correct results when it doesn't get the whole stream

output no end (control. no end)

stream data:  1
stream data:  2
stream data:  3
stream end
stream close
stream close

output end (with server ending midstream)

stream data:  1
stream end
stream close
stream close
end server.

also multiple close events is an issue i would like to pr perhaps in muxdemux

{
"name": "multilevel-end-test",
"description": "analyze end behavior",
"version": "0.0.0",
"repository": {
"url": "git@gist.github.com:6034011"
},
"main": "index.js",
"scripts": {
"test": "tape test/*.js"
},
"author": "Ryan Day",
"dependencies": {
"through": "~2.3.4",
"multilevel": "~5.1.0",
"level": "~0.10.0"
},
"devDependencies": {
"tape": "*"
}
}
// require something
var level = require("level")
, multilevel = require("multilevel")
, through = require('through')
var db = level('./tmpdb');
, client = multilevel.client()
, server = multilevel.server(db)
, fakeOutConnection = through()
, fakeInConnection = through()
server.pipe(fakeOutConnection).pipe(client.createRpcStream()).pipe(fakeInConnection).pipe(server)
client.batch([
{type:"put",key:"a",value:1},
{type:"put",key:"b",value:2},
{type:"put",key:"c",value:3}
],function(err,data){
if(err) throw err;
client.createReadStream().on("data",function(data){
console.log('stream data: ',data.value);
}).once("data",function(){
// terminate the server connection mid stream.
// comment out this line and notice that you get all of the data with the same interface.
fakeInConnection.end();
console.log('end server.')
}).on('end',function(){
console.log('stream end');
}).on('close',function(){
console.log('stream close');
})
})
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment