callback stream.md

Stream based on callbacks

Its almost a port of the w3c specc, but uses callbacks instead of promises.

Why?

To see if i could, the w3c specc is one of the best speccs i have seen, very readable and easy to follow. The only thing i question is the use of promises in that situation. The only place you can reasnobly call read is from within the promise resolution. If you dont use the good parts of promises (cached values for multiple, possible late, readers) but treat them like a one shot event, it seams like a simple callback is the thing your looking for.

Why not?

Could be that browsers (c++) have an easier time makeing promises than callbacks without the overhead that it seems to be in JS.

Divergance from the specc

• There is no Stream, just the read function small difference :)
  • Specc:
    stream.read(arg?) => Promise<value>
  • My way:
    readable(arg?, cb) => void Just do stream = { read: readable } if you miss the extra typing.
  • Reason:
    Reading streams is common and should be easy. Writing premission should be set when the stream is created, providing a write method prohibits that. The common case is to simply do s = new Stream(); s.write(source) and never call write again, so lets make that simple readable = Stream.from(source). If you want to you can get the same api anyway, writable = Stream.writable(); writable.write(dynamicSource).
• Eof is called end, end of stream is a better alternative than file, but just end is clear enough and dont mix in any other concepts with the stream.
• End is passed as an error rather than a value, after the last value.
  • Specc way:
    • Data:
      • Value: { data: realValue, eof: false }
      • End: { data: lastValue, eof: true }
    • Error:
      • Value: null
      • End: null
  • My way:
    • Data:
      • Value: realValue / lastValue
      • End: null
    • Error:
      • Value: null
      • End: { data: 'Streamation ended', isEnd: true }
  • Reason:
    Think of a stream as a two lane thing, the data lane and a fast lane. Any stream modifier (map, filter...) will have to fast track all errors and ends, puting them in the same place makes it easy to fast track both at once.

// I was wrong, dont fast track eof here, since they carry data too
function map(stream, mapFn) {
  var mappedStream = new Stream()
  stream.read(arg).then(function (v) {
    v.data = mapFn(v.data) // Dont fast track eof.
    mappedStream.write(v)
  }, function(e) {
    throw e // Is that how you fast track errors to following promise?
  })
  return mappedStream
}

function map(myStream, mapFn) {
  return function mappedStream(arg, cb) {
    return myStream(arg, function (err, val) {
      return (err)
        ? cb(err) // Fast track errors and ends
        : cb(err, mapFn(val))
    })
  }
}

Considerations

This is JS so mutation is what we do, that said this is a read once stream. Maybe look at Clojure and there reduce/reducer instead, should be possible to do on a stream. Not sure what that would do to re-readability. But it would unify streams with every other collection we have. And make the reducers (readers in my streams) reusable, good stuff.

Or the Haskell pattern of having readers that return next-reader instead of calling stream.read from the callback. Basially the steram pulls readers out of the last reader until done instead of the reader pulling data out of the stream. Again not sure what it would do to re- readability, but it would make the iteratees reusable on other streams.