gistfile1.txt

stream workshop ideas

[big file]
read a small file into memory to get its size
read a big file into memory, see that break
fix it with a stream that counts the bytes as they come in

[zip]
zip a file
unzip a file
zip and unzip a file

[hasher]
hash one file
log speed and progress twice a second
have it deliver the hash values as a stream
have it hash all the files in a directory and then end

[duplicate copies]
copy a file
add a second destination to make two copies at the same time

[simple spouts]
source produces unlimited random data
source counts to the number you give it and ends

[simple sinks]
sink counts the number of times it sees the ascii byte A
run that from a file
type at that from a terminal

[simple transforms]
transform stream removes all the vowels from a text file
transform stream reports how big the chunks you give it are

[echo]
make a duplex stream that echos back at you
put that in the middle of the file copier

[line overflow]
turn binary stream of text from a file into an object stream of lines
an unusually long line overflows the buffer

[buffering options]
type at a stream that only writes when you put period
change that to do it every 10 characters instead

[moving meter]
write a little meter that measures size of first, last, average, and common chunks
put that into the stream that creates the destination file
refactor to be in a transform stream
refactor to be in a writable stream daisy-chained on

[really long chain]
copy one file to another
add pass-through stream in the middle
put 50 pass-through streams in the middle, and compare performance

[errors]
reader that throws 10 seconds in
writer that throws if you hand it an E below reader that does this after 10 seconds

A good breakdown of native stream implementations (readable, writable, transform) and two modules that are good helpers:

general: https://bl.ocks.org/joyrexus/10026630
through2: https://github.com/rvagg/through2
stream utilities: https://github.com/maxogden/mississippi