Using Node streams to append audio files

node-streams.md

Today I decided to clean up and organise files on windows computer at work. This process involved deleting files I no longer have use for and creating grouping the rest in folders. So I ended up with folders for audio, video, projects, ebooks, e.t.c.

One of the folders that I came across contained about 50 small mp3 files. Before deleting the files I had to play them just to make sure they didn't contain anything important. What do I know, turns out they clips of a DHH(the ROR creator) interview - please don't ask me how they got to my computer. He's quite an opinionated guy and I decided I wanted to keep the files. But 50 files? Wouldn't it be nice if I could glue them together into a single file? Well, being the hacker that I am I popped open my terminal and started writing a program.

I broke down the task to two main things.

1. Read and sort the files according to their sequence
2. Somehow combine the files and write them to a single file

The first thing turned out to be quite easy to do because the files were named with a date/time stamp in the following format: Mon Jan 03 10;02;07 2011.mp3. This meant that I could extract the time from the file name and then sort them accordingly. To simplify it further i decided to rename the files with an easier format for sorting - for this I created the rename.js file:

/*
   I placed all the audio files in ./files directory
*/
var fs = require('fs'),
    files = fs.readdirSync('./files');

files.forEach(function (file) {
    var oldname = './files/' + file, 
        date = getDate(file),
        newname = './files/' + date.split(';').join('') + '.mp3';

    renameFile(oldname, newname); 
    console.log('Renamed ' + oldname + ' to ' + newname);    
}); 

function renameFile(oldname, newname) {
    fs.renameSync(oldname, newname);
}

function getDate(name) {
    var d = name.substring(11, 19);

    return d;
}

console.log('Done');

Up next I had to create the main program that combines the files and writes them to a single file. Streams to the rescue!

What are streams?

Streams are a powerful feature in Node.js, they create a transportation system that moves your data in chunks. The source of the data is a readable stream and the destination a writable stream. The file system module has 2 methods that you can use to take advantage of Node streams, fs.createReadStream and fs.createWriteStream. I created a new file for the main program, stream.js

var fs = require('fs'),
    files = fs.readdirSync('./files'),
    chunks = [],
    stream,
    currentfile,
    dhh = fs.createWriteStream('./dhh-interview.mp3');

// create an array with filenames (time)
files.forEach(function (file) {
    chunks.push(file.substring(0, 6));  
});

// Sort
chunks.sort(function (a, b) {
    return a - b;
});

// recursive function
function main() {
    if (!chunks.length) {
        dhh.end("Done");
        return;
    }

    currentfile = './files/' + chunks.shift() + '.mp3';
    stream = fs.createReadStream(currentfile);

    stream.pipe(dhh, {end: false});

    stream.on("end", function() {
        console.log(currentfile + ' appended');
        streamer();        
    });
}


main();

That's it. Lastly I needed to run my programs.

node rename.js
node stream.js

Note I have included a few files in my repo so that you be able to try this out.

Hey qawemlilo, thanks so much for this example. I need to do a similar thing with multiple audio files, but I need it to be high performance. I tried a similar implementation to what you've shown above, and while the resultant audio plays fine, it audio players struggle to seek it. I feel like the encoding gets a bit messed up simply streaming byte arrays one after the other.

Any advice?