Example of using gramophone ngrams as input into Natural's TF-IDF function. It's adapted from the example in the Natural docs. NOTE that all arguments to the tfidf methods must be arrays. If they are strings, the tfidf object will run the default tokenizer over the argument. I might submit a pull request to natural so that `natural.TfIdf` could …

gistfile1.js

var natural = require('natural'),
  TfIdf = natural.TfIdf,
  tfidf = new TfIdf();

var gramophone = require('gramophone');

var docs = [
  'this document is about node programming language.',
  'this document is about ruby programming language.',
  'this document is about the ruby programming language and node programming language.',
  'this document is about node programming language. it has node programming language examples'
];

docs.forEach(function(doc, index){
  var ngrams = gramophone.extract(doc, { min: 1 , flatten: true});
  console.error('ngrams for doc ' + index + ':');
  console.error(ngrams);
  tfidf.addDocument(ngrams);
});

console.log('node programming language -----------');
tfidf.tfidfs(['node programming language'], function(i, measure) {
  console.log('document #' + i + ' is ' + measure);
});

console.log('"document" --------------------------------');
tfidf.tfidfs('"document"', function(i, measure) {
  console.log('document #' + i + ' is ' + measure);
});

running this produces the following output:

ngrams for doc 0:
[ 'node programming language', 'document' ]
ngrams for doc 1:
[ 'ruby programming language', 'document' ]
ngrams for doc 2:
[ 'ruby programming language',
  'node programming language',
  'document' ]
ngrams for doc 3:
[ 'node programming language',
  'node programming language',
  'programming language examples',
  'document' ]
node programming language -----------
document #0 is 0
document #1 is 0
document #2 is 0
document #3 is 0
"document" --------------------------------
document #0 is -0.1823215567939546
document #1 is -0.1823215567939546
document #2 is -0.1823215567939546
document #3 is -0.1823215567939546

modules:

• gramophone 0.0.3
• natural 0.1.23

Am I missing something ?

edit: I have the same problem using only natural's tf-idf example, therefore my problem has nothing to do with gramophone