tf-idf.php

These weights are often combined into a tf-idf value, simply by multiplying them together. The best scoring words under tf-idf are uncommon ones which are repeated many times in the text, which lead early web search engines to be vulnerable to pages being stuffed with repeated terms to trick the search engines into ranking them highly for those keywords. For that reason, more complex weighting schemes are generally used, but tf-idf is still a good first step, especially for systems where no one is trying to game the system.

There are a lot of variations on the basic tf-idf idea, but a straightforward implementation might look like:

    <?php
    $tfidf = $term_frequency *  // tf
        log( $total_document_count / $documents_with_term, 2); // idf
    ?>

It's worth repeating that the IDF is the total document count over the count of the ones containing the term. So, if there were 50 documents in the collection, and two of them contained the term in question, the IDF would be 50/2 = 25. To be accurate, we should include the query in the IDF calculation, so if in the collection there are 50 documents, and 2 contain a term from the query, the actual calculation would be 50+1/2+1 = 51/3, as the query becomes part of the collection when we're searching.

http://phpir.com/simple-search-the-vector-space-model
http://phpir.com/text-classification