Simple Factor spider

eden.md

Private API

Returns a season's url on the Audiodramax blog.

: season-url ( season -- url ) 
    "http://www.audiodramax.com/tag/eden-saison-%d" sprintf ;

Filters episodes from a list of urls. At the moment we are only interested on .mp3 files.

: filter-episode-urls ( page -- vector )
    [ path>> R/ .mp3/ re-contains? ] filter ;

Provides a Spider that will find all pages for a given season.

: <eden-season-spider> ( season -- spider )
    season-url <spider>
        t >>follow-robots?
        5 >>max-depth
        t >>quiet?
        1.5 seconds >>sleep
        4 >>#threads
        { [ path>> R/ .*page.[\d]/ matches? ] } >>filters ;

Public API

Downloads an Eden season to the specified directory by running the spider to find all the pages related to this season. I think that a cleaner interface would probably leave the make-directories and with-directory parts.

: eden-download-season ( season dest -- )
    dup make-directories
    [ <eden-season-spider> run-spider spidered>> values
      [ links>> filter-episode-urls ] map concat
      [ dup download-name exists? not [ download t ] when ] map drop 
    ] with-directory ;