Skip to content

Instantly share code, notes, and snippets.

@alastair
Last active June 23, 2017 14:22
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save alastair/7459a5a9fe6540fec585efd9ff3405e6 to your computer and use it in GitHub Desktop.
Save alastair/7459a5a9fe6540fec585efd9ff3405e6 to your computer and use it in GitHub Desktop.
last.fm incremental import
psuedo code - last.fm import js:
get last import date from server
for page in all_scrobbles:
store newest scrobble date in the list
send page
if earliest item in page < last import date:
stop submitting
submit 'last import date' to server, which is the date of the most recent scrobble (or now)
psuedo code - last.fm import js:
get last import date from server
get total number of pages in all scrobbles
do some sort of binary search to find the page which contains the last imported item
for page in all_scrobbles, reversed from the page which we found:
submit page
[sever] stores most recent date in this submission as last import date
-> client doesn't need to submit last import date, can be stopped and restarted at any time
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment