Skip to content

Instantly share code, notes, and snippets.

@moagrius
Last active April 26, 2020 19:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save moagrius/bb45755c3917e3c2b217b694d6854047 to your computer and use it in GitHub Desktop.
Save moagrius/bb45755c3917e3c2b217b694d6854047 to your computer and use it in GitHub Desktop.

Queue Downloading

Tern Downloading

  1. Fetch book meta: https://www.safaribooksonline.com/api/v1/book/9780132808200/, and create a directory for that book using the book id: 9780132808200

  2. Fetch all chapter meta. Get the chapters field from the book meta json, which looks like this:

     "chapters": [
         "https://www.safaribooksonline.com/api/v1/book/9780132808200/chapter/cover.html",
         "https://www.safaribooksonline.com/api/v1/book/9780132808200/chapter/ch01.html",
         ...
     ]
    

    This does not get saved, just read it in memory.

  3. For each chapter...

    3A. Read the chapter meta json into memory (using the URI provided).

    3B. Save the chapter html. Fetch from the chapter meta content field, which looks like this:

     "content": "https://www.safaribooksonline.com/api/v1/book/9780132808200/chapter-content/ch03.html"
    

    The chapter html will have relative paths like this:

     <img src="graphics/03fig01.jpg" alt="Image">
    

    Use a local file name equal to the full_path property from the chapter meta, which looks like this:

      "full_path": "ch03.html"
    

    So the local file is saved at 9780132808200/ch03.html

    3C. Save the chapter styles. Use the stylesheets property from the chapter meta:

     "stylesheets": [
         {
             "full_path": "9780132808187.css",
             "url": "https://www.safaribooksonline.com/library/css/tcpip-illustrated-volume/9780132808200/9780132808187.css",
             ...
         }
    

    Save the chapter styles using the full_path property, so the file is saved at 9780132808200/9780132808187.css.

    3D. Save the site styles. Use the site_styles property from the chapter meta:

     "site_styles": [
         "https://www.safaribooksonline.com/static/CACHE/css/a1cf77ac2ed3.css"
     ]
    

    Save the local file to the path following the remote domain, so the saved file will be at 9780132808200/static/CACHE/css/a1cf77ac2ed3.css.

    3E. Save the chapter images. Use the images property of the chapter meta, which looks like this:

     "images": [
         "graphics/03fig05.jpg",
         "graphics/03fig23.jpg"
     ]
    

    Save each local file using the path provided, e.g., 9780132808200/graphics/03fig05.jpg

    3F. Storage: The Tern DB saves offline item data to 2 tables, ITEM and OFFLINEITEM. The former is each asset; the latter is just the book. However, ITEM does not provide local storage location, so isn't very helpful. OFFLINEITEM has columns for ITEMID, OFFLINECOUNT, and STORAGELOCATION. If the total number of files migrated equals OFFLINECOUNT, then you have successfully migrated.

Kook Downloads

Migrating

When migrating, get shared prefs for "lastUserId" - this is the directory to move all downloaded content into. Create this directory. Poll OFFLINEITEM table; for each row, get STORAGELOCATION (currently something like /data/user/0/com.safariflow.queue/files) and rename that File to {userid}.

Update the 2 CHAPTER_COLLECTION tables to indicate it's downloaded.

Use getFilesDir() + SEPARATOR + book.getIdentifier() + '/' as base_url for webview for downloaded books.

SD Card? TDB.

Other

@jameyd full_path of site styles member in new endpoint is currently null, should be from /static/... e.g., https://www.safaribooksonline.com/api/v1/book/9780133261615/assets/

@chankeypathak
Copy link

Hello, the HTML files in com.safariflow.queue/files folder seem to be encrypted. Any idea how to decode them?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment