Skip to content

Instantly share code, notes, and snippets.

@fabricedesre
Created July 21, 2022 17:33
Show Gist options
  • Save fabricedesre/741b2695581993e35a4307794861df91 to your computer and use it in GitHub Desktop.
Save fabricedesre/741b2695581993e35a4307794861df91 to your computer and use it in GitHub Desktop.

In Capyloon we use a virtual file system to store user data in a broad sense. It can be usual objects like pictures, downloaded documents etc. but also contacts, navigation history entries, media playback sessions. These are organized as a hierarchy, but leafs are not plain files: they are resources that can be represented by different variants: for instance, an image will have a default variant with the full size version, but also a thumbnail variant and a exif-metadata variant if applicable. Media playback sessions store the page url, title and track info in the default variant, but also a poster image for the video as a variant, etc. Resources also have custom metadata: we maintain a "visit score" state that helps with surfacing the most relevant results in search queries (similar to what Firefox uses in the awesomebar). We want to persist these resources in wnfs, so we need some solution to map resources with variants to the more simple model of wnfs. I can think of a couple of options:

  1. Turn each resource into a directory, named eg. $RESOURCE_NAME.capyloon and store variants in that directory.
  • Pros: simple and direct mapping with the Capyloon vfs data model.
  • Cons: quite ugly since it's adding a whole additional depth layer. Could be mitigated by teaching wnfs browsers about the special directory structure.
  1. Store the default variant as a regular file, and only the non-default ones in a sub-direcetory.
  • Pros: provides direct access to the default variant in wnfs tools.
  • Cons: Easy to get out of sync with the vfs model: what if the default variant is deleted but not the sub-directory?
  1. Use mime multipart to store all variants in a single file.
  • Pros: Keep the overall hierarchy shape clean, and easily syncable.
  • Cons: Implementation overhead, and potential storage overhead too (updating a single variant would cause the whole multipart encrypted blob to change). Need special support in wnfs browsing tools.
  1. Teach wnfs that "files" could be more than just a unixfs file.
  • Pros: suits Capyloon well!
  • Cons: adds overall complexity?

Does that make any sense?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment