Skip to content

Instantly share code, notes, and snippets.

@mzgoddard
Created December 6, 2017 22:53
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mzgoddard/63b194cb0ec5d47a29b76e9b77fae3ef to your computer and use it in GitHub Desktop.
Save mzgoddard/63b194cb0ec5d47a29b76e9b77fae3ef to your computer and use it in GitHub Desktop.
  • read
    • read table
    • each log
      • open
      • check block status
      • add block to file
      • if file complete emit it
  • batch put/del
    • read table
    • open current log
    • each op
      • if op is put
        • split into blocks
        • each block
          • add to log
          • if log is at max
            • close current log
            • open new log as current
        • replace file entry in table
      • if op is del
        • remove file entry in table
    • close current log
    • atomically write table
    • if size needed is > min collect size and size used / size needed > 1.5 => garbage collect
  • garbage collect
    • read table
    • open output log
    • each log
      • open
      • each block
        • if block is used
          • write to output
          • update related file entry
          • if output is at max
            • close output
            • open new output
    • close output
    • write table atomically
    • delete old log files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment