Skip to content

Instantly share code, notes, and snippets.

@zhangsoledad
Created March 18, 2020 09:05
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save zhangsoledad/529c2b16fc48a005835d7f7164252e38 to your computer and use it in GitHub Desktop.
Save zhangsoledad/529c2b16fc48a005835d7f7164252e38 to your computer and use it in GitHub Desktop.
  1. tikv gc TiDB 的事务的实现采用了 MVCC(多版本并发控制)机制,当新写入的数据覆盖旧的数据时,旧的数据不会被替换掉,而是与新写入的数据同时保留,并以时间戳来区分版本。GC 的任务便是清理不再需要的旧数据。 https://pingcap.com/docs-cn/stable/reference/garbage-collection/overview/#gc-%25E6%259C%25BA%25E5%2588%25B6%25E7%25AE%2580%25E4%25BB%258B https://github.com/tikv/tikv/tree/master/src/server/gc_worker

  2. delete-range

  • The most common pattern we see is scan-and-delete, i.e., advance an iterator through the to-be-deleted range, and issue a Delete for each key. This is slow (involves read I/O) so cannot be done in any critical path. Additionally, it creates many tombstones, which slows down iterators and doesn’t offer a deadline for space reclamation.
  • Deleting a range of keys is a common pattern in RocksDB. Most systems built on top of RocksDB have multi-component key schemas, where keys sharing a common prefix are logically related
  • DeleteFilesInRange can be used prior to compacting the deleted range as long as snapshot readers do not need to access them. It drops files that are completely contained in the deleted range. https://rocksdb.org/blog/2018/11/21/delete-range.html https://github.com/facebook/rocksdb/wiki/DeleteRange
  1. Memory-base pruning openethereum/openethereum#176

  2. BIP159 https://github.com/bitcoin/bips/blob/master/bip-0159.mediawiki

  3. monitor https://pingcap.com/docs-cn/stable/reference/key-monitoring-metrics/tikv-dashboard/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment