A layer is an irmin-pack store that can be opened as a normal store, or that can be combine with other layers, to form a layered store. As in Overlayfs, the lower layer is readonly, whereas the upper layer is used for adding the latest values. add is always performed on upper. find first tries to retrieve data from upper, and then defaults to lower.
Unlike Overlayfs, data from lower is never copied back in upper: every time a data from lower is needed it is accessed on lower.
The following benchmarks are from bootstrapping the layered store and calling a freeze every 50, 500 or 4000 commits. The x axis represents the completion time of requests successfully validated by the block validator, or the prevalidator, as they are read from the logs. The x axis does not represent the commits, so a freeze does not necessarily occur at the 50th mark on the axis. The logs as well as the scripts for generating the graphs are here.
Sometimes I'll use a local benchmark, provided in the layered store branch. The default setup for these benchmark is that every commit add 1000 nodes to the previous commit and a freeze is called every 50 commits.
They can be run by executing dune exec -- ./test/irmin-pack/layered_bench.exe --nbatches=4 --depth=1000 2> outs for instance and then using the scripts here to generate the graphs.
The bootstr
Two proposals for a layered store with two lower stores:
| Configuration: | |
| Key size: 32 | |
| Value size: 13 | |
| Number of bindings: 10000000 | |
| Log size: 500000 | |
| Seed: 0 | |
| Metrics: true | |
| Frequency: 10 | |
| Replace in random order and time each operation |