gistfile1.txt

Here's some of the major new features in Pike for Swift:

* Support for globally-distributed erasure codes. This is made up of
  * Replicated erasure code fragments
  * Composite rings for more explicit data placement
  * Per-policy config options

Global erasure codes are implemented by replicating the erasure-coded
fragments of an object. This "EC replication" allows each independent
region to function even if the cross-region network is down, and it
allows for failures in one region to use the remote region to recover.

In order to implement global erasure codes, we first had to support
"composite rings". A composite ring is a data placement ring that is
made up of two or more "normal" rings. The component rings are built
independently, using distinct devices in distinct regions. Building
the composite rings in this way allows dispersion of replicas or
fragments in a more explicit way (e.g. you can specify 4x replication
with 2x in each region or you can specify 10+4 EC replicated across 2
regions).

We also added the ability to override proxy config options on a per-
policy basis. This allows, for example, the ability to set read
affinity for only some storage policies.

* Significant performance improvements in the background consistency engine

We've improved the performance of the object-reconstructor process by
allowing it to scale to multiple concurrent processes per server. The
upshot is that when using erasure codes, Swift now recovers from
failed hardware more quickly and can ingest new capacity more quickly.

* During the Pike cycle we also found and closed several significant
  bugs. We found and closed a bug related to logging in object server
  workers. We also found a config constraint in a particular erasure
  code library that could cause data to be corrupted, and we now
  prevent policies with known-bad configurations from being used.