Skip to content

Instantly share code, notes, and snippets.

@alistairncoles
Last active March 21, 2018 12:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save alistairncoles/27c086457a1251e9f7f4b85f57a3b553 to your computer and use it in GitHub Desktop.
Save alistairncoles/27c086457a1251e9f7f4b85f57a3b553 to your computer and use it in GitHub Desktop.
hack object to remove x-delete-at
#!/usr/bin/python
import os
import sys
from swift.common.utils import Timestamp
from swift.obj import diskfile
#
# make this file executable.
#
# hack_object_metadata.py <object data dir>
#
# e.g.:
# hack_object_metadata.py /mnt/sdb1/2/node/sdb2/objects/425/508/6a5875aa6f189856ad72817fa8788508
#
def remove_x_delete_at(data_dir):
for filename in os.listdir(data_dir):
path = os.path.join(data_dir, filename)
if not filename.endswith(('.data', '.meta')):
print('Ignoring %s' % path)
continue
with open(path, 'wb') as fd:
metadata = diskfile.read_metadata(fd)
popped = []
for k in metadata.keys():
if k.lower() == 'x-delete-at':
popped.append(metadata.pop(k))
diskfile.write_metadata(fd, metadata)
if popped:
isoformats = [Timestamp(p).isoformat for p in popped]
print('Removed x-delete-at value(s) in %s:' % path)
for p, iso in zip(popped, isoformats):
print(' %s (%s)' % (p, iso))
else:
print('No x-delete-at values found in %s' % path)
if __name__ == '__main__':
remove_x_delete_at(sys.argv[1])
@alistairncoles
Copy link
Author

alistairncoles commented Mar 21, 2018

This is not a great idea in production...

This script would need to be applied to every replica of an object for the expiry-reversal to be durable. That is hard to achieve when replicas may be on handoffs or old primaries after a rebalance. The object replicator would NOT replicate the change to a single replica. Unchanged replicas would still 404 and the proxy would eventually (I think) find an 'unexpired' replica - but if the unexpired replica were lost then the original expired replicas would be replicated and the object would become expired again.

If the expirer ran it would delete any replicas not fixed by this script. Perhaps the fixed replicas would then be replicated to replace the removed ones...except they would conflict with the tombstones at same timestamp, so perhaps not. The object would have inconsistent replicas and not be durable.

Perhaps a .meta file could be synthesised? I'm not sure that is possible via the diskfile interface for an expired object.

Conclusion: I would not recommend doing this in a production system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment