Skip to content

Instantly share code, notes, and snippets.

@shiplunc
Last active August 29, 2015 14:07
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save shiplunc/a0116db87a2a2e0b6237 to your computer and use it in GitHub Desktop.
Save shiplunc/a0116db87a2a2e0b6237 to your computer and use it in GitHub Desktop.
Estimated time calculator for Getty Re indexing
# Put this file on scripts folder and run using following command
# sudo DJANGO_SETTINGS_MODULE=theplatform.settings /root/python-env/bin/python estimated-time-for-getty-reindex.py /home/ubuntu/source-72-combine-getty-assetgroups.checkpoint
import sys
import datetime
def print_progress(filename):
with open(filename, 'r') as f:
content = f.read()
(dur, checkpoint_info) = eval(content)
(_, _, checkpoint_id, num_err, num_suc) = checkpoint_info
from theplatform.newscred.models import Source, Image
GETTY_SOURCE = Source.objects.get(name='Getty Images')
remn = Image.objects.filter(source=GETTY_SOURCE, id__lt=checkpoint_id).count()
rdur = dur*remn/(num_err+num_suc)
print "Estimated remaining time to complete: %s" % rdur
def main():
print_progress(sys.argv[1])
if __name__ == '__main__':
main()
@shiplunc
Copy link
Author

shiplunc commented Nov 6, 2014

Thu Nov  6 06:56:45 UTC 2014
[datetime.timedelta(12, 45005, 701027), ('theplatform.newscred.models.image', 'Image', 39274555L, 10, 15260202)]
Estimated remaining time to complete: 15 days, 14:49:36.770005

@shiplunc
Copy link
Author

shiplunc commented Nov 9, 2014

Sun Nov  9 11:18:37 UTC 2014
[datetime.timedelta(15, 59694, 894247), ('theplatform.newscred.models.image', 'Image', 21917683L, 10, 22911520)]
Estimated remaining time to complete: 7 days, 19:06:00.577613

@shiplunc
Copy link
Author

Wed Nov 12 07:42:19 UTC 2014
[datetime.timedelta(18, 47659, 520068), ('theplatform.newscred.models.image', 'Image', 13151317L, 10, 30149640)]
Estimated remaining time to complete: 2 days, 13:12:52.935378

@shiplunc
Copy link
Author

Thu Nov 13 03:47:50 UTC 2014
[datetime.timedelta(19, 33546, 400610), ('theplatform.newscred.models.image', 'Image', 9952649L, 10, 32288263)]
Estimated remaining time to complete: 1 day, 4:55:02.921159

@shiplunc
Copy link
Author

Thu Nov 13 05:48:24 UTC 2014
[datetime.timedelta(19, 40793, 689414), ('theplatform.newscred.models.image', 'Image', 9709388L, 10, 32507338)]
Estimated remaining time to complete: 1 day, 1:41:50.633443

@shiplunc
Copy link
Author

The back pop is completed. Content of the checkpoint after completion is

[datetime.timedelta(20, 7796, 861578), ('theplatform.newscred.models.image', 'Image', 4063647L, 10, 34294634)]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment