Last active
August 29, 2015 14:07
-
-
Save shiplunc/a0116db87a2a2e0b6237 to your computer and use it in GitHub Desktop.
Estimated time calculator for Getty Re indexing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Put this file on scripts folder and run using following command | |
# sudo DJANGO_SETTINGS_MODULE=theplatform.settings /root/python-env/bin/python estimated-time-for-getty-reindex.py /home/ubuntu/source-72-combine-getty-assetgroups.checkpoint | |
import sys | |
import datetime | |
def print_progress(filename): | |
with open(filename, 'r') as f: | |
content = f.read() | |
(dur, checkpoint_info) = eval(content) | |
(_, _, checkpoint_id, num_err, num_suc) = checkpoint_info | |
from theplatform.newscred.models import Source, Image | |
GETTY_SOURCE = Source.objects.get(name='Getty Images') | |
remn = Image.objects.filter(source=GETTY_SOURCE, id__lt=checkpoint_id).count() | |
rdur = dur*remn/(num_err+num_suc) | |
print "Estimated remaining time to complete: %s" % rdur | |
def main(): | |
print_progress(sys.argv[1]) | |
if __name__ == '__main__': | |
main() |
Estimated remaining time to complete: 35 days, 22:16:02.392597
Estimated remaining time to complete: 27 days, 4:53:26.189401
Estimated remaining time to complete: 25 days, 21:56:56.987482
Estimated remaining time to complete: 24 days, 16:43:26.354451
Estimated remaining time to complete: 22 days, 8:20:59.484419
Checkpoint file content
[datetime.timedelta(10, 39264, 842892), ('theplatform.newscred.models.image', 'Image', 45468250L, 10, 10930461)]
[datetime.timedelta(11, 36965, 643737), ('theplatform.newscred.models.image', 'Image', 42212783L, 10, 13119968)]
Estimated remaining time to complete: 18 days, 10:41:50.924602
Wed Nov 5 08:47:52 UTC 2014
[datetime.timedelta(11, 51730, 817895), ('theplatform.newscred.models.image', 'Image', 41916685L, 10, 13319818)]
Estimated remaining time to complete: 18 days, 6:21:16.829798
Thu Nov 6 06:56:45 UTC 2014
[datetime.timedelta(12, 45005, 701027), ('theplatform.newscred.models.image', 'Image', 39274555L, 10, 15260202)]
Estimated remaining time to complete: 15 days, 14:49:36.770005
Sun Nov 9 11:18:37 UTC 2014
[datetime.timedelta(15, 59694, 894247), ('theplatform.newscred.models.image', 'Image', 21917683L, 10, 22911520)]
Estimated remaining time to complete: 7 days, 19:06:00.577613
Wed Nov 12 07:42:19 UTC 2014
[datetime.timedelta(18, 47659, 520068), ('theplatform.newscred.models.image', 'Image', 13151317L, 10, 30149640)]
Estimated remaining time to complete: 2 days, 13:12:52.935378
Thu Nov 13 03:47:50 UTC 2014
[datetime.timedelta(19, 33546, 400610), ('theplatform.newscred.models.image', 'Image', 9952649L, 10, 32288263)]
Estimated remaining time to complete: 1 day, 4:55:02.921159
Thu Nov 13 05:48:24 UTC 2014
[datetime.timedelta(19, 40793, 689414), ('theplatform.newscred.models.image', 'Image', 9709388L, 10, 32507338)]
Estimated remaining time to complete: 1 day, 1:41:50.633443
The back pop is completed. Content of the checkpoint after completion is
[datetime.timedelta(20, 7796, 861578), ('theplatform.newscred.models.image', 'Image', 4063647L, 10, 34294634)]
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
After 10 hours.