Skip to content

Instantly share code, notes, and snippets.

@wiredfool

wiredfool/gist:5754498

Created June 11, 2013 04:36
Show Gist options
  • Save wiredfool/5754498 to your computer and use it in GitHub Desktop.
Save wiredfool/5754498 to your computer and use it in GitHub Desktop.
Download ZIP
comparision of thumbnail generation locally and delivering to s3.
Raw
gistfile1.txt
from before, no s3:
--------------------------------------------------------------------------------
Command: python manage.py generate_all_thumbnail_aliases
Massif arguments: (none)
ms_print arguments: massif.out.16695
--------------------------------------------------------------------------------
MB
126.7^ #
| #
| #
| :#::::
| :#::: ::::
| :#::: : :: :
| :#::: : :: ::::: :
| ::: :::::#::: : :: ::: :@ ::: ::::
| ::: ::: :#::: : : :: ::: :@ ::: :: : @
| ::: ::: :#::: ::@ : : :: ::: :@ ::: :: : :@:
| ::: ::: :#::: ::@ : : :: ::: :@ ::::: :: : :@:
| :::::: ::: :#::: ::@:@@ : : :::: : ::: :@ : ::: :: : :@:
| :::::::: : ::: :#::: ::@:@ : : ::: :::::: :@ : ::: ::: : :@:
| : :::::: : ::: :#::: ::@:@ : : ::: ::::: ::: :@::: ::: ::: @:::@:@
| : :::::: ::::::: :#::: ::@:@ :::: ::: :: :: ::: :@: : :::::::: @:@:@:@
| : :::::: ::: ::: :#::: ::@:@ ::: : ::: :: :: ::: :@: : :::: ::: @:@:@:@
| : :::::: ::: ::: :#::: ::@:@ ::: : ::: :: :: ::: :@: : :::: ::: @:@:@:@
| : :::::: ::: ::: :#::: ::@:@ ::: : ::: :: :: ::: :@: : :::: ::: @:@:@:@
| : :::::: ::: ::: :#::: ::@:@ ::: : ::: :: :: ::: :@: : :::: ::: @:@:@:@
| : :::::: ::: ::: :#::: ::@:@ ::: : ::: :: :: ::: :@: : :::: ::: @:@:@:@
0 +----------------------------------------------------------------------->Gi
0 827.5
Number of snapshots: 86
Detailed snapshots: [15 (peak), 21, 23, 39, 50, 60, 70, 80]
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
0 0 0 0 0 0
1 18,993,536,769 59,185,856 58,837,121 348,735 0
2 42,660,437,431 53,416,112 53,069,484 346,628 0
3 57,278,269,515 91,078,816 90,713,939 364,877 0
4 69,349,649,159 89,137,312 88,769,496 367,816 0
5 80,989,618,898 87,781,920 87,409,416 372,504 0
6 97,967,475,460 63,120,176 62,739,465 380,711 0
7 110,902,737,331 63,142,824 62,756,033 386,791 0
8 130,853,803,478 46,138,392 45,743,481 394,911 0
9 146,351,340,518 54,766,072 54,341,779 424,293 0
10 160,232,357,089 39,946,608 39,526,841 419,767 0
11 179,951,871,863 90,935,232 90,473,417 461,815 0
12 190,420,997,108 90,945,392 90,480,537 464,855 0
13 209,763,723,800 87,700,880 87,231,524 469,356 0
14 222,970,580,429 117,270,760 116,791,124 479,636 0
15 238,882,834,007 132,897,560 132,411,972 485,588 0
99.63% (132,411,972B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->69.32% (92,127,312B) 0x7AAD2EA: ImagingNewArray (Storage.c:287)
with s3, this keeps ramping up memory.
--------------------------------------------------------------------------------
Command: python manage.py generate_all_thumbnail_aliases
Massif arguments: (none)
ms_print arguments: massif.out.25793
--------------------------------------------------------------------------------
MB
210.3^ #
| @ #::
| @ :@:: :#::: ::@
| @ @::: :@:: :#::: ::@
| :@:: @ :::: :@:: :#::: ::@:
| @ :@:: @ :::: :@@:: @:#:::@:::@:
| :@::: :@:: @ :::: :@@:::@:#:::@:::@:
| :@: : :@:: : @ ::::@:@@:::@:#:::@:::@:
| @ :@: : : :@:::: :: @ ::::@:@@:::@:#:::@:::@:
| @@@:::@: : ::: : :@:::::::::::@ ::::@:@@:::@:#:::@:::@:
| @ @:::@: : :::::: : :@:::::: ::: @ ::::@:@@:::@:#:::@:::@:
| :::: @ @:::@: : ::::: :@@:@:::::: ::: @ ::::@:@@:::@:#:::@:::@:
| :: : @ @:::@: : ::::: ::@ :@:::::: ::: @ ::::@:@@:::@:#:::@:::@:
| :: :: @ @:::@: ::::::: ::@ :@:::::: ::: @ ::::@:@@:::@:#:::@:::@:
|@@::: :::: :@ @:::@: ::::::: ::@ :@:::::: ::: @ ::::@:@@:::@:#:::@:::@:
|@ ::: ::: :::@ @:::@: ::::::: ::@ :@:::::: ::: @ ::::@:@@:::@:#:::@:::@:
|@ ::: ::: :::@ @:::@: ::::::: ::@ :@:::::: ::: @ ::::@:@@:::@:#:::@:::@:
|@ ::: ::: :::@ @:::@: ::::::: ::@ :@:::::: ::: @ ::::@:@@:::@:#:::@:::@:
|@ ::: ::: :::@ @:::@: ::::::: ::@ :@:::::: ::: @ ::::@:@@:::@:#:::@:::@:
|@ ::: ::: :::@ @:::@: ::::::: ::@ :@:::::: ::: @ ::::@:@@:::@:#:::@:::@:
0 +----------------------------------------------------------------------->Gi
0 836.4
Number of snapshots: 97
Detailed snapshots: [1, 13, 14, 18, 29, 31, 41, 47, 49, 53, 54, 55, 65, 71, 72 (peak), 82, 92]
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
0 0 0 0 0 0
1 9,408,039,792 73,399,536 72,989,881 409,655 0
99.44% (72,989,881B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->47.31% (34,722,576B) 0x7AAD2EA: ImagingNewArray (Storage.c:287)
| ->47.31% (34,722,576B) 0x7A90289: _new (_imaging.c:633)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment