Skip to content

Instantly share code, notes, and snippets.

@wiredfool
Created June 7, 2013 16:20
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save wiredfool/5730478 to your computer and use it in GitHub Desktop.
Save wiredfool/5730478 to your computer and use it in GitHub Desktop.
valgrind run.
(et-test)erics@builder-1204-x64:~/et-test/et-issue-example$ valgrind --tool=massif python manage.py generate_all_thumbnail_aliases
==16695== Massif, a heap profiler
==16695== Copyright (C) 2003-2011, and GNU GPL'd, by Nicholas Nethercote
==16695== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==16695== Command: python manage.py generate_all_thumbnail_aliases
==16695==
1; RSS: 142912; Unshared: 0; Obj: 27683
2; RSS: 142912; Unshared: 0; Obj: 27535
3; RSS: 177228; Unshared: 0; Obj: 27563
4; RSS: 177228; Unshared: 0; Obj: 27591
5; RSS: 177228; Unshared: 0; Obj: 27619
6; RSS: 177228; Unshared: 0; Obj: 27647
7; RSS: 177228; Unshared: 0; Obj: 27675
8; RSS: 177228; Unshared: 0; Obj: 27703
9; RSS: 177228; Unshared: 0; Obj: 27731
10; RSS: 205720; Unshared: 0; Obj: 27559
11; RSS: 205720; Unshared: 0; Obj: 27587
12; RSS: 205720; Unshared: 0; Obj: 27615
13; RSS: 205720; Unshared: 0; Obj: 27643
14; RSS: 205720; Unshared: 0; Obj: 27671
15; RSS: 205720; Unshared: 0; Obj: 27699
16; RSS: 205720; Unshared: 0; Obj: 27727
17; RSS: 205720; Unshared: 0; Obj: 27755
18; RSS: 205720; Unshared: 0; Obj: 27783
19; RSS: 205720; Unshared: 0; Obj: 27811
20; RSS: 205720; Unshared: 0; Obj: 27638
21; RSS: 205720; Unshared: 0; Obj: 27666
22; RSS: 205720; Unshared: 0; Obj: 27694
23; RSS: 205720; Unshared: 0; Obj: 27722
24; RSS: 205720; Unshared: 0; Obj: 27750
25; RSS: 205720; Unshared: 0; Obj: 27778
26; RSS: 205720; Unshared: 0; Obj: 27806
27; RSS: 205720; Unshared: 0; Obj: 27834
28; RSS: 205720; Unshared: 0; Obj: 27862
29; RSS: 205720; Unshared: 0; Obj: 27689
30; RSS: 205720; Unshared: 0; Obj: 27717
31; RSS: 205720; Unshared: 0; Obj: 27745
32; RSS: 205720; Unshared: 0; Obj: 27773
33; RSS: 205720; Unshared: 0; Obj: 27801
34; RSS: 205720; Unshared: 0; Obj: 27829
It took 3826.99 seconds to generate all thumbnails and aliases.
All thumbnail aliases generated succesfully.
==16695==
(et-test)erics@builder-1204-x64:~/et-test/et-issue-example$
And this is the massif memory profile:
--------------------------------------------------------------------------------
Command: python manage.py generate_all_thumbnail_aliases
Massif arguments: (none)->13.14% (17,456,640B) 0x7AAD35D: ImagingNewBlock (Storage.c:333)
| ->13.14% (17,456,640B) 0x7AAD40A: ImagingNew (Storage.c:373)
ms_print arguments: massif.out.16695
--------------------------------------------------------------------------------
MB
126.7^ #
| #
| #
| :#::::
| :#::: ::::
| :#::: : :: :
| :#::: : :: ::::: :
| ::: :::::#::: : :: ::: :@ ::: ::::
| ::: ::: :#::: : : :: ::: :@ ::: :: : @
| ::: ::: :#::: ::@ : : :: ::: :@ ::: :: : :@:
| ::: ::: :#::: ::@ : : :: ::: :@ ::::: :: : :@:
| :::::: ::: :#::: ::@:@@ : : :::: : ::: :@ : ::: :: : :@:
| :::::::: : ::: :#::: ::@:@ : : ::: :::::: :@ : ::: ::: : :@:
| : :::::: : ::: :#::: ::@:@ : : ::: ::::: ::: :@::: ::: ::: @:::@:@
| : :::::: ::::::: :#::: ::@:@ :::: ::: :: :: ::: :@: : :::::::: @:@:@:@
| : :::::: ::: ::: :#::: ::@:@ ::: : ::: :: :: ::: :@: : :::: ::: @:@:@:@
| : :::::: ::: ::: :#::: ::@:@ ::: : ::: :: :: ::: :@: : :::: ::: @:@:@:@
| : :::::: ::: ::: :#::: ::@:@ ::: : ::: :: :: ::: :@: : :::: ::: @:@:@:@
| : :::::: ::: ::: :#::: ::@:@ ::: : ::: :: :: ::: :@: : :::: ::: @:@:@:@
| : :::::: ::: ::: :#::: ::@:@ ::: : ::: :: :: ::: :@: : :::: ::: @:@:@:@
0 +----------------------------------------------------------------------->Gi
0 827.5
about 70% of the peak memory is is used in ImagingNewArray, which is PIL's storage for entire images, A bunch in new block. Nothing else is in PIL/low. (much snippage below)
99.63% (132,411,972B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->69.32% (92,127,312B) 0x7AAD2EA: ImagingNewArray ([[Pillow/libImaging/]]Storage.c:287)
| ->69.32% (92,127,312B) 0x7A90289: _new ([[PIL/]]_imaging.c:633)
...
->13.14% (17,456,640B) 0x7AAD35D: ImagingNewBlock (Storage.c:333)
| ->13.14% (17,456,640B) 0x7AAD40A: ImagingNew (Storage.c:373)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment