- We were discussing the massive amounts of space that MongoDB consumes during build (as @infowolfe has PORTAGE_TMPDIR="/dev/shm")
- @robbat2 was planning on committing a warning message/disk space prerequisite check to the MongoDB ebuild on Gentoo Linux noting that during both build and install it takes up an impressive amount of disk space (@infowolfe saw >12GB of used tmpfs prior to merge).
- @robbat2 wanted to know exactly how much space was being consumed but didn't personally have MongoDB installed. He then asked me (@infowolfe) to run a perl one-liner[1] twice (once with /usr/ and once without). We calculated the differential between MongoDB's /usr and non-/usr packaged files to be roughly 180KiBytes on my system (
nostrip&&CFLAGS=-g).
- / (all filesystems) 2,473,771,415 (bytes)
- /usr/ (only /usr): 2,473,662,740 (bytes)
- differential: 108,675 bytes
- @robbat2 suggested that I drill down further into MongoDB's CONTENTS file, to see exactly where the ~2.4GB in total disappeared to. My first result outputted in raw bytes, not calculated like the current script:
#!/bin/bash
mcont="/var/db/pkg/dev-db/mongodb-2.*/CONTENTS"
for dir in $(grep ^dir ${mcont} | sed -e 's~^dir ~~') ; do
pdir=$(echo ${dir} | sed -e 's~/~\\/~g' -e 's~.*~&\\/~')
echo -en "${dir}:\t"
perl -e 'while(<>) { chomp; next unless $_ =~ /obj '${pdir}'/; @a = split /\s+/, $_; $sum += -s $a[1]; }; print $sum,"\n"; ' ${mcont}
done | awk {'printf "%-45s\t%-8s %s %s\n", $1, $2, $3, $4'}I didn't notice the output initially, but @robbat2 did, and pointed out this (note the 3 places of difference between directories):
freya:~$ ./run_perl.sh | egrep -e '^/usr(:|/include:|/lib/debug:)'
/usr: 2473662740
/usr/include: 1973187
/usr/lib/debug: 2340973686
Because I completely failed to see the above, I decided it would be prudent (if this script were ever to be reused), to do human-readable calculations on the output. This particular "tool" can actually be pointed at any Gentoo package's /var/db/pkg/$category/$PN-$PV/CONTENTS file to get a readout of disk space used on a by-directory basis. Because it was still not super easy to differentiate between say GiB and MiB when stacked vertically, I also colorized the bash output: blue (bytes), green (kilobytes, yellow (megabytes) and red (gigabytes). All figures are in base2, not base10 (Mibibyte = 1,048,576 vs Megabyte = 1,000,000). Anyway, if you have a use for this, enjoy - my portion (the bash) is being released into the public domain: this means you may use it any way you'd like including modifying and redistriibuting, and you don't need to cite my name if you do so. I also will not under any circumstances be supporting this or accepting patches. @robbat2 just informed me that his perl is licensed BSD-2.
Note: wrapped for convenience, please extract perl from above/below bash scripts for use.
perl -e 'while(<>) { chomp; next unless $_ =~ /obj \/usr\//;
@a = split /\s+/, $_; $sum += -s $a[1]; }; print $sum,"\n";'
/var/db/pkg/dev-db/mongodb-2.*/CONTENTS
