Moved to: https://docs.google.com/document/d/1JnQZ-k63D2BN6-u8cZhqZzDsj1R7vKl9nWbHeDI4oeI/edit
Attendees:
- Greg Tatum
- Stephen Fink
- Jon Coppeard
The garbage collector in JavaScript does not use malloc to get its memory. It uses mmap on Unix systems (and another function on windows), similar to how jemalloc manages to get pages of memory. This bypasses the hooks.
A straightforward follow-up would be to hook into the GC's use of mmap for access the "chunks" of memory, which are allocated in 1mb sizes, or hooking into its use of "arenas" which allocate 4kb into the "chunks".
Talking through the problem, some memory is allocated through malloc, and some is being allocated through the GC's chunks. For instance, allocating a Float32Array would create a JS object in the GC memory, and then the actual array data would be stored in malloc.
For someone writing JavaScript, the abstraction of how much memory their JavaScript is using somewhat breaks down due to the level that the JS engine shares memory and reuses memory internally when creating objects. To my feeling, this makes it somewhat hard to really instrument the JS memory size for user-written code. I think the heap walking code in the DevTools memory tool probably provides the best data for this, but the UX of using the tool is still quite difficult.
We reviewed the JS allocations information. The general view was that the memory usage being reported was quite low. The sampling does not account for byte size, which may contribute to this. In addition, I'm not confident we're getting all the correct information. It would be worth chatting with fitzgen or jimb a bit about these concerns.
The GCRuntime already keeps track of heap memory usage. It would be fairly straightforward to turn that into counter information using public apis.
Garbage Collector: https://searchfox.org/mozilla-central/source/js/src/gc/GC.cpp
mmap call: (Is this right?) https://searchfox.org/mozilla-central/rev/cbd110d718bc89a499d3f996af24532abbf6f8ea/js/src/gc/Memory.cpp#931
mmap in jemalloc: https://searchfox.org/mozilla-central/rev/cbd110d718bc89a499d3f996af24532abbf6f8ea/memory/build/mozjemalloc.cpp#235
Potential hook location for arenas: https://searchfox.org/mozilla-central/rev/cbd110d718bc89a499d3f996af24532abbf6f8ea/js/src/gc/Allocator.cpp#689
Heap size is stored here: https://searchfox.org/mozilla-central/rev/cbd110d718bc89a499d3f996af24532abbf6f8ea/js/src/gc/GCRuntime.h#810
Public link: https://searchfox.org/mozilla-central/rev/cbd110d718bc89a499d3f996af24532abbf6f8ea/js/src/gc/GC.cpp#1399
Demo profile: https://perfht.ml/3aQk5Td