Skip to content

Instantly share code, notes, and snippets.

@jokea
Created May 25, 2012 08:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save jokea/2786537 to your computer and use it in GitHub Desktop.
Save jokea/2786537 to your computer and use it in GitHub Desktop.
fork+multithreaded+jemalloc = deadlock?
# cat a.c
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include "zmalloc.h"
void *worker(void *arg) {
int j;
char *p;
while (1) {
for (j=1; j<5000; ++j) {
p = zmalloc(j);
zfree(p);
}
}
return NULL;
}
int main(int argc, char **argv) {
int pid;
int j;
char *p;
pthread_t tid;
int n = 20;
fprintf(stderr, "%s\n", ZMALLOC_LIB);
if (argc > 1)
n = atoi(argv[1]);
for (j=0; j<10; ++j)
pthread_create(&tid, NULL, worker, NULL);
int i = 0;
while (i++<n) {
if ((pid = fork()) == 0) {
/* child */
fprintf(stderr, "child %d\n", i);
int cnt;
for (cnt=0; cnt<100; ++cnt) {
for (j=1; j<5000; ++j) {
p = zmalloc(j);
zfree(p);
}
}
exit(0);
}
usleep(10000);
}
sleep(1);
while (n--) {
fprintf(stderr, "%d children running...\n", n+1);
pid = wait(NULL);
}
return 0;
}
1. compile against jemalloc-2.2.5:
# gcc -g -ggdb -o 2.2.5 a.c zmalloc.c -DUSE_JEMALLOC ../deps/jemalloc/lib/libjemalloc.a -I../deps/jemalloc/include -lpthread -ldl
# ./2.2.5 4
jemalloc-2.2.5
child 1
child 2
child 3
child 4
4 children running...
3 children running...
2 children running...
1 children running...
^C
A child process gets blocked in jemalloc, backtrace:
(gdb) bt
#0 0x0000003e4ea08d4c in pthread_mutex_lock () from /lib64/libpthread.so.0
#1 0x000000000040242f in malloc_mutex_lock () at include/jemalloc/internal/mutex.h:52
#2 malloc_init_hard () at src/jemalloc.c:679
#3 0x0000000000402ef5 in malloc_init (size=1) at src/jemalloc.c:339
#4 je_malloc (size=1) at src/jemalloc.c:915
#5 0x000000000040163c in zmalloc (size=1) at zmalloc.c:97
#6 0x000000000040154e in main (argc=2, argv=0x7fffca705128) at a.c:42
2. jemalloc-3.0.0 has a bugfix(http://www.canonware.com/cgi-bin/gitweb.cgi?p=jemalloc.git;a=commit;h=4e2e3dd9cf19ed5991938a708a8b50611aa5bbf8) which may relate to this, try 3.0.0:
# gcc -g -ggdb -o 3.0.0 a.c zmalloc.c -DUSE_JEMALLOC ../deps/jemalloc/lib/libjemalloc.a -I../deps/jemalloc/include -lpthread -ldl
# ./3.0.0 4
jemalloc-3.0.0
child 1
child 2
child 3
child 4
4 children running...
3 children running...
2 children running...
^C
Two children are blocked, the first one has exactly the same backtrace with jemalloc-2.2.5,
the second child is a little different:
(gdb) bt
#0 0x0000003e4ea0d594 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003e4ea08e8a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003e4ea08d4c in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x0000000000401cdf in malloc_mutex_lock () at include/jemalloc/internal/mutex.h:77
#4 malloc_init_hard () at src/jemalloc.c:611
#5 0x00000000004037c5 in malloc_init (size=1) at src/jemalloc.c:282
#6 je_malloc (size=1) at src/jemalloc.c:804
#7 0x00000000004014fc in zmalloc (size=1) at zmalloc.c:97
#8 0x000000000040140e in main (argc=2, argv=0x7fffaaf17428) at a.c:42
3. compile against libc malloc:
# gcc -g -ggdb -o libc a.c zmalloc.c -lpthread -ldl
# ./libc 4
libc
child 1
child 2
child 3
child 4
4 children running...
3 children running...
2 children running...
1 children running...
#
@jokea
Copy link
Author

jokea commented Jun 19, 2012

Update:

  1. jemalloc below 3.0.0 will deadlock;
  2. adding a call to malloc before pthread_create in main() will not block using jemalloc-3.0.0;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment