Skip to content

Instantly share code, notes, and snippets.

@m6w6
Created August 21, 2014 16:53
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save m6w6/2a71d5f9a4db108362b2 to your computer and use it in GitHub Desktop.
Save m6w6/2a71d5f9a4db108362b2 to your computer and use it in GitHub Desktop.
Here's a simple program to replicate the tackled problem.
#include <stdlib.h>
#include <stdio.h>
#include <curl/curl.h>
int main(int argc, char *argv[]) {
CURLMcode mrc;
CURLM *multi = NULL;
CURL *easy[100] = {NULL};
int i, running, finished;
FILE *devnull;
curl_global_init(CURL_GLOBAL_NOTHING);
devnull = fopen("/dev/null", "w");
if (!devnull)
goto kthxbye;
multi = curl_multi_init();
if (!multi)
goto kthxbye;
for (i=0; i<100; ++i) {
easy[i] = curl_easy_init();
if (!easy[i])
goto kthxbye;
curl_easy_setopt(easy[i], CURLOPT_WRITEDATA, devnull);
curl_easy_setopt(easy[i], CURLOPT_CONNECTTIMEOUT, 30);
curl_easy_setopt(easy[i], CURLOPT_TIMEOUT, 30);
curl_easy_setopt(easy[i], CURLOPT_URL, "http://www.google.com");
curl_multi_add_handle(multi, easy[i]);
}
do {
CURLMsg *msg;
while (CURLM_CALL_MULTI_PERFORM == (mrc = curl_multi_perform(multi, &running)));
while ((msg = curl_multi_info_read(multi, &finished))) {
long code;
if (msg->data.result == CURLE_OK)
curl_easy_getinfo(msg->easy_handle, CURLINFO_HTTP_CODE, &code);
else
code = -((long) msg->data.result);
printf("%ld ", code);
curl_multi_remove_handle(multi, msg->easy_handle);
}
} while (mrc == CURLM_OK && running > 0);
kthxbye:
if (multi)
curl_multi_cleanup(multi);
for (i=0; i<100; ++i)
if (easy[i])
curl_easy_cleanup(easy[i]);
curl_global_cleanup();
exit(0);
}
$ time ./100googles
302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 -6 -6 -6 -6 -6 -6 -6 -6 -6 -6 -6 -6 -6 -6 -6 -6 -6 -6 -6 302 302 302 -6 -6 -6 -6 -6 -6 -6 -6 -6 -6 -6 -6 ./100googles 12.81s user 2.27s system 100% cpu 15.037 total
$ time LD_PRELOAD=./lib/.libs/libcurl.so ./100googles
302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 LD_PRELOAD=./lib/.libs/libcurl.so ./100googles 0.07s user 0.07s system 119% cpu 0.117 total
@m6w6
Copy link
Author

m6w6 commented Aug 21, 2014

The first run with stock libcurl completes in 5 to 15 seconds (probably depending on your resolver setup) and has lots of COULDNT_RESOLVE_HOST or OPERATION_TIMEDOUT result codes.

The fixed version completes in nearly a centisecond.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment