Skip to content

Instantly share code, notes, and snippets.

@chhantyal
Last active June 14, 2022 14:37
Show Gist options
  • Save chhantyal/101f5d99a63f245ba94db91ec134f589 to your computer and use it in GitHub Desktop.
Save chhantyal/101f5d99a63f245ba94db91ec134f589 to your computer and use it in GitHub Desktop.
One thousands (1000) HTTP requests: async using NodeJs vs sync using Python vs using Python (with threads)
// npm install request
let request = require('request')
let range = n => Array.from(Array(n).keys())
data = range(1000)
data.forEach(function (item) {
request.get("https://httpbin.org/ip", function (error, response, body){
console.log("Request " + item + " complete.")
});
})
# pip install requests
import requests
data = range(1000)
for item in data:
requests.get("https://httpbin.org/ip")
print("Request {} complete.".format(item))
# pip install requests
import requests
from concurrent import futures
data = range(1000)
def send_request(item):
requests.get("https://httpbin.org/ip")
print("Request {} complete.".format(item))
executor = futures.ThreadPoolExecutor(max_workers=25)
futures_ = [executor.submit(send_request, x) for x in data]
for f in futures.as_completed(futures_):
f.result()

Results

TL;DR Async is very fast 🚀

But with threading, Python gives similar speed.

Async

$ time node send.js

real  0m15.233s
user  0m3.500s
sys  0m0.583s

Sync

$ time python send.py

real  11m38.011s
user  0m19.493s
sys  0m1.433s

Threads

$ time python send_thread.py

real  0m23.800s
user  0m25.393s
sys	 0m3.243s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment