Skip to content

Instantly share code, notes, and snippets.

@nickfishman
Last active July 21, 2021 22:16
Show Gist options
  • Star 6 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
  • Save nickfishman/5499763 to your computer and use it in GitHub Desktop.
Save nickfishman/5499763 to your computer and use it in GitHub Desktop.
Second attempt at conditional gzip decoding based on the Content-Encoding response header with the mikeal/request library for Node.js. This DOES work.
var request = require('request'),
zlib = require('zlib');
var headers = {
"accept-charset" : "ISO-8859-1,utf-8;q=0.7,*;q=0.3",
"accept-language" : "en-US,en;q=0.8",
"accept" : "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"user-agent" : "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/537.13+ (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2",
"accept-encoding" : "gzip,deflate",
};
var options = {
url: "http://google.com",
headers: headers
};
var requestWithEncoding = function(options, callback) {
var req = request.get(options);
req.on('response', function(res) {
var chunks = [];
res.on('data', function(chunk) {
chunks.push(chunk);
});
res.on('end', function() {
var buffer = Buffer.concat(chunks);
var encoding = res.headers['content-encoding'];
if (encoding == 'gzip') {
zlib.gunzip(buffer, function(err, decoded) {
callback(err, decoded && decoded.toString());
});
} else if (encoding == 'deflate') {
zlib.inflate(buffer, function(err, decoded) {
callback(err, decoded && decoded.toString());
})
} else {
callback(null, buffer.toString());
}
});
});
req.on('error', function(err) {
callback(err);
});
}
requestWithEncoding(options, function(err, data) {
if (err) console.log(err);
else console.log(data);
})
@Bersam
Copy link

Bersam commented Dec 15, 2013

Yes, this is working like a charm, thanks!

@colthreepv
Copy link

Still works like a charm.

In case anyone would like to make it better: the callback function could have the same request parameters (error, response, body), with the body one deflated.

@beygi
Copy link

beygi commented Oct 7, 2014

and still works like a charm

@Deathspike
Copy link

Helped me, thanks! To support deflate content encoding, change inflate to inflateRaw.

@dmasior
Copy link

dmasior commented Nov 16, 2015

great :)

@xudaolong
Copy link

非常感谢,已经学习到~(≧▽≦)/~啦啦啦....Thank you very much, has been learning to \ ^ _ ^/ La La La. ..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment