Using squid and data compression proxy to reduce traffic for cat pictures

squid-with-data-compression.md

Squid

Squid is a HTTP(S) caching proxy to reduce bandwidth and improves response times.

data compression proxy

Data compression proxy is a http proxy by google aimming to reduce cellular data usage by gzip and webp (Yes, it will compress jpg, png and gif to webp).

Squid + data compression proxy

The data compression proxy requires an extra http header "Chrome-Proxy", the details of this field can be found here

To combine them together, I've patched the Squid souce code to add this header for each out going requests.

Squid + data compression proxy + WPAD

To deploy this proxy to all clients on my home work, I am using WPAD For the WPAD, in my case, I am hosting this wpad.dat on openwrt router and I've configured both DHCP and DNS to cover most devices in my home network.

Screenshots

Here is a comparsion of data usage for opening a webpage with lots of pictures.

Without "Squid + data compression proxy"

With "Squid + data compression proxy"

That's a 2.7MB saving for a single page!

TODO

• Some websites doesn't work well with the compression proxy, maybe we could teach squid to connect the target site directly
• Stats for compression and data saving
• Better whitelist for "DIRECT" sites in wpad.dat

wpad.dat

function FindProxyForURL(url, host) {

// If the requested website is hosted within the internal network, send direct.
    if (isPlainHostName(host) ||
        shExpMatch(host, "*.local") ||
        isInNet(dnsResolve(host), "10.0.0.0", "255.0.0.0") ||
        isInNet(dnsResolve(host), "172.16.0.0",  "255.240.0.0") ||
        isInNet(dnsResolve(host), "192.168.0.0",  "255.255.0.0") ||
        isInNet(dnsResolve(host), "127.0.0.0", "255.255.255.0"))
        return "DIRECT";

// If HTTP, send to squid at potato.
    if (url.substring(0, 5)=="http:")
        return "PROXY potato:3128; DIRECT";

    return "DIRECT";
}

Hi, have you by chance updated this recently? I have successfully rolled a squid package with the diffs from your tree, but Google's proxy is throwing errors when connecting to it. Based on a review of the Chrome plugin it looks like the proxy header might have changed. Before I try to go about figuring that out for myself I thought I might ask if you had already done it. Thanks!

FWIW, the auth header functions from the JS plugin appear to be here:
https://github.com/cnbeining/datacompressionproxy/blob/master/background.js