Skip to content

Instantly share code, notes, and snippets.

@darragh

darragh/a-curl-gzip-sfo

Created August 3, 2012 11:37
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save darragh/3246837 to your computer and use it in GitHub Desktop.
Save darragh/3246837 to your computer and use it in GitHub Desktop.
Download ZIP
empty-image-file-from-some-cloudfront-edge-locations
Raw
a-curl-gzip-sfo
$ curl -i -H "Accept-Encoding: gzip,deflate" -v -o /dev/null 'https://dmhx3adjqsy1o.cloudfront.net/assets/marketing/whatwedo0201-cf9b0a547fed9462515d48cfac18135c.jpg'* About to connect() to dmhx3adjqsy1o.cloudfront.net port 443 (#0)
* Trying 205.251.203.120... % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0connected
* Connected to dmhx3adjqsy1o.cloudfront.net (205.251.203.120) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Server hello (2):
{ [data not shown]
* SSLv3, TLS handshake, CERT (11):
{ [data not shown]
* SSLv3, TLS handshake, Server finished (14):
{ [data not shown]
* SSLv3, TLS handshake, Client key exchange (16):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Finished (20):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
{ [data not shown]
* SSLv3, TLS handshake, Finished (20):
{ [data not shown]
* SSL connection using RC4-MD5
* Server certificate:
* subject: C=US; ST=Washington; L=Seattle; O=Amazon.com, Inc.; CN=*.cloudfront.net
* start date: 2010-11-10 00:00:00 GMT
* expire date: 2013-11-13 23:59:59 GMT
* subjectAltName: dmhx3adjqsy1o.cloudfront.net matched
* issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert High Assurance CA-3
* SSL certificate verify ok.
> GET /assets/marketing/whatwedo0201-cf9b0a547fed9462515d48cfac18135c.jpg HTTP/1.1
> User-Agent: curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5
> Host: dmhx3adjqsy1o.cloudfront.net
> Accept: */*
> Accept-Encoding: gzip,deflate
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Cache-Control: public, max-age=2592000
< Content-Encoding: gzip
< Content-Type: image/jpeg
< Date: Fri, 13 Jul 2012 16:25:27 GMT
< Last-Modified: Fri, 13 Jul 2012 15:40:48 GMT
< Status: 200 OK
< Age: 1740787
< Content-Length: 0
< Vary: Accept-Encoding
< X-Cache: Hit from cloudfront
< X-Amz-Cf-Id: W7OGpcCvbF8XSKCg53_ZWxkptWVpD7_Fq54GlUOxWUIq-DFZDMqXSw==
< Via: 1.0 d87c8384a8e238b5a83c2709b74c48e7.cloudfront.net (CloudFront)
< Connection: close
<
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
} [data not shown]
Raw
b-curl-gzip-dub
$ curl -i -H "Accept-Encoding: gzip,deflate" -v -o /dev/null 'https://dmhx3adjqsy1o.cloudfront.net/assets/marketing/whatwedo0201-cf9b0a547fed9462515d48cfac18135c.jpg'
* About to connect() to dmhx3adjqsy1o.cloudfront.net port 443 (#0)
* Trying 54.240.184.203...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* connected
* Connected to dmhx3adjqsy1o.cloudfront.net (54.240.184.203) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Server hello (2):
{ [data not shown]
* SSLv3, TLS handshake, CERT (11):
{ [data not shown]
* SSLv3, TLS handshake, Server finished (14):
{ [data not shown]
* SSLv3, TLS handshake, Client key exchange (16):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Finished (20):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
{ [data not shown]
* SSLv3, TLS handshake, Finished (20):
{ [data not shown]
* SSL connection using RC4-MD5
* Server certificate:
* subject: C=US; ST=Washington; L=Seattle; O=Amazon.com, Inc.; CN=*.cloudfront.net
* start date: 2010-11-10 00:00:00 GMT
* expire date: 2013-11-13 23:59:59 GMT
* subjectAltName: dmhx3adjqsy1o.cloudfront.net matched
* issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert High Assurance CA-3
* SSL certificate verify ok.
> GET /assets/marketing/whatwedo0201-cf9b0a547fed9462515d48cfac18135c.jpg HTTP/1.1
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
> Host: dmhx3adjqsy1o.cloudfront.net
> Accept: */*
> Accept-Encoding: gzip,deflate
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Cache-Control: public, max-age=2592000
< Content-Encoding: gzip
< Content-Type: image/jpeg
< Date: Fri, 13 Jul 2012 19:30:12 GMT
< Last-Modified: Fri, 13 Jul 2012 17:13:41 GMT
< Status: 200 OK
< Age: 1729640
< Content-Length: 18889
< Vary: Accept-Encoding
< X-Cache: Hit from cloudfront
< X-Amz-Cf-Id: gfDkyqycVi-onlJxiXysOe174FhEAEVLW-L0F_xSE3ePBB55pTs4TQ==
< Via: 1.0 b8923d74909732ec15f370879cbee3ce.cloudfront.net (CloudFront)
< Connection: close
<
{ [data not shown]
100 18889 100 18889 0 0 22008 0 --:--:-- --:--:-- --:--:-- 121k
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
} [data not shown]
Raw
c-dig identity.cloudfront.net txt
$ dig identity.cloudfront.net txt
; <<>> DiG 9.7.3-P3 <<>> identity.cloudfront.net txt
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5966
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2
;; QUESTION SECTION:
;identity.cloudfront.net. IN TXT
;; ANSWER SECTION:
identity.cloudfront.net. 60 IN TXT "ns-lax1-02.cloudfront.net.lax1"
;; AUTHORITY SECTION:
cloudfront.net. 163105 IN NS ns-02.cloudfront.net.
cloudfront.net. 163105 IN NS ns-01.cloudfront.net.
;; ADDITIONAL SECTION:
ns-01.cloudfront.net. 172796 IN A 207.171.170.1
ns-02.cloudfront.net. 172784 IN A 207.171.179.1
;; Query time: 23 msec
;; SERVER: 192.168.255.2#53(192.168.255.2)
;; WHEN: Thu Aug 2 13:03:22 2012
;; MSG SIZE rcvd: 156
Raw
d-dig resolver-identity.cloudfront.net txt
$ dig resolver-identity.cloudfront.net txt
; <<>> DiG 9.7.3-P3 <<>> resolver-identity.cloudfront.net txt
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 9730
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2
;; QUESTION SECTION:
;resolver-identity.cloudfront.net. IN TXT
;; ANSWER SECTION:
resolver-identity.cloudfront.net. 10 IN TXT "64.71.0.34"
;; AUTHORITY SECTION:
cloudfront.net. 169442 IN NS ns-02.cloudfront.net.
cloudfront.net. 169442 IN NS ns-01.cloudfront.net.
;; ADDITIONAL SECTION:
ns-01.cloudfront.net. 172747 IN A 207.171.170.1
ns-02.cloudfront.net. 172718 IN A 207.171.179.1
;; Query time: 34 msec
;; SERVER: 192.168.255.2#53(192.168.255.2)
;; WHEN: Thu Aug 2 13:03:03 2012
;; MSG SIZE rcvd: 145
Raw
e-traceroute dmhx3adjqsy1o.cloudfront.net
$ traceroute dmhx3adjqsy1o.cloudfront.net
traceroute: Warning: dmhx3adjqsy1o.cloudfront.net has multiple addresses; using 205.251.203.248
traceroute to dmhx3adjqsy1o.cloudfront.net (205.251.203.248), 64 hops max, 52 byte packets
1 192.168.1.1 (192.168.1.1) 2.660 ms 1.198 ms 1.288 ms
2 192.168.255.2 (192.168.255.2) 1.962 ms 4.641 ms 1.707 ms
3 64-71-28-57.static.wiline.com (64.71.28.57) 2.683 ms 2.288 ms 2.192 ms
4 sfosf0116r352401-v22.wiline.com (64.71.19.178) 4.515 ms 3.200 ms 3.583 ms
5 sfosf0002r372401-v24.wiline.com (64.71.19.246) 4.156 ms 3.884 ms 4.234 ms
6 sfosf0264r352401-v45.wiline.com (108.60.125.2) 3.898 ms 6.502 ms 3.579 ms
7 sfosf0114r352401-v22.wiline.com (108.60.124.250) 6.985 ms 4.826 ms 5.096 ms
8 sfosf0350r352401-v20.wiline.com (108.60.120.22) 6.431 ms 6.917 ms 7.170 ms
9 sfosf0014r352401-v22.wiline.com (70.35.44.193) 7.929 ms 6.772 ms 6.746 ms
10 sfosf0072r372401-v23.wiline.com (108.60.117.65) 6.014 ms 11.228 ms 9.808 ms
11 sfosf0003r372403-v12.wiline.com (64.71.4.5) 10.760 ms 5.037 ms 5.801 ms
12 sfosf0003rm10i01-ae2-v3.wiline.com (67.207.106.185) 8.385 ms 5.194 ms 6.042 ms
13 port-channel201.car2.sanfrancisco1.level3.net (4.53.130.57) 84.865 ms 11.930 ms 61.453 ms
14 ae-2-4.bar2.sanfrancisco1.level3.net (4.69.133.158) 15.394 ms 8.456 ms 5.886 ms
15 ae-6-6.ebr2.sanjose1.level3.net (4.69.140.154) 6.949 ms 7.032 ms 6.961 ms
16 ae-2-2.ebr2.sanjose5.level3.net (4.69.148.141) 6.671 ms 9.583 ms 13.849 ms
17 ae-6-6.ebr2.losangeles1.level3.net (4.69.148.201) 18.459 ms 20.789 ms 16.701 ms
18 ae-92-92.csw4.losangeles1.level3.net (4.69.137.30) 20.035 ms
ae-62-62.csw1.losangeles1.level3.net (4.69.137.18) 17.852 ms
ae-82-82.csw3.losangeles1.level3.net (4.69.137.26) 17.561 ms
19 ae-3-80.edge5.losangeles1.level3.net (4.69.144.139) 21.387 ms *
ae-2-70.edge5.losangeles1.level3.net (4.69.144.75) 28.120 ms
20 amazon.com.edge5.losangeles1.level3.net (205.129.4.26) 19.931 ms 20.756 ms 19.400 ms
21 server-205-251-203-248.lax3.r.cloudfront.net (205.251.203.248) 25.667 ms 17.019 ms 22.572 ms
Raw
f-length-ordered-by-length
zgrep whatwedo0201-cf9b0a547fed9462515d48cfac18135c *.gz | cut -f3,4,6,9 | grep -v 304 | sort | uniq | sort -k 2 -n
LAX3 457 GET 200
LAX3 513 GET 200
LAX3 514 GET 200
LAX3 515 GET 200
LAX3 516 GET 200
LAX3 537 GET 200
GRU1 7787 GET 200
SFO5 19350 GET 200
IAD12 19351 GET 200
JFK5 19351 GET 200
LHR5 19351 GET 200
LHR3 19352 GET 200
AMS1 19353 GET 200
ARN1 19353 GET 200
CDG3 19353 GET 200
CDG50 19353 GET 200
DUB2 19353 GET 200
FRA2 19353 GET 200
FRA6 19353 GET 200
GRU1 19353 GET 200
IAD12 19353 GET 200
IND6 19353 GET 200
JAX1 19353 GET 200
JFK1 19353 GET 200
JFK5 19353 GET 200
LAX3 19353 GET 200
LHR5 19353 GET 200
SEA4 19353 GET 200
SFO5 19353 GET 200
SIN2 19353 GET 200
SIN3 19353 GET 200
AMS50 19405 GET 200
IAD12 19406 GET 200
IND6 19406 GET 200
JAX1 19406 GET 200
LHR5 19406 GET 200
AMS1 19407 GET 200
FRA2 19407 GET 200
IAD12 19407 GET 200
JAX1 19407 GET 200
JFK5 19407 GET 200
LHR5 19407 GET 200
SEA4 19407 GET 200
SFO5 19407 GET 200
AMS1 19408 GET 200
CDG3 19408 GET 200
DUB2 19408 GET 200
EWR2 19408 GET 200
FRA2 19408 GET 200
FRA6 19408 GET 200
IAD12 19408 GET 200
IND6 19408 GET 200
JFK5 19408 GET 200
LHR5 19408 GET 200
SFO5 19408 GET 200
STL2 19408 GET 200
CDG3 19409 GET 200
DUB2 19409 GET 200
FRA2 19409 GET 200
IAD2 19409 GET 200
IND6 19409 GET 200
JFK5 19409 GET 200
LHR3 19409 GET 200
LHR5 19409 GET 200
SFO5 19409 GET 200
AMS1 19410 GET 200
CDG3 19410 GET 200
CDG50 19410 GET 200
DFW3 19410 GET 200
DFW50 19410 GET 200
EWR2 19410 GET 200
FRA6 19410 GET 200
GRU1 19410 GET 200
HKG1 19410 GET 200
IAD12 19410 GET 200
IND6 19410 GET 200
JAX1 19410 GET 200
JFK1 19410 GET 200
JFK5 19410 GET 200
LAX1 19410 GET 200
LAX3 19410 GET 200
LHR5 19410 GET 200
MIA3 19410 GET 200
MXP4 19410 GET 200
SEA4 19410 GET 200
SFO5 19410 GET 200
SIN2 19410 GET 200
SIN3 19410 GET 200
STL2 19410 GET 200
SYD1 19410 GET 200
SIN2 19411 GET 200
AMS1 19426 GET 200
AMS50 19426 GET 200
CDG3 19426 GET 200
DUB2 19426 GET 200
EWR2 19426 GET 200
FRA2 19426 GET 200
FRA6 19426 GET 200
IAD12 19426 GET 200
IND6 19426 GET 200
JAX1 19426 GET 200
JFK5 19426 GET 200
LHR5 19426 GET 200
MXP4 19426 GET 200
SEA4 19426 GET 200
SFO5 19426 GET 200
STL2 19426 GET 200
SEA4 22073 GET 200
JFK5 22074 GET 200
SFO5 22074 GET 200
Raw
g-LAX3-good-to-bad
zgrep whatwedo0201-cf9b0a547fed9462515d48cfac18135c *.gz | sort --key=4 | grep LAX3 | sort | cut -f1,2,3,4,6,9
E1UP7EXJME74HF.2012-07-13-01.bOzeekgW.gz:2012-07-13 01:01:43 LAX3 19410 GET 200
E1UP7EXJME74HF.2012-07-13-02.DfZtOUs3.gz:2012-07-13 02:25:35 LAX3 19410 GET 200
E1UP7EXJME74HF.2012-07-13-02.Fhqdtyup.gz:2012-07-13 02:52:23 LAX3 19353 GET 200
E1UP7EXJME74HF.2012-07-13-04.XN2rGaQP.gz:2012-07-13 04:44:30 LAX3 19353 GET 200
E1UP7EXJME74HF.2012-07-13-05.M8CXL9WE.gz:2012-07-13 05:11:15 LAX3 19353 GET 200
E1UP7EXJME74HF.2012-07-13-05.xo1kZ95n.gz:2012-07-13 05:14:05 LAX3 19410 GET 200
E1UP7EXJME74HF.2012-07-13-14.DcZFGPBV.gz:2012-07-13 14:52:48 LAX3 19410 GET 200
E1UP7EXJME74HF.2012-07-13-16.6YOEzOS4.gz:2012-07-13 16:48:01 LAX3 514 GET 200
E1UP7EXJME74HF.2012-07-13-16.9yddi1lZ.gz:2012-07-13 16:37:58 LAX3 513 GET 200
E1UP7EXJME74HF.2012-07-13-16.Q7vM1d7X.gz:2012-07-13 16:25:38 LAX3 537 GET 200
E1UP7EXJME74HF.2012-07-13-17.bEumEd7E.gz:2012-07-13 17:30:42 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-18.JQV8ayw2.gz:2012-07-13 18:29:21 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-18.bMTeBbEo.gz:2012-07-13 18:57:04 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-18.bo2I5q30.gz:2012-07-13 18:02:09 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-18.bo2I5q30.gz:2012-07-13 18:02:15 LAX3 457 GET 200
E1UP7EXJME74HF.2012-07-13-18.xESRmbWu.gz:2012-07-13 18:43:16 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-19.mW7xeLNj.gz:2012-07-13 19:03:06 LAX3 514 GET 200
E1UP7EXJME74HF.2012-07-13-20.HBne1WkM.gz:2012-07-13 20:49:04 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-20.QdQQR10r.gz:2012-07-13 20:02:41 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-20.ab7efgRR.gz:2012-07-13 20:46:50 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-20.i08rg9TC.gz:2012-07-13 20:15:47 LAX3 516 GET 200
E1UP7EXJME74HF.2012-07-13-20.wuQDsfFe.gz:2012-07-13 20:23:15 LAX3 516 GET 200
E1UP7EXJME74HF.2012-07-13-20.yNLoBqyH.gz:2012-07-13 20:07:04 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-21.Iq0bpIex.gz:2012-07-13 21:22:12 LAX3 516 GET 200
E1UP7EXJME74HF.2012-07-13-21.RLGTWQxR.gz:2012-07-13 21:16:59 LAX3 516 GET 200
E1UP7EXJME74HF.2012-07-13-21.sC4byq4W.gz:2012-07-13 21:14:50 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-21.zxAztuQD.gz:2012-07-13 21:36:28 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-22.83w22xHt.gz:2012-07-13 22:17:06 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-22.FdWu55rf.gz:2012-07-13 22:47:22 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-22.q1YtCY0n.gz:2012-07-13 22:58:22 LAX3 516 GET 200
E1UP7EXJME74HF.2012-07-13-23.KgpwrV4P.gz:2012-07-13 23:22:23 LAX3 515 GET 200
E1UP7EXJME74HF.2012-07-13-23.SgDVzdXo.gz:2012-07-13 23:18:39 LAX3 515 GET 200
Raw
h-GRU1-looks-weird-too
Looks like it's neither an empty nor file of expected size:
E1UP7EXJME74HF.2012-07-13-22.EQlyyH3n.gz:2012-07-13 22:03:02 GRU1 7787 GET 200
Raw
i-0-byte-200-ok-heroku-only-head
(unfortunately we've only got 1 week of retention on our logs :( - but in the last 7 days there have been no occurrences of 0 length filtes served by heroku.
All these requests are HEAD => meant to be 0 bytes.
237 <158>1 2012-07-30T20:56:56+00:00 heroku router - - HEAD www.intercom.io/assets/marketing/whatwedo0303-7057356f300a76d415b482c4409d9611.png dyno=web.6 queue=0 wait=0ms service=1ms status=200 bytes=0
237 <158>1 2012-07-30T20:56:56+00:00 heroku router - - HEAD www.intercom.io/assets/marketing/whatwedo0401-9e1d66a9e90e6c0791c2daf3b84750bd.jpg dyno=web.2 queue=0 wait=0ms service=2ms status=200 bytes=0
237 <158>1 2012-07-30T20:56:56+00:00 heroku router - - HEAD www.intercom.io/assets/marketing/whatwedo0307-df2e3dfbda6e749a5b166b33d0f2a0c2.png dyno=web.1 queue=0 wait=0ms service=5ms status=200 bytes=0
237 <158>1 2012-07-30T20:56:57+00:00 heroku router - - HEAD www.intercom.io/assets/marketing/whatwedo0103-04a3573d122f743b643f1726b2dc209e.png dyno=web.2 queue=0 wait=0ms service=2ms status=200 bytes=0
237 <158>1 2012-07-30T20:56:57+00:00 heroku router - - HEAD www.intercom.io/assets/marketing/whatwedo0304-740a87100550cc5247306856d35ac7ca.jpg dyno=web.6 queue=0 wait=0ms service=2ms status=200 bytes=0
237 <158>1 2012-07-30T20:56:57+00:00 heroku router - - HEAD www.intercom.io/assets/marketing/whatwedo0301-daa2022ee61e82bd55ba1737b52b7af2.jpg dyno=web.5 queue=0 wait=0ms service=1ms status=200 bytes=0
237 <158>1 2012-07-30T20:56:57+00:00 heroku router - - HEAD www.intercom.io/assets/marketing/whatwedo0308-266bd5a255eb9721a3a0bf162a9ca7e1.jpg dyno=web.3 queue=0 wait=0ms service=3ms status=200 bytes=0
237 <158>1 2012-07-30T20:56:57+00:00 heroku router - - HEAD www.intercom.io/assets/marketing/whatwedo0201-cf9b0a547fed9462515d48cfac18135c.jpg dyno=web.5 queue=0 wait=0ms service=5ms status=200 bytes=0
237 <158>1 2012-07-30T20:56:58+00:00 heroku router - - HEAD www.intercom.io/assets/marketing/whatwedo0310-0bfac2f3502a03e4931bf7e55c23b805.png dyno=web.2 queue=0 wait=0ms service=9ms status=200 bytes=0
237 <158>1 2012-08-02T16:01:26+00:00 heroku router - - HEAD www.intercom.io/assets/marketing/whatwedo0101-5ae1b6d86617ec413caf47399a5bdcb6.jpg dyno=web.6 queue=0 wait=0ms service=1ms status=200 bytes=0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment