Skip to content

Instantly share code, notes, and snippets.

@austinfrey
Created November 13, 2020 04:16
Show Gist options
  • Save austinfrey/74859e7cc48530578c2cf73ce0827345 to your computer and use it in GitHub Desktop.
Save austinfrey/74859e7cc48530578c2cf73ce0827345 to your computer and use it in GitHub Desktop.
WARC/1.1
WARC-Filename: my-web-archive.warc
WARC-Date: 2020-11-13T04:09:42.143Z
WARC-Type: warcinfo
WARC-Record-ID: <urn:uuid:2ce22571-38e4-4528-91cb-4f9bfdd4a904>
Content-Type: application/warc-fields
Content-Length: 475
software: Webrecorder wabac.js/warcio.js
format: WARC File Format 1.1
isPartOf: My Web Archive
json-metadata: {"desc":"","title":"My Web Archive","pages":[{"title":"Example Domain","url":"http://example.com/","date":"2020-11-13T04:09:08.238Z","id":"j6rektpcjxeg0dvgg1wcr5"},{"title":"http://localhost:3000/","url":"http://localhost:3000/","date":"2020-11-13T04:08:59.267Z","id":"m1zf4unmxraaqwhavlmuu"}],"pageLists":[],"config":{"useSurt":false,"decodeResponses":false}}
WARC/1.1
WARC-Page-ID: j6rektpcjxeg0dvgg1wcr5
WARC-Payload-Digest: sha-256:ea8fac7c65fb589b0d53560f5251f74f9e9b243478dcb6b3ea79b5e36449c8d9
WARC-Target-URI: http://example.com/
WARC-Date: 2020-11-13T04:09:08.238Z
WARC-Type: response
WARC-Record-ID: <urn:uuid:e6274fea-178f-448c-ba34-e382b4c78037>
Content-Type: application/http; msgtype=response
WARC-Block-Digest: sha-256:ddb380cd606a03b9eeeb4bf56a8b60c5259ba21bdd681268c013d79ec96c9eba
Content-Length: 1667
HTTP/1.1 200 OK
Age: 303071
Cache-Control: max-age=604800
Connection: Keep-Alive
Content-Encoding: gzip
Content-Length: 648
Content-Type: text/html; charset=UTF-8
Date: Fri, 13 Nov 2020 04:09:08 GMT
Etag: "3147526947+gzip"
Expires: Fri, 20 Nov 2020 04:09:08 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Proxy-Connection: Keep-Alive
Server: ECS (phd/FD6D)
Vary: Accept-Encoding
X-Cache: HIT
<!doctype html>
<html>
<head>
<title>Example Domain</title>
<meta charset="utf-8" />
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<style type="text/css">
body {
background-color: #f0f0f2;
margin: 0;
padding: 0;
font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
}
div {
width: 600px;
margin: 5em auto;
padding: 2em;
background-color: #fdfdff;
border-radius: 0.5em;
box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);
}
a:link, a:visited {
color: #38488f;
text-decoration: none;
}
@media (max-width: 700px) {
div {
margin: 0 auto;
width: auto;
}
}
</style>
</head>
<body>
<div>
<h1>Example Domain</h1>
<p>This domain is for use in illustrative examples in documents. You may use this
domain in literature without prior coordination or asking for permission.</p>
<p><a href="https://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>
WARC/1.1
WARC-Page-ID: j6rektpcjxeg0dvgg1wcr5
WARC-Concurrent-To: <urn:uuid:e6274fea-178f-448c-ba34-e382b4c78037>
WARC-Target-URI: http://example.com/
WARC-Date: 2020-11-13T04:09:08.238Z
WARC-Type: request
WARC-Record-ID: <urn:uuid:f2f26eb3-94f9-4bc3-8060-95afc5cc2c10>
Content-Type: application/http; msgtype=request
WARC-Payload-Digest: sha-256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
WARC-Block-Digest: sha-256:56d4a3641f287f12575470ff5a97fb5558f7c003aa6b5f78fb7ad6f8154c75b4
Content-Length: 480
HTTP/1.1 200 OK
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Cache-Control: no-cache
Host: example.com
Pragma: no-cache
Proxy-Connection: keep-alive
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36
WARC/1.1
WARC-Page-ID: j6rektpcjxeg0dvgg1wcr5
WARC-Payload-Digest: sha-256:ea8fac7c65fb589b0d53560f5251f74f9e9b243478dcb6b3ea79b5e36449c8d9
WARC-Target-URI: http://example.com/favicon.ico
WARC-Date: 2020-11-13T04:09:08.355Z
WARC-Type: revisit
WARC-Profile: http://netpreserve.org/warc/1.1/revisit/identical-payload-digest
WARC-Refers-To-Target-URI: http://example.com/
WARC-Refers-To-Date: 2020-11-13T04:09:08.238Z
WARC-Record-ID: <urn:uuid:1ff5eee1-22a3-4459-94eb-ffc2e12687cb>
Content-Type: application/http; msgtype=response
Content-Length: 411
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 83316
Cache-Control: max-age=604800
Connection: Keep-Alive
Content-Encoding: gzip
Content-Length: 648
Content-Type: text/html; charset=UTF-8
Date: Fri, 13 Nov 2020 04:09:08 GMT
Expires: Fri, 20 Nov 2020 04:09:08 GMT
Last-Modified: Thu, 12 Nov 2020 05:00:32 GMT
Proxy-Connection: Keep-Alive
Server: ECS (phd/FD5F)
Vary: Accept-Encoding
X-Cache: 404-HIT
WARC/1.1
WARC-Page-ID: j6rektpcjxeg0dvgg1wcr5
WARC-Concurrent-To: <urn:uuid:1ff5eee1-22a3-4459-94eb-ffc2e12687cb>
WARC-Target-URI: http://example.com/favicon.ico
WARC-Date: 2020-11-13T04:09:08.355Z
WARC-Type: request
WARC-Record-ID: <urn:uuid:e7fffbb7-9c48-4c40-a831-5c566d72667f>
Content-Type: application/http; msgtype=request
WARC-Payload-Digest: sha-256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
WARC-Block-Digest: sha-256:bcc88e599c818995fdae9ff4f5fbfc4fbb4cc7e0df0f9c5f0e2d3aac343d19da
Content-Length: 395
HTTP/1.1 200 OK
Accept: image/avif,image/webp,image/apng,image/*,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Cache-Control: no-cache
Host: example.com
Pragma: no-cache
Proxy-Connection: keep-alive
Referer: http://example.com/
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36
WARC/1.1
WARC-Page-ID: m1zf4unmxraaqwhavlmuu
WARC-Target-URI: http://localhost:3000/
WARC-Date: 2020-11-13T04:08:59.267Z
WARC-Type: response
WARC-Record-ID: <urn:uuid:4334dc6d-01b5-4790-9760-fbeb3d18f4df>
Content-Type: application/http; msgtype=response
WARC-Payload-Digest: sha-256:7f83b1657ff1fc53b92dc18148a1d65dfc2d4b1fa3d677284addd200126d9069
WARC-Block-Digest: sha-256:e0fab66c20baa82e37ff12d65a165f8ead7fa0a0969256f30c8c9de658b8b07e
Content-Length: 216
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 12
Content-Type: text/html; charset=utf-8
Date: Fri, 13 Nov 2020 04:08:59 GMT
ETag: W/"c-Lve95gjOVATpfV8EL5X4nxwjKHE"
X-Powered-By: Express
Hello World!
WARC/1.1
WARC-Page-ID: m1zf4unmxraaqwhavlmuu
WARC-Concurrent-To: <urn:uuid:4334dc6d-01b5-4790-9760-fbeb3d18f4df>
WARC-Target-URI: http://localhost:3000/
WARC-Date: 2020-11-13T04:08:59.267Z
WARC-Type: request
WARC-Record-ID: <urn:uuid:24ed5ed6-3555-48f5-bf9a-17192b6b1257>
Content-Type: application/http; msgtype=request
WARC-Payload-Digest: sha-256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
WARC-Block-Digest: sha-256:9f0a0e522a0f8c6d526ff8ee1bf56f07874e71e387668301647e94cc1a55ebad
Content-Length: 595
HTTP/1.1 200 OK
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cache-Control: no-cache
Connection: keep-alive
Host: localhost:3000
Pragma: no-cache
Referer: http://localhost:3000/
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36
WARC/1.1
WARC-Page-ID: m1zf4unmxraaqwhavlmuu
WARC-Target-URI: http://localhost:3000/redirect
WARC-Date: 2020-11-13T04:09:07.988Z
WARC-Type: response
WARC-Record-ID: <urn:uuid:2ec1eb86-4caf-41b8-a656-059da1821888>
Content-Type: application/http; msgtype=response
WARC-Payload-Digest: sha-256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
WARC-Block-Digest: sha-256:814dfa54bf66eb4d5ed24f4a331204d90f975b9a1d80a0d873873a0b19622b40
Content-Length: 207
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 80
Content-Type: text/html; charset=utf-8
Date: Fri, 13 Nov 2020 04:09:07 GMT
Location: http://example.com
Vary: Accept
X-Powered-By: Express
WARC/1.1
WARC-Page-ID: m1zf4unmxraaqwhavlmuu
WARC-Concurrent-To: <urn:uuid:2ec1eb86-4caf-41b8-a656-059da1821888>
WARC-Target-URI: http://localhost:3000/redirect
WARC-Date: 2020-11-13T04:09:07.988Z
WARC-Type: request
WARC-Record-ID: <urn:uuid:29075a10-939f-4fc3-acec-ac9d2f3cfbc8>
Content-Type: application/http; msgtype=request
WARC-Payload-Digest: sha-256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
WARC-Block-Digest: sha-256:235ee0114246553d9ca2b59c10738627947f46fe2c76948e15340b77c7cfcd0c
Content-Length: 575
HTTP/1.1 200 OK
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cache-Control: no-cache
Connection: keep-alive
Host: localhost:3000
Pragma: no-cache
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36
WARC/1.1
WARC-Filename: my-web-archive.warc
WARC-Date: 2020-11-13T04:09:42.156Z
WARC-Type: warcinfo
WARC-Record-ID: <urn:uuid:6fee0ac0-d9f0-416a-92f2-2244095ed480>
Content-Type: application/warc-fields
Content-Length: 98
software: Webrecorder wabac.js/warcio.js
format: WARC File Format 1.1
isPartOf: My Web Archive
WARC/1.1
Content-Type: text/plain; charset="UTF-8"
WARC-Target-URI: urn:text:20201113040908238/http://example.com/
WARC-Date: 2020-11-13T04:09:08.238Z
WARC-Type: resource
WARC-Record-ID: <urn:uuid:e663e760-f70d-447d-a3e0-d55dd8e2488b>
WARC-Payload-Digest: sha-256:a60a4daa7945cc758859b55f24a6bafaea8110b4bcd9db94bc38824a38697d75
WARC-Block-Digest: sha-256:a60a4daa7945cc758859b55f24a6bafaea8110b4bcd9db94bc38824a38697d75
Content-Length: 191
Example Domain
This domain is for use in illustrative examples in documents. You may use this
domain in literature without prior coordination or asking for permission.
More information...
WARC/1.1
Content-Type: text/plain; charset="UTF-8"
WARC-Target-URI: urn:text:20201113040859267/http://localhost:3000/
WARC-Date: 2020-11-13T04:08:59.267Z
WARC-Type: resource
WARC-Record-ID: <urn:uuid:56ca5352-6903-4f0f-a430-cb060819a0c0>
WARC-Payload-Digest: sha-256:7f83b1657ff1fc53b92dc18148a1d65dfc2d4b1fa3d677284addd200126d9069
WARC-Block-Digest: sha-256:7f83b1657ff1fc53b92dc18148a1d65dfc2d4b1fa3d677284addd200126d9069
Content-Length: 12
Hello World!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment