Skip to content

Instantly share code, notes, and snippets.

@atomotic
Last active October 19, 2017 04:29
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save atomotic/c6d6c9f59bdd4fa22d166690a4cc5dc6 to your computer and use it in GitHub Desktop.
Save atomotic/c6d6c9f59bdd4fa22d166690a4cc5dc6 to your computer and use it in GitHub Desktop.
poor man's WARC viewer

https://asciinema.org/a/FqmauknkDWf8eIXHbyd77aJct


install rat https://github.com/ericfreese/rat

go get github.com/ericfreese/rat

install warcio and warctools

pip3 install warcio
pip3 install warctools

edit ~/.config/rat/ratrc

mode warc.index
  annotate match warc 	-- ls *.warc.gz
  bindkey enter warc warc.payload -- > warcio index %(warc)
end

mode warc.payload
  annotate match offset 	--  warcio index %(warc) | jq -r .offset
  bindkey enter offset preview -- > warcpayload %(warc):%(offset)
end

run

rat --mode warc.index --cmd "find . -name '*.warc.gz'"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment