Each of these commands will run an ad hoc http static server in your current (or specified) directory, available at http://localhost:8000. Use this power wisely.
$ python -m SimpleHTTPServer 8000
.-(ed@curry 05:24:32) ~ | |
`-->wget --recursive --warc-file=c4lj.warc.gz http://journal.code4lib.org | |
FINISHED --2011-12-02 05:17:11-- | |
Total wall clock time: 19m 24s | |
Downloaded: 1524 files, 99M in 4m 17s (395 KB/s) | |
.-(ed@curry 05:45:22) ~ | |
`-->ls -lh c4lj.warc.gz.warc.gz | |
-rw-rw-r-- 1 ed ed 85M 2011-12-02 05:17 c4lj.warc.gz.warc.gz |
Each of these commands will run an ad hoc http static server in your current (or specified) directory, available at http://localhost:8000. Use this power wisely.
$ python -m SimpleHTTPServer 8000
Do you have a WARC file of a website all downloaded and ready to be added to the Internet Archive? Great! You can do that with the Internet Archive's web-based uploader, but it's not ideal and it can't handle really big uploads. Here's how you can upload your WARC files to the IA from the command line, and without worrying about a size restriction.
First, you need to get your Access Key and Secret Key from the Internet Archive for the S3-like API. Here's where you can get that for your IA account: http://archive.org/account/s3.php Don't share those with other people!
Here's their documentation file about how to use it, if you need some extra help: http://archive.org/help/abouts3.txt
Next, you should copy the following files to a text file and edit them as needed:
export IA_S3_ACCESS_KEY="YOUR-ACCESS-KEY-FROM-THE-IA-GOES-HERE"
NOTE: This is a question I found on StackOverflow which I’ve archived here, because the answer is so effing phenomenal.
If you are not into long explanations, see [Paolo Bergantino’s answer][2].
openssl dgst -sha1 -binary $1 | python -c "import base64,sys; print base64.b32encode(sys.stdin.read())" |
IPLD-based Version History
This is just a sketch of a possibility. If we just want a git-style toolchain with git version graph, it might be better to just put an ipfs storage adapter behind go-git -- basically putting IPFS unixfs trees where git usually creates git tree objects. In that case you would have regular git commit objects, not IPLD objects. That would be less reusable beyond the git context but it would fit better with existing git-based tooling.
Keep in mind: it will be really useful to be able to use IPFS files api to manipulate these trees, allowing you to do things like modify part of a large dataset and commit the changes without ever pulling the whole dataset -- you can just pull the parts that you're changing.
https://asciinema.org/a/FqmauknkDWf8eIXHbyd77aJct
install rat https://github.com/ericfreese/rat
go get github.com/ericfreese/rat
install warcio and warctools
install solr and create a core (books
)
brew install solr
solr start
solr create -c books -d /usr/local/Cellar/solr/7.2.1/example/files/conf
index a pdf
post -c books /tmp/gabriella-giannachi-archive-everything-mapping-the-everyday.pdf
The subfile package is a much more efficient way to bring in the sub parts of a large latex document.
Each part of the complete document that is split up into its own text file brought is added to the document from the main tex file via
\subfile{file}
Each subfile has the following structure
\documentclass[main.tex]{subfiles}
#!/usr/bin/env bash | |
#### | |
# Original version of this file was created by Dr. Chuck Cartledge | |
# as a make file and can be found in his WS-DL blog post: | |
# https://ws-dl.blogspot.com/2014/07/2014-0-7-2-ode-to-margin-police-or-how.html | |
#### | |
#### Set these variables to match your setup | |
CHECK="msThesis" |