Skip to content

Instantly share code, notes, and snippets.

@privatezero
Last active March 20, 2019 18:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save privatezero/b4a29744672850ec64a2c5dfc0130eee to your computer and use it in GitHub Desktop.
Save privatezero/b4a29744672850ec64a2c5dfc0130eee to your computer and use it in GitHub Desktop.
var="100"
curl 'http://research.wsulibs.wsu.edu:8080/oai/request?verb=ListRecords&metadataPrefix=oai_dc' | xmlstarlet format -s 2 > dspace-dump_000.xml
while [ $var -lt 13300 ] ; do
echo "Gathering records from number $var"
curl "http://research.wsulibs.wsu.edu:8080/oai/request?verb=ListRecords&resumptionToken=oai_dc////$var" | xmlstarlet format -s 2 >> "dspace-dump_$var.xml"
var=$((var + 100))
done
echo '<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="static/style.xsl"?>
<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
<responseDate>2019-03-20T16:45:57Z</responseDate>
<request verb="ListRecords" metadataPrefix="oai_dc">http://research.wsulibs.wsu.edu:8080/oai/request</request>
<ListRecords>' > ~/Desktop/dspace-dump_combined.xml
while read TARGET ; do cat "$TARGET" | tail -n +7 | head -n -3 >> ~/Desktop/dspace-dump_combined.xml ; done < <(ls -tr)
echo ' </ListRecords>
</OAI-PMH>' >> ~/Desktop/dspace-dump_combined.xml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment