Yahoo Pipes is closing, downloading http://pipes.yahoo.com/pipes/pipe.info?_out=json&_id= via browser for hundreds of pipes infeasible. There is https://gist.github.com/psychemedia/2351614 but it only works for public pipes and you don't want to make all your pipes public.
- Login to Yahoo Pipes, open debug console (F12 in Chrome/FF), click the "json feed" button to get to a page with this url scheme:
http://pipes.yahoo.com/pipes/person.info?_out=json&display=pipes&guid= - In the debug console get the cURL command for the current request (Net/Network tab, copy as cURL)
- Execute the command which we will create in the following to download all descriptions of your pipes including their ID
- Prepend
for page in ``seq 1 10``; doto (2.), where you replace 10 by the number of pages you have in your pipes overview - After
_out=json&display=pipesin (2.) insert&page=$page - Append
-o pipes_$page.json; doneto (2.)
- Now that you have files including all IDs of your pipes you will extract all IDs into a single file
- Install http://stedolan.github.io/jq/ ("jq is like sed for JSON data")
- Execute
jq .value.items[].id pipes*.json > pipeids.txt
- Next you will curl all your pipe definitions, start with the plain cURL command from (2.) again and then execute the command we create in the next three substeps
- Prepend
while read -r line; do curl "http://pipes.yahoo.com/pipes/pipe.info?_out=json&_id=$line" - Remove the old curl url (e.g.,
curl "http://pipes.yahoo.com/pipes/person.info?_out=json&display=pipes&guid=...") - Append
-o pipe_$line.json; done < pipeids.txt
- Now you have all json definitions of all your pipes and you can continue converting them to python: https://github.com/ggaughan/pipe2py or to node.js: https://github.com/neyric/pipes2js
Thank you for detailed description! It saved me time figuring all these details myself (to store my 385 pipes).
The only one thing that I encounter is that
jqoutput strings with"character, that should be stripped from thepipeids.txtlines.