Skip to content

Instantly share code, notes, and snippets.

@peterc
Created January 17, 2017 22:43
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save peterc/13858a03d631e7f9046bf1ed40ade113 to your computer and use it in GitHub Desktop.
Save peterc/13858a03d631e7f9046bf1ed40ade113 to your computer and use it in GitHub Desktop.
Downloads Vines associated with a certain account
# vinescrape.rb goes through an account's Vines and downloads them, skipping revines
# I had to write it because I didn't get notified of the download deadline and
# with no email address on their site, it wouldn't let me get my archive..
# so I came up with this approach instead :-D
#
# To use, just change the VINE_USER_ID string below to the long set of digits that makes up YOUR Vine ID
# Then ruby vinescrape.rb
# Works fine with no dependencies on Ruby 2.4 on OS X, should work fine elsewhere too
VINE_USER_ID = "970482265149054976"
USER_AGENT = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/#{41 + rand(10)}.0.2228.0 Safari/537.36"
require 'open-uri'
require 'json'
# Given a URL, we go GET it
def fetch_data(url)
open(url, "User-Agent" => USER_AGENT).read
end
page = 1
# First, we need to fetch the first 'page' of results
url = "https://vine.co/api/timelines/users/#{VINE_USER_ID}?page=#{page}"
data = JSON.parse(fetch_data(url))
total_vines = data['data']['count'].to_i
STDERR.puts "#{total_vines} vines to fetch"
sleep 1
i = 0
downloaded = 0
loop do
url = "https://vine.co/api/timelines/users/#{VINE_USER_ID}?page=#{page}"
STDERR.puts "Fetching page #{page} of vines"
data = JSON.parse(fetch_data(url))
vines = data['data']['records']
vines.each do |vine|
i += 1
STDERR.puts "Processing #{i}/#{total_vines}"
# Skip "revines"
unless vine['userId'].to_s == VINE_USER_ID
STDERR.puts " Revine - not downloading"
next
end
# Get the URL of the MP4 video
url = vine['videoDashUrl']
# Download and save the video locally
STDERR.puts " Downloading #{url}"
begin
File.open("#{vine['postId']}.mp4", "w") { |f| f.puts fetch_data(url) }
STDERR.puts " Downloaded!"
downloaded += 1
# Sleep a bit to not look too heavy on the server
sleep rand(3) + 1
rescue
STDERR.puts " Download FAILED"
end
end
break if i >= total_vines
page += 1
sleep rand(2) + 1
end
STDERR.puts "#{i} vines processed, #{downloaded} vines downloaded"
@peterc
Copy link
Author

peterc commented Jan 17, 2017

And yes the code is a mess. I looked at how Vine's private API worked and threw it together in 20 minutes before they stopped me being able to download my Vines. If it breaks for you or if Vine blocks it, sorry!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment