Skip to content

Instantly share code, notes, and snippets.

@complexsplit
Last active August 29, 2015 14:07
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save complexsplit/326d2034ac840077b750 to your computer and use it in GitHub Desktop.
Download VMworld 2014 Content
# replace names and emails with your own.
require 'watir-webdriver'
require 'phantomjs'
require 'open-uri'
require 'progressbar'
b = Watir::Browser.new :phantomjs
b.goto 'http://www.vmworld.com/community/sessions/2014'
# possibly repalce url w/ http://vmware.mediasite.com/mediasite/Catalog/Full/daea26ff56894512a04687ead2afc51821 to escape frames
b.text_field(:name => 'input_firstName').set 'Frank'
b.text_field(:name => 'input_lastName').set 'Castle'
b.text_field(:name => 'input_email').set 'frank.castle@example.com'
$humantest = b.forms[1].text
$captcharesult = eval($humantest.split('What is')[1].chop.strip)
b.text_field(:id => 'BotBootInput').set $captcharesult
b.button(:name => 'submit').click
# b.alert.ok # acknowledge javascript alert - only needed with !phantomjs
b.span(:id => 'loginFormControl_Username').wait_until_present
sleep 3
loop do
presentationtable = b.table(id: 'VideoPreviewCardDataList')
presentationentries = presentationtable.tds(id: 'tdPresentationDetails')
presentationentries.each do |row|
if row.link(:text, "Download").exists?
row.link(:text, "Download").click
end
b.div(:class => 'downloadPresentationLink').wait_until_present
contenturl = b.div(:class => 'downloadPresentationLink').link.href
basefilename = File.basename(contenturl).split('?')[1].split('&')[0].split('=')[1].tr("+", " ")
progressbar = nil
File.open(basefilename, "wb") do |localfile|
localfile.write open(contenturl, :content_length_proc => lambda {|total|
if total && 0 < total
progressbar = ProgressBar.new("...", total)
progressbar.file_transfer_mode
end
},
:progress_proc => lambda {|step|
progressbar.set step if progressbar
}).read
end
b.button(:text => 'Close').click
end
b.window.resize_to(1024,768)
break if b.elements(:id => 'NextButtonImage', :class => 'ButtonDisabled').size == 1
b.span(id: 'NextButtonImage', class: 'ButtonEnabled').click
sleep 3
end

Notes

Uses Watir to control a web browser to download all VMworld 2014 content.

The team at VMware has contracted with a third party to provide VMworld content. As such, the content portal now uses Mediasite Catalog by Sonic Foundry. This catalog software utilizes a lot of Javascript to build the page and navigate.

It does not appear this change was not intended to dissuade downloading of materials. https://twitter.com/VMware/status/515638081488056320

To use the script just change your first and last names and email address on the first few lines.

Requires the 'watir-webdriver', 'phantoms', and 'progress bar' gems.

Todo:

  • Threaded downloads
  • Replace sleeps with content or page load detection
@rnelson0
Copy link

rnelson0 commented Oct 7, 2014

Using Windows 7 / Firefox 32.0.3 / Ruby 1.9.3:

It works great, till you do anything else with your PC. The moment you do, the Ruby process appears to halt, never resuming, and you have to start from the top again. Since it's the work day, I can't do much else for testing at the moment. I can try leaving it going overnight and see how well that works.

@complexsplit
Copy link
Author

Hmm. Will look into it. I developed against OSX 10.9.5 / Firefox 32.0.3 / Ruby 2.0.0.

Frankly, I'm surprised it works at all! :) It could be something with application focus? If so, we could make b.visible = false.

@rnelson0
Copy link

rnelson0 commented Oct 7, 2014

Would that go between line 6 and 7 or thereabouts?

@complexsplit
Copy link
Author

I've replaced Firefox with PhantomJS so things now run 'headless'. No more focus issues!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment