Skip to content

Instantly share code, notes, and snippets.

View HunnyJummani's full-sized avatar

HunnyJummani-Tntra HunnyJummani

View GitHub Profile
@HunnyJummani
HunnyJummani / static_web_scraper.rb
Created December 23, 2019 13:40
Static Web Scraping- Kimurai ( Ruby on Rails )
class WebScrapper < Kimurai::Base
@name = "web_scrapper_spider"
@engine = :mechanize
@start_urls = ["https://metaruby.com/"]
@config = {
user_agent: "Chrome/68.0.3440.84"
}
def parse(response, url:, data: { })
blogs = []
@HunnyJummani
HunnyJummani / infinite_scrolling_web_scraper.rb
Created December 23, 2019 13:37
Infinite scrolling web scraping in Kimurai - Ruby on Rails
require 'kimurai'
class WebScrapper < Kimurai::Base
@name = "web_scrapper_spider"
@engine = :selenium_chrome
@start_urls = ["https://metaruby.com/"]
@config = {
user_agent: "Chrome/68.0.3440.84"
}