Skip to content

Instantly share code, notes, and snippets.

@miharekar
Last active Mar 20, 2022
Embed
What would you like to do?
Stoic quotes from Goodreads
require:
- rubocop-performance
AllCops:
NewCops: enable
Layout/SpaceInsideHashLiteralBraces:
Enabled: true
EnforcedStyle: no_space
Layout/MultilineMethodCallIndentation:
Enabled: true
EnforcedStyle: indented
Layout/LineLength:
Enabled: false
Layout/DotPosition:
Enabled: true
EnforcedStyle: trailing
Style/RaiseArgs:
Enabled: false
Style/NumericLiterals:
Enabled: false
Style/Send:
Enabled: true
Metrics/ClassLength:
Enabled: false
Metrics/MethodLength:
Enabled: false
Metrics/BlockLength:
Enabled: false
Metrics/CyclomaticComplexity:
Enabled: false
Metrics/PerceivedComplexity:
Enabled: false
Metrics/AbcSize:
Enabled: false
Style/Documentation:
Enabled: false
Style/AndOr:
Enabled: true
EnforcedStyle: always
Style/StringLiterals:
Enabled: true
EnforcedStyle: double_quotes
# frozen_string_literal: true
source "https://rubygems.org"
ruby "3.1.1"
gem "concurrent-ruby"
gem "nokogiri"
gem "pry"
gem "rubocop"
gem "rubocop-performance"
GEM
remote: https://rubygems.org/
specs:
ast (2.4.2)
coderay (1.1.3)
concurrent-ruby (1.1.9)
method_source (1.0.0)
mini_portile2 (2.8.0)
nokogiri (1.13.3)
mini_portile2 (~> 2.8.0)
racc (~> 1.4)
parallel (1.21.0)
parser (3.1.1.0)
ast (~> 2.4.1)
pry (0.14.1)
coderay (~> 1.1)
method_source (~> 1.0)
racc (1.6.0)
rainbow (3.1.1)
regexp_parser (2.2.1)
rexml (3.2.5)
rubocop (1.26.0)
parallel (~> 1.10)
parser (>= 3.1.0.0)
rainbow (>= 2.2.2, < 4.0)
regexp_parser (>= 1.8, < 3.0)
rexml
rubocop-ast (>= 1.16.0, < 2.0)
ruby-progressbar (~> 1.7)
unicode-display_width (>= 1.4.0, < 3.0)
rubocop-ast (1.16.0)
parser (>= 3.1.1.0)
rubocop-performance (1.13.3)
rubocop (>= 1.7.0, < 2.0)
rubocop-ast (>= 0.4.0)
ruby-progressbar (1.11.0)
unicode-display_width (2.1.0)
PLATFORMS
ruby
DEPENDENCIES
concurrent-ruby
nokogiri
pry
rubocop
rubocop-performance
RUBY VERSION
ruby 3.1.1p18
BUNDLED WITH
2.3.4
# frozen_string_literal: true
require "pry"
require "rubygems"
require "bundler"
Bundler.require(:default)
require "concurrent"
require "open-uri"
require "json"
class QuoteDownloader
attr_reader :author
BASE_URL = "https://www.goodreads.com/"
def initialize(author)
@quotes = Concurrent::Array.new
@author = author
end
def quotes
download_quotes if @quotes.empty?
@quotes
end
private
def download_quotes
load_quotes_from("#{BASE_URL}#{author}?page=1")
1.upto(number_of_pages).map do |page|
Concurrent::Future.execute { load_quotes_from("#{BASE_URL}#{author}?page=#{page}") }
end.map(&:value)
end
def load_quotes_from(url)
puts url
Nokogiri::XML(URI.open(url)).css(".quoteText").each do |quote_text|
text = quote_text.children.select { |c| c.is_a? Nokogiri::XML::Text }.map(&:text).join(" ").strip
author_or_title = quote_text.at_css(".authorOrTitle")
author_name = if author_or_title
author_or_title.text.strip.sub(/,$/, "")
else
current_author_name
end
@quotes << {text:, author: author_name}
end
end
def number_of_pages
main_doc.css("a").select { |a| a[:href].to_s.include?("page=") }.map { |a| a["href"][/page=(\d+)/, 1].to_i }.max || 1
end
def current_author_name
@current_author_name ||= main_doc.css("h1 a").children.last.text
end
def main_doc
@main_doc ||= Nokogiri::XML(URI.open("#{BASE_URL}#{author}"))
end
end
all_quotes = Concurrent::Array.new
["author/quotes/13852.Epictetus", "author/quotes/4918776.Seneca", "author/quotes/17212.Marcus_Aurelius", "author/quotes/833825.Zeno_of_Citium", "quotes/tag/stoicism"].map do |author|
Concurrent::Future.execute { all_quotes << QuoteDownloader.new(author).quotes }
end.map(&:value)
File.write("quotes.json", {quotes: all_quotes.flatten}.to_json)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment