Skip to content

Instantly share code, notes, and snippets.

@amitpatelx
Last active September 21, 2020 12:41
Show Gist options
  • Save amitpatelx/dd8ccc7eb9cd7c3a9500187bdbf1be23 to your computer and use it in GitHub Desktop.
Save amitpatelx/dd8ccc7eb9cd7c3a9500187bdbf1be23 to your computer and use it in GitHub Desktop.
Extract property details
# frozen_string_literal: true
class FranceSpider < ApplicationSpider
APARTMENT_TYPES = %w(lejlighed appartement apartment piso flat atico penthouse duplex t1 t2 t3 t4 t5 t6)
def apartment_types
APARTMENT_TYPES
end
HOUSE_TYPES = %w(hus chalet bungalow maison house home villa)
def house_types
HOUSE_TYPES
end
STUDIO_TYPES = ['studio']
def studio_types
STUDIO_TYPES
end
ROOM_TYPES = ['chambre']
def room_types
ROOM_TYPES
end
end
# You mostly won't require to modify this
def extract_property_type(details)
if studio_types.any? { |type| details.include?(type) }
return :studio
elsif apartment_types.any? { |type| details.include?(type) }
return :apartment
elsif house_types.any? { |type| details.include?(type) }
return :house
elsif room_types.any? { |type| details.include?(type) }
return :room
end
nil
end
class YourSpder < FranceSpider
...
....
....
item[:property_type] = extract_property_type("pass string description containing the property type detail")
# examples
item[:property_type] = extract_property_type(data['bien']['type_bien'])
item[:property_type] = extract_property_type(item[:title].downcase)
rooms_detail = elem.at_css('p.bedrooms').text
item[:property_type] = extract_property_type(rooms_detail)
....
....
end
# Note above files are for reference, you minght not need to change that file frequently.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment