Skip to content

Instantly share code, notes, and snippets.

@skpyns
Last active August 14, 2018 11:28
Show Gist options
  • Save skpyns/d3193514f74ef3c589dc7984b06ed4bd to your computer and use it in GitHub Desktop.
Save skpyns/d3193514f74ef3c589dc7984b06ed4bd to your computer and use it in GitHub Desktop.
Fetching a number of ads of used bicycles
import requests, bs4
# Making a request
r = requests.get('https://www.2bike.rs/cikloberza/mali-oglasi/delovi-20/ramovi-i-delovi-za-ram-181/ramovi-21?fl20[]=138&fl12[]=72')
if r.status_code != 200:
raise Exception("Error!")
# Making a soup and locating an object to be retrieved
soup = bs4.BeautifulSoup(r.text, 'lxml')
ads = soup.find('div', {'class': 'total'})
print('Number of ads: ' + ads.text)
@ChadRoberts21
Copy link

As you requested feed back on Reddit here is some.

You can be more specific in the error message you print like:
raise Exception(f'Error! The status code {r.status_code} was returned from the server')

Also when web scrapping any site you need to remember that stuff can change and any point and won't be consistent. So with that in mind I would change this line ads = soup.find('div', {'class': 'total'}) to ads = soup.find_all('div', {'class': 'total'}) as there are two of them on the page. And you never know they might be different values in each. You can then decide what you want to do with the "ads" list. You can print all of them, merge them, just print the frist one, etc.

By the work Good Work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment