Scrape opensea floor prices with python cloudscraper package

scrape_opensea_floor_prices.py

import cloudscraper
import json

def filter_typename(dict):
  return dict["__typename"] == "AssetQuantityType"

def filter_quantityInEth_exists(dict):
  if "quantityInEth" in dict:
    return True
  else:
    return False

def get_floor_price_in_eth(dict):
  return float(dict["quantity"]) / 1000000000000000000

def get_floor_prices(slug):
  scraper = cloudscraper.create_scraper(
    browser={
      'browser': 'chrome',
      'platform': 'android',
      'desktop': False
    }
  )
  url = "https://opensea.io/collection/{}?search[sortAscending]=true&search[sortBy]=PRICE&search[toggles][0]=BUY_NOW".format(slug);
  html = scraper.get(url).text
  json_string = html.split("</script>",2)[0].split("window.__wired__=",2)[1]
  data = json.loads(json_string)
  data_values = data["records"].values() # get all values type...
  data_list = [*data_values] # convert to list =~ array in js
  data_list = list(filter(filter_typename, data_list))
  data_list = list(filter(filter_quantityInEth_exists, data_list))
  data_list = list(map(get_floor_price_in_eth, data_list))
  return data_list


# scraping floor prices from opensea
print("RUNNING FOR cool-cats-nft")
print(get_floor_prices("cool-cats-nft"))
print("RUNNING FOR treeverse")
print(get_floor_prices("treeverse"))

This works locally, cloudflare is bypassed. Unfortunately when deployed to any cloud it stopps working. I tried:

• google cloud functions (with python3.9 runtime)
• google cloud run
• heroku with python container

I suspect cloudflare has all major cloud providers flagged and rejects any request coming from cloud IPs. Any help or ideas are appreciated.

another remark: for cloudscraper to work you also need the following dependencies installed:

requirements.txt

cloudscraper==1.2.58
Requests>=2.9.2
requests_toolbelt>=0.9.1

Hi dcts, is it possible to use your code to scrape specific trading data of an NFT, like last sales price?

You can use the official Opensea API to get last sales prices, only the floor prices are not in realtime thats why scraping them makes sense to get more precise data points.

To retrieve latest sales data check official Opensea API documentation.

Heres an example (it fetches the latest sales from treeverse):
https://api.opensea.io/api/v1/events?collection_slug=treeverse&event_type=successful&only_opensea=false&offset=0&limit=20

Hi, thanks for your kind answer! I know about the API, but I'm trying to get the data via scraping, because I found it to be more of an interesting challenge.

Oh ok, well, you for sure can get it through scraping, probably I would do the following:

• open the url with the activity tab open https://opensea.io/collection/sneaky-vampire-syndicate?tab=activity
• then scrape the table with the recent sales

Good luck with the challenge! 🚀💪

This works locally, cloudflare is bypassed. Unfortunately when deployed to any cloud it stopps working. I tried

take a look at https://rapidapi.com/restyler/api/scrapeninja it seems to work fine with opensea, at least with api.opensea graphql requests.