Skip to content

Instantly share code, notes, and snippets.

@helb
Last active October 2, 2017 15:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save helb/be5263f7ce9d83e7dbe8c11277363814 to your computer and use it in GitHub Desktop.
Save helb/be5263f7ce9d83e7dbe8c11277363814 to your computer and use it in GitHub Desktop.
bs4 table parsing

https://stackoverflow.com/questions/46528003/how-to-clean-up-the-data-from-this-webscraping-script

for table in tables:
    rows = table.find_all("tr")
    for row in rows:
        cells = row.find_all("td")
        if len(cells) == 7:  # this filters out rows with 'Term', 'Instructor Name' etc.
            for cell in cells:
                print(cell.text + "\t", end="")  # \t is a Tab character, and end="" prevents a newline between cells
            print("")  # newline after each row
    print("-------------")  # table delimiter
>>> for table in tables:
...     rows = table.find_all("tr")
...     for row in rows:
...         cells = row.find_all("td")
...         if len(cells) == 7:
...             for cell in cells:
...                 print(cell.text + "\t", end="")
...             print("")
...     print("-------------")
... 
Question	No Response	Excellent	Very Good	  Good	   Fair	  Poor	
Description of course objectives and assignments	0.0%	63.9%	19.4%	11.1%	5.6%	0.0%	
Communication of ideas and information	0.0%	55.6%	22.2%	13.9%	5.6%	2.8%	
Expression of expectations for performance in this class	0.0%	61.1%	25.0%	11.1%	2.8%	0.0%	
Availability to assist students in or out of class	5.6%	58.3%	16.7%	16.7%	2.8%	0.0%	
Respect and concern for students	2.8%	58.3%	13.9%	16.7%	5.6%	2.8%	
Stimulation of interest in course	2.8%	58.3%	11.1%	13.9%	5.6%	8.3%	
Facilitation of learning	0.0%	55.6%	22.2%	11.1%	11.1%	0.0%	
Overall assessment of instructor	0.0%	61.1%	11.1%	22.2%	5.6%	0.0%	
-------------
Question	No Response	Excellent	Very Good	  Good	   Fair	  Poor	
Description of course objectives and assignments	0.0%	54.5%	0.0%	27.3%	9.1%	9.1%	
Communication of ideas and information	9.1%	36.4%	9.1%	18.2%	18.2%	9.1%	
Expression of expectations for performance in this class	0.0%	36.4%	9.1%	54.5%	0.0%	0.0%	
Availability to assist students in or out of class	0.0%	36.4%	27.3%	36.4%	0.0%	0.0%	
Respect and concern for students	9.1%	45.5%	36.4%	9.1%	0.0%	0.0%	
Stimulation of interest in course	9.1%	27.3%	9.1%	27.3%	9.1%	18.2%	
Facilitation of learning	9.1%	36.4%	9.1%	18.2%	18.2%	9.1%	
Overall assessment of instructor	0.0%	27.3%	27.3%	18.2%	27.3%	0.0%
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment