Skip to content

Instantly share code, notes, and snippets.

Avatar
:octocat:

Dave Lester davelester

:octocat:
View GitHub Profile
@davelester
davelester / gist:2939847
Created Jun 16, 2012
Subset of Year 2001 Syllabus Finder urls (html, htm, shtml, and php extensions only)
View gist:2939847
http://faculty.oxy.edu/horowitz/courses/Hist221/hist221f2001.html
http://religion.rutgers.edu/courses/212/212_winter2001.html
http://www.music.utk.edu/keathley/MH220.2001.syllabus.html
http://www.stcloudstate.edu/~plamen/2001_2002/im260/spring02/syllabus.html
http://www.lib.iastate.edu/commons/hist307/2001spring/syllabus.html
http://www.math.grin.edu/~rebelsky/Courses/CS362/2001S/Handouts/instructions.html
http://www.math.grin.edu/~rebelsky/Courses/CS151/2001S/Handouts/instructions.html
http://www.unc.edu/courses/2001fall/anth/010/006/Syllabus1.html
http://cas-courses.buffalo.edu/classes/psy/segal/4212001/421Syl2001.htm
http://bss.sfsu.edu/tygiel/Hist427/427s2001online.html
@davelester
davelester / gist:2939839
Created Jun 16, 2012
Scraper hack. Code I am writing to (hopefully) scrape course syllabi. A mix of new code, and a previous project. Currently broken.
View gist:2939839
import urllib.robotparser
import urllib.request
import urllib.parse
from bs4 import BeautifulSoup
import time
import re
import string
from collections import deque
import os
import fileinput
@davelester
davelester / gist:2939780
Created Jun 16, 2012
breakdown of urls mention in syllabus finder corpus over the span of 9 specified years.
View gist:2939780
year | # syllabi
2001 - 13729
2002 - 26001
2003 - 34601
2004 - 41163
2005 - 39927
2006 - 39599
2007 - 42413
2008 - 42992
2009 - 18057
@davelester
davelester / gist:2939776
Created Jun 16, 2012
Snippet of PHP I wrote to generate list of urls for each year, separated by new line. called on command line, 'php filename > 2001urls.txt'
View gist:2939776
<?php
$mysqli = new mysqli('localhost', 'root', '', 'syllabi_scraper');
if ($mysqli->connect_error) {
die('Connect Error (' . $mysqli->connect_errno . ') '
. $mysqli->connect_error);
}
@davelester
davelester / gist:2939766
Created Jun 16, 2012
Year 2009 Syllabus Finder urls
View gist:2939766
This file has been truncated, but you can view the full file.
http://ksghome.harvard.edu/~.nmiller.Academic.KSG/API%20109i%20Syllabus%20F2001%20090101.doc
http://info-center.ccit.arizona.edu/~curric/Cb20/Cb2009/2009h.htm
http://www.personal.psu.edu/faculty/j/j/jjk10/SYLLABUS20099.htm
http://www.ivcc.edu/flm2009/Syllabus.html
http://www.mcc.cccoes.edu/ABM/ABM%20111%20Master%20Syllabus%2009-05-02.pdf
http://www.mcc.cccoes.edu/abm/ABM%20136%20Master%20Syllabus%2009-05-02.pdf
http://www.mcc.cccoes.edu/abm/ABM%20135%20Master%20Syllabus%2009-05-02.pdf
http://www.mcc.cccoes.edu/ABM/ABM%20112%20Master%20Syllabus%2009-05-02.pdf
http://www.tamuk.edu/edu/Files/Online%20Academy%2009-24-2001/PBS/a203/support/a203b0_10100.html
@davelester
davelester / gist:2939765
Created Jun 16, 2012
Year 2008 Syllabus Finder urls
View gist:2939765
This file has been truncated, but you can view the full file.
http://www.columbia.edu/~sam2008/HrSyl.html
http://www.columbia.edu/~sam2008/CC12001SYL.html
http://www.columbia.edu/~sam2008/IhSyl.html
http://www.columbia.edu/~sam2008/STS12001.html
http://www.columbia.edu/~sam2008/law.html
http://www.columbia.edu/~sam2008/IhSyl2.html
http://www.phys.tcu.edu/~dingram/phys20083/su02_syll.html
http://www.phys.tcu.edu/~dingram/phys20083/su02_syll_lab.html
http://www.zoo.ufl.edu/bouchard/BSC_2008/BSC_2008_Syllabus.htm
@davelester
davelester / gist:2939760
Created Jun 16, 2012
Year 2007 Syllabus Finder urls
View gist:2939760
This file has been truncated, but you can view the full file.
http://www.phys.tcu.edu/~dingram/phys20073/su02_syll.html
http://www.phys.tcu.edu/~dingram/phys20073/su02_syll_lab.html
http://web.gc.cuny.edu/ees/weather_climate/Fall%202002%20LAB%2007%20Syllabus%20for%20PGEOG130.pdf
http://www2.semo.edu/foreignlang/plan_del_cursosn100%2007.htm
http://www.uncp.edu/home/albrecht/pspa101_%2007f02.htm
http://www.cambridgecollege.edu/eforum/upload/PM%20II%20Syllabus%20072002.doc
http://www.eiu.edu/~multilit/english2007-f00/syllabus.htm
http://www.unc.edu/depts/comm/CourseSyllabi/Fall%202002/029%20Comm%20072.1%20Syllabus.doc
http://somc.csudh.edu/fa02_pub512/Week%2007%20Locke/Locke%20new.htm
@davelester
davelester / gist:2939757
Created Jun 16, 2012
Year 2006 Syllabus Finder urls
View gist:2939757
This file has been truncated, but you can view the full file.
http://web.mit.edu/firstyear/2006/rightnow/folder.html
http://classof2006.uchicago.edu/mailings/ies/course_descriptions.cfm
http://www.unc.edu/depts/comm/CourseSyllabi/Fall%202002/029%20Comm%20063.1%20Syllabus.doc
http://www.alvernia.edu/2006/schwangersyllabus.pdf
http://www.alvernia.edu/2006/css.htm
http://www.uncp.edu/home/albrecht/pspa101_%2006f02.htm
http://www4.nas.edu/cls/ijhome.nsf/44bf87db309563a0852566f2006d63bb/796c3396bb2d51e9852568dd005fe4e2?OpenDocument
http://parents2006.sacredheart.edu/FS101.html
http://pirate.shu.edu/~zinicode/2006/edst2006syllabus.html
@davelester
davelester / gist:2939753
Created Jun 16, 2012
Year 2005 Syllabus Finder urls
View gist:2939753
This file has been truncated, but you can view the full file.
http://www.columbia.edu/~clr2005/enl.html
http://www.columbia.edu/itc/english/f2005/course_syllabus.html
http://psol.gmu.edu/psol/degree2.nsf/3e51d7cb9addf238852569f40058c78a/f861c8fc026231dd85256a320055dd46?OpenDocument
http://psol.gmu.edu/psol/degree2.nsf/b63fef409e778e818525696c0073bd72/f861c8fc026231dd85256a320055dd46?OpenDocument
http://www.econ.vt.edu/~rdmurphy/ec2005/syllabus.pdf
http://icg.harvard.edu/~gov2005/syllabus/gtsyllabus2.pdf
http://faculty.darden.virginia.edu/colleyj/pdf/CS%20Syllabus%20Rev%2005%209-12-01.pdf
http://www.ans.latech.edu/homes/kennedy/Syllabus%20418%20051.doc
http://www.sbuniv.edu/~bkitchin.wh.bol/PHS2005/syllabus.htm
@davelester
davelester / Year 2004 Syllabus Finder urls
Created Jun 16, 2012
Year 2004 Syllabus Finder URLS
View Year 2004 Syllabus Finder urls
This file has been truncated, but you can view the full file.
http://sobek.colorado.edu/SOC/syllabi_files/SOCY2004-AR.htm
http://www.gmu.edu/depts/psychology/homepage/syllabi/syllfall01/372004.doc
http://studyat.anu.edu.au/DisplayCourse.asp?role=1&course=HIST2004
http://www.majbill.vt.edu/history/hirsh/hist2004/hist2004syl-f00.htm
http://reach.ucf.edu/~dep2004/syllabus.html
http://www.majbill.vt.edu/history/hirsh/hist2004/hist2004syl-f00.pdf
http://www.lrdc.pitt.edu/schunn/courses/0420/cogpsy%200421%20syllabus.pdf
http://www.lrdc.pitt.edu/schunn/courses/0420/cogpsy%200420%20syllabus.pdf
http://www.cvm.msstate.edu/Academics/2004/Radiology02-03.pdf