Skip to content

Instantly share code, notes, and snippets.

@Helw150
Last active April 25, 2017 18:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Helw150/a99f61191eabdedd5136012951b1cbc3 to your computer and use it in GitHub Desktop.
Save Helw150/a99f61191eabdedd5136012951b1cbc3 to your computer and use it in GitHub Desktop.
Scrapes all repos that have descriptions, aren't private and aren't forks. Places them in a JS list of Objects for import.
'''
This file is a helper function to scrape Github for Repo's to add to my website.
It can be set to run whenever Gatsby is called in order to add new Repo's as they arrive.
Usage: python RepoScraper.py SampleOutput.js
'''
from sys import argv
from github import Github
import re
def formatName(name):
name = re.sub("([a-z])([A-Z])","\g<1> \g<2>", name)
name = re.sub("([A-Z][A-Z])([a-z])","\g<1> \g<2>", name)
name = name.replace("_", " ")
return name
g = Github()
script, filename = argv
JS = open(filename, 'w')
JS.write('export default [\n')
for repo in g.get_user(login="Helw150").get_repos():
if not repo.fork and not repo.private and repo.description:
JS.write(' {\n')
JS.write(' title: "' + formatName(repo.name) + '",\n')
JS.write(' date: "' + repo.created_at.strftime("%m/%d/%Y") + '",\n')
JS.write(' url: "' + repo.html_url + '",\n')
JS.write(' description: "' + repo.description + '"\n')
JS.write(' },\n')
JS.write('];\n')
JS.close()
export default [
{
title: "CSO",
date: "09/23/2016",
url: "https://github.com/Helw150/CSO",
description: "Repo with CSO work"
},
{
title: "E.B.Held",
date: "12/24/2016",
url: "https://github.com/Helw150/E.B.Held",
description: "A Jquery website for my father: Author Edward Bruce Held"
},
{
title: "EDS control",
date: "09/22/2016",
url: "https://github.com/Helw150/EDScontrol",
description: "Methods to control NYU AD Engineering Design Studio using Amazon Alexa"
},
{
title: "Helw150.github.io",
date: "10/01/2016",
url: "https://github.com/Helw150/Helw150.github.io",
description: "A JQuery Personal website with brief descriptions of projects and links."
},
{
title: "IPA Incremental DAWG",
date: "10/15/2016",
url: "https://github.com/Helw150/IPA_Incremental_DAWG",
description: "Directed Acyclic Word Graph implementation in Go"
},
{
title: "Linear Equation Programs",
date: "04/14/2015",
url: "https://github.com/Helw150/LinearEquationPrograms",
description: "Senior Project work in Linear Equations"
},
{
title: "Shibboleth Login",
date: "11/30/2016",
url: "https://github.com/Helw150/ShibbolethLogin",
description: "An automatic clicker to log into the Shibboleth NYU Log in"
},
{
title: "String Alignment",
date: "10/31/2016",
url: "https://github.com/Helw150/StringAlignment",
description: "A library to align strings in Golang"
},
];
filteredProjectsList.map((project, i) => {
return (
<Col lg={4} md={4} sm={4} xs={12} key={i}>
<a href={project.url} target="_blank">
<Panel className="project-item">
<div className="project-title">
<h3>
{project.title}
</h3>
<span className={"project-description"}>{project.description}</span>
<h6 className={"project-date"}>{project.date}</h6>
</div>
</Panel>
</a>
</Col>
);
})
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment