Create a gist now

Instantly share code, notes, and snippets.

@scitasy /main.java
Last active Dec 16, 2015

What would you like to do?
Using Selenium to SCRAPE DAT WEB.
public class Main
{
public static void main(String[] args)
{
//Create the Selenium WebDriver object
WebDriver driver = new FirefoxDriver();
//Navigate to the subway locations page
driver.get("http://wiki.openstreetmap.org/wiki/List_of_London_Underground_stations");
//Find and store the table element
WebElement table = driver.findElement(By.className("wikitable"));
//Store all the rows in the table
List<WebElement> list = table.findElements(By.xpath("/html/body/div[3]/div[2]/div[4]/table[2]/tbody/tr"));
//Go through each row in the table
for(int x = 0; x<list.size(); x++)
{
//Store the columns of each row in a new list
List<WebElement> newList = list.get(x).findElements(By.xpath("td"));
//Print out the data of every column in the current row
for(int y = 0; y<newList.size(); y++)
{
System.out.println(y + " : " + newList.get(y).getText());
}
}
//Hit it and quit it. I'M NOT A DOUCHE.
driver.quit();
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment