Skip to content

Instantly share code, notes, and snippets.

@amite
Created June 22, 2012 17:58
Show Gist options
  • Save amite/2974247 to your computer and use it in GitHub Desktop.
Save amite/2974247 to your computer and use it in GitHub Desktop.
Deduping a list of urls using PHP
<?php
// get all urls from the home page that are
$html = file_get_contents('http://www.catwalkyourself.com');
$dom = new DOMDocument();
@$dom->loadHTML($html);
// grab all the on the page
$xpath = new DOMXPath($dom);
$hrefs = $xpath->evaluate("/html/body//a");
for ($i = 0; $i < $hrefs->length; $i++) {
$href = $hrefs->item($i);
$url = $href->getAttribute('href');
if($url = preg_match_all('((www|http://)(www)?.catwalkyourself.com\/?.*)', $url, $matches[0])){
$urls = $matches[0][0][0];
$list = implode( ', ', array_unique( explode(", ", $urls) ) );
echo $list . '<br/>';
//print_r($list);
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment