Skip to content

Instantly share code, notes, and snippets.

@Swimburger
Last active November 4, 2020 04:17
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Swimburger/21d69c3ebb29a09178664ea9fdd4f681 to your computer and use it in GitHub Desktop.
Save Swimburger/21d69c3ebb29a09178664ea9fdd4f681 to your computer and use it in GitHub Desktop.
Function CrawlSitemap
{
Param(
[parameter(Mandatory=$true)]
[string] $SiteMapUrl
);
$SiteMapXml = Invoke-WebRequest -Uri $SiteMapUrl -UseBasicParsing -TimeoutSec 180;
$Urls = ([xml]$SiteMapXml).urlset.ChildNodes
ForEach ($Url in $Urls){
$Loc = $Url.loc;
try{
$result = Invoke-WebRequest -Uri $Loc -UseBasicParsing -TimeoutSec 180;
Write-Host $result.StatusCode - $Loc;
}catch [System.Net.WebException] {
Write-Warning (([int]$_.Exception.Response.StatusCode).ToString() + " - " + $Loc);
}
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment