Skip to content

Instantly share code, notes, and snippets.

@fijiaaron
Created December 21, 2011 23:51
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save fijiaaron/1508261 to your computer and use it in GitHub Desktop.
Check a list of URLs for content using PHP & CURL
http://one-shore.com/
http://google.com/
http://yahoo.com/
@oitconz
Copy link

oitconz commented May 20, 2022

Hi. Using this on Windows 10, Xampp, PHP 7.4 I had no end of trouble with fopen($url)

I traded out check_link_fopen(url) for

function get_http_response_code($url) {
$headers = get_headers($url);
return substr($headers[0], 9, 3);
}

then compared the result
$is_valid_url = get_http_response_code($url);
if ( 200 == $is_valid_url ){
print " valid $url\n\r";
$valid_links[] = $url;
} else {
print " invalid $url [$is_valid_url]\n\r";
$invalid_links[] = $url;
}

It got rid of issues with php and fopen against the URL.

I also had issues with invalid URL when I got past this. It was because of Windows and newlines.
So I stripped newlines and carriage returns out of the URL before running the check.
$url = str_replace(array("\n", "\r"), '', $url);

Its why the print statements above have /r/n at the end ( to replace those for the print out).

HTH

The final code is below:
`<?php
// usage: cat urls.txt | php checklinks.php --use-curl

if ( ini_get( 'allow_url_fopen' ) ) {
echo "allow_url_fopen is ENABLED.\n";
} else {
echo "allow_url_fopen is DISABLED.\n";
exit (1);
}

// initialize results arrays
$valid_links = array();
$invalid_links = array();

// read a list of files from stdin
$stdin = fopen('url.txt', "r");

// check each one
while ($url = @fgets($stdin))
{
$url = str_replace(array("\n", "\r"), '', $url);
$is_valid_url = get_http_response_code($url);

if ($is_valid_url == "200")
{
	print " valid $url\n\r";
	$valid_links[] = $url;
}
else 
{
	print " invalid $url [$is_valid_url]\n\r";
	$invalid_links[] = $url;
}

}

// report totals
print "valid links: " . count($valid_links) . "\n";
print "invalid links: " . count($invalid_links) . "\n";

function get_http_response_code($url) {
$headers = get_headers($url);
return substr($headers[0], 9, 3);
}

?>`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment