Skip to content

Instantly share code, notes, and snippets.

@ChrisMcKee
Created October 18, 2011 16:26
Show Gist options
  • Save ChrisMcKee/1295870 to your computer and use it in GitHub Desktop.
Save ChrisMcKee/1295870 to your computer and use it in GitHub Desktop.
Scrape BBC Sports Mobile and Return Headers + Body in JSON
<?php
header('Cache-Control: no-cache, must-revalidate');
header('Expires: Mon, 26 Jul 1997 05:00:00 GMT');
header('Content-type: application/json');
$url = "http://news.bbc.co.uk/sport1/mobile/default.stm";
$options = array(
CURLOPT_RETURNTRANSFER => true, // return web page
CURLOPT_HEADER => true, // return headers
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_ENCODING => "", // handle all encodings
CURLOPT_USERAGENT => "spider", // who am i
CURLOPT_AUTOREFERER => true, // set referer on redirect
CURLOPT_CONNECTTIMEOUT => 120, // timeout on connect
CURLOPT_TIMEOUT => 120, // timeout on response
CURLOPT_MAXREDIRS => 10, // stop after 10 redirects
CURLOPT_SSL_VERIFYHOST => 0, // don't verify ssl
CURLOPT_SSL_VERIFYPEER => false, //
CURLOPT_VERBOSE => 1 //
);
$ch = curl_init($url);
curl_setopt_array($ch,$options);
$content = curl_exec($ch);
$err = curl_errno($ch);
$errmsg = curl_error($ch) ;
$header = curl_getinfo($ch);
curl_close($ch);
list($header, $body) = explode("\r\n\r\n", $content, 2);
$doc = json_encode(array("header" => $header, "body" => $body));
print_r($doc);
?>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment