Skip to content

Instantly share code, notes, and snippets.

@SteveBate
Last active August 29, 2015 14:18
Show Gist options
  • Save SteveBate/f859b36d538050b4362b to your computer and use it in GitHub Desktop.
Save SteveBate/f859b36d538050b4362b to your computer and use it in GitHub Desktop.
Example of scraping urls asnchronously using async, await and tasks
async System.Threading.Tasks.Task Main()
{
await Get("http://bbc.co.uk").ContinueWith(async r => {
using(var resp = r.Result.GetResponseStream())
{
var content = new MemoryStream();
await resp.CopyToAsync(content);
var html = System.Text.Encoding.Default.GetString(content.ToArray());
html.Dump(); // linqpad
}
});
await Get("http://amazon.co.uk").ContinueWith(async r => {
using(var resp = r.Result.GetResponseStream())
{
var content = new MemoryStream();
await resp.CopyToAsync(content);
var html = System.Text.Encoding.Default.GetString(content.ToArray());
html.Dump(); // linqpad
}
});
}
public async System.Threading.Tasks.Task<System.Net.WebResponse> Get(string url)
{
var req = System.Net.WebRequest.Create(url);
req.Method = "GET";
var resp = await req.GetResponseAsync();
return resp;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment