Skip to content

Instantly share code, notes, and snippets.

@guitarrapc
Last active May 28, 2018 00:49
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save guitarrapc/2df4a50d3608a328eda739656fe75ba4 to your computer and use it in GitHub Desktop.
PowerShellっぽく陸上自衛隊のイラク派遣日報をまとめてダウンロードしてみるのC#版
using System;
using System.IO;
using System.Linq;
using System.Net.Http;
using System.Threading.Tasks;
using AngleSharp.Dom.Html;
using AngleSharp.Parser.Html;
async Task Main()
{
var basePath = @"C:\Users\username\Downloads\test2";
await DownloadIraqReportAsync(basePath);
}
async Task DownloadIraqReportAsync(string directory)
{
var doc = default(IHtmlDocument);
var client = new HttpClient() { Timeout = TimeSpan.FromMinutes(5) };
using (var stream = await client.GetStreamAsync("https://www.asahi.com/articles/ASL4J669JL4JUEHF016.html"))
{
var parser = new HtmlParser();
doc = await parser.ParseAsync(stream);
}
await Task.WhenAll(doc.Links.OfType<IHtmlAnchorElement>()
.Where(x => x.PathName.EndsWith(".pdf"))
.GroupBy(x => x.Href, x => x)
.Select(x => new Uri(x.Key))
.Select(async x =>
{
var file = Path.Combine(directory, x.Segments.Last());
using (var res = await client.GetAsync(x, HttpCompletionOption.ResponseHeadersRead))
using (var fileStream = new FileStream(file, FileMode.OpenOrCreate))
{
var s = await res.Content.ReadAsStreamAsync();
await s.CopyToAsync(fileStream);
}
}));
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment