Skip to content

Instantly share code, notes, and snippets.

@CXuesong
Last active February 11, 2020 16:58
Show Gist options
  • Save CXuesong/96336f4e6733ccc8c1a8826da0a9b0f9 to your computer and use it in GitHub Desktop.
Save CXuesong/96336f4e6733ccc8c1a8826da0a9b0f9 to your computer and use it in GitHub Desktop.
Make a list of 2 level category items on MediaWiki site really fast
uti id title platforms packages
com.xamarin.workbook
57dd1e8a-dfa3-4ab8-9bcb-74aee12e18bf
Untitled
DotNetCore
id version
CXuesong.MW.WikiClientLibrary
0.7.0-int.9
#r "WikiClientLibrary"
using WikiClientLibrary.Client;
using WikiClientLibrary.Sites;
using WikiClientLibrary.Generators;
using System.IO;
var client = new WikiClient();
var site = new WikiSite(client, await WikiSite.SearchApiEndpointAsync(client, "mlp.fandom.com"));
await site.Initialization;
var gen0 = new CategoryMembersGenerator(site, "Characters") { PaginationSize = 400, MemberTypes = CategoryMemberTypes.Subcategory }
var gen1 = new CategoryMembersGenerator(site, gen0.CategoryTitle) { PaginationSize = 400, MemberTypes = CategoryMemberTypes.Page }
var items = await gen0.EnumItemsAsync()
        .SelectMany(c => new CategoryMembersGenerator(site, c.Title) { PaginationSize = 400, MemberTypes = CategoryMemberTypes.Page }
        .EnumItemsAsync()
        .Concat(gen1.EnumItemsAsync())
        .Select(s => s.Title)).Distinct().ToList();
File.WriteAllLines("D:/mlp-en.txt", items);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment