Skip to content

Instantly share code, notes, and snippets.

@citizen428
Created July 4, 2019 14:04
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save citizen428/2631a265bc6d08db2d6b326f5ecd87b2 to your computer and use it in GitHub Desktop.
Save citizen428/2631a265bc6d08db2d6b326f5ecd87b2 to your computer and use it in GitHub Desktop.
A simple dev.to scraper
#r "fsharp-data/lib/net45/FSharp.Data.dll"
open FSharp.Data
let doc = HtmlDocument.Load("https://dev.to/t/fsharp/top/infinity")
let posts = doc.CssSelect(".single-article")
let firstChildText selector (post : HtmlNode) =
post.CssSelect(selector).[0].DirectInnerText().Trim()
let cleanName (name : string) = name.Replace("・", "")
let top3 =
posts
|> Seq.map(fun post ->
post |> firstChildText ".content h3",
post |> firstChildText "h4 > a" |> cleanName,
post |> firstChildText ".reactions-count .engagement-count-number")
|> Seq.take 3
for (title, author, reactions) in top3 do
printf "\"%s\" (%s): %s\n" title author reactions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment