Skip to content

Instantly share code, notes, and snippets.

@Goodnessuc
Last active May 24, 2022 06:40
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Goodnessuc/e98f4c656b361a179050cd9e5390970a to your computer and use it in GitHub Desktop.
Save Goodnessuc/e98f4c656b361a179050cd9e5390970a to your computer and use it in GitHub Desktop.
scraping hackernews using the Goquery package
package main
import (
"fmt"
"github.com/PuerkitoBio/goquery"
"net/http"
)
type Information struct {
link string
title string
}
func main() {
info := make([]Information, 0)
webUrl := "https://news.ycombinator.com/"
response, _ := http.Get(webUrl)
// get status code (article should contain this)
document, _ := goquery.NewDocumentFromReader(response.Body)
document.Find("tr.athing").Each(func(index int, selector *goquery.Selection) {
title := selector.Find("td.title").Text()
getLink, _ := selector.Find("a.titlelink").Attr("href")
info = append(info, Information{
title: title,
link: getLink,
})
})
fmt.Println(info)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment