Skip to content

Instantly share code, notes, and snippets.

@cn007b
Created May 3, 2019 09:14
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save cn007b/7380f681b01107ffbe05546134a5d146 to your computer and use it in GitHub Desktop.
Save cn007b/7380f681b01107ffbe05546134a5d146 to your computer and use it in GitHub Desktop.
Fetch (scrape) product's data by net/http not uflfetch
package main
import (
"fmt"
"io/ioutil"
"net/http"
"os"
)
func main() {
client := http.Client{}
u := ""
if len(os.Args) > 1 {
u = os.Args[1]
} else {
fmt.Printf("please provide URL")
return
}
req, err := http.NewRequest(http.MethodGet, u, nil)
if err != nil {
fmt.Printf("failed to create request for product's html, error: %s", err)
return
}
req.Header.Set("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36")
res, err := client.Do(req)
if err != nil {
fmt.Printf("failed to get product's html, error: %s", err)
return
}
body, err := ioutil.ReadAll(res.Body)
if err != nil {
fmt.Printf("failed to read response body, error: %s", err)
return
}
fmt.Printf("%s", body)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment