Skip to content

Instantly share code, notes, and snippets.

@montanaflynn
Last active October 19, 2023 23:54
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save montanaflynn/9049fc14193d83b5cd981ba582e772f4 to your computer and use it in GitHub Desktop.
Save montanaflynn/9049fc14193d83b5cd981ba582e772f4 to your computer and use it in GitHub Desktop.
Example of using Golang errgroup with context.WithTimeout()
package main
import (
"context"
"fmt"
"io"
"log"
"net/http"
"os"
"time"
"golang.org/x/sync/errgroup"
)
var (
httpClient = http.Client{}
baseURL = "https://funkeinteraktiv.b-cdn.net"
currentDataEndpoint = "/current.v4.csv"
historicalDataEndpoint = "/history.light.v4.csv"
currentDataURL = fmt.Sprintf("%s%s", baseURL, currentDataEndpoint)
historicalDataURL = fmt.Sprintf("%s%s", baseURL, historicalDataEndpoint)
)
func saveData(ctx context.Context, url, file string) error {
errChan := make(chan error)
go func() {
req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
if err != nil {
errChan <- err
return
}
req.Header.Add("user-agent", "Mozilla/5.0")
res, err := httpClient.Do(req)
if err != nil {
errChan <- err
return
}
defer res.Body.Close()
f, err := os.Create(file)
if err != nil {
errChan <- err
return
}
defer f.Close()
_, err = io.Copy(f, res.Body)
if err != nil {
errChan <- err
return
}
errChan <- nil
}()
for {
select {
case <-ctx.Done():
return ctx.Err()
case err := <-errChan:
if err != nil {
return fmt.Errorf("saveData: %w", err)
}
return nil
}
}
return nil
}
func saveOriginalData() error {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
g, ctx := errgroup.WithContext(ctx)
defer cancel()
fn := func(ctx context.Context, url, file string) func() error {
return func() error {
return saveData(ctx, url, file)
}
}
g.Go(fn(ctx, currentDataURL, "./data/current.csv"))
g.Go(fn(ctx, historicalDataURL, "./data/historical.csv"))
return g.Wait()
}
func main() {
err := saveOriginalData()
if err != nil {
log.Fatal(err)
}
}
@youngsailor
Copy link

good

@chenfeng8087
Copy link

chenfeng8087 commented Mar 10, 2023

I have some questions: say L85 starts go-routine A and L86 starts go-routine B.

  • Will L27~L57 starts new go-routines? Say A start X and B start Y.
  • Suppose X runs 100ms then failed due to 5xx. And Y runs 5 seconds then successes.
  • When will A/B/Y be finished/cancelled?
  • In my understanding, 1) X fails first after 100ms due to 5xx, 2) then A fails due to L63, which returns error to errgroup in L65. 3) then B fails due to L61. 4) Y successes after 5s cause no one stops it.
    Am I right? Should Y be stopped correctly?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment