Semaphore Concurrency Pattern with Buffered Channel in Go

semaphore.go

package main

import (
	"fmt"
	"time"
)

const (
	NUM_WORKERS = 100
	NUM_JOBS    = 1000
)

type Token struct{}

func main() {
	// A semaphore is a buffered channel
	semaphore := make(chan Token, NUM_WORKERS)

	for i := 1; i <= NUM_JOBS; i++ {

		job := i

		// Simply start goroutines when you have concurrent work.
		semaphore <- Token{}
		go func() {
			fmt.Printf("Performing %d th job\n", job)
			// fmt.Printf("Current Goroutines: %d\n", runtime.NumGoroutine())
			time.Sleep(100 * time.Millisecond)
			<-semaphore
		}()
	}

	// This ensures the main goroutine does not ends before all the jobs are performed
	for i := 0; i < NUM_WORKERS; i++ {
		semaphore <- Token{}
	}
}

The Worker Pool pattern might be regarded as an anti-pattern because of inefficiency, which means it allows idle goroutines especially at the end of a job batch. Above concurrency pattern using semaphore channel eliminates the issue.

Simply, start goroutines when you have concurrent work.