How would you define a pool of goroutines to be executed at once?

by

I would spawn 4 worker goroutines that read the tasks from a common channel. Goroutines that are faster than others (because they are scheduled differently or happen to get simple tasks) will receive more task from this channel than others. In addition to that, I would use a sync.WaitGroup to wait for all workers to finish. The remaining part is just the creation of the tasks. You can see an example implementation of that approach here:

package main

import (
    "os/exec"
    "strconv"
    "sync"
)

func main() {
    tasks := make(chan *exec.Cmd, 64)

    // spawn four worker goroutines
    var wg sync.WaitGroup
    for i := 0; i < 4; i++ {
        wg.Add(1)
        go func() {
            for cmd := range tasks {
                cmd.Run()
            }
            wg.Done()
        }()
    }

    // generate some tasks
    for i := 0; i < 10; i++ {
        tasks <- exec.Command("zenity", "--info", "--text="Hello from iteration n."+strconv.Itoa(i)+""")
    }
    close(tasks)

    // wait for the workers to finish
    wg.Wait()
}

There are probably other possible approaches, but I think this is a very clean solution that is easy to understand.

More Related Contents:

  1. Is this an idiomatic worker thread pool in Go?
  2. Number of threads used by Go runtime
  3. What is a race condition?
  4. What is the difference between a process and a thread?
  5. What is the difference between concurrency, parallelism and asynchronous methods?
  6. Multi-threading in VBA
  7. What is a mutex?
  8. What is a semaphore?
  9. What is the Re-entrant lock and concept in general?
  10. Thread context switch Vs. process context switch
  11. How to Multi-thread an Operation Within a Loop in Python
  12. How Do I Choose Between the Various Ways to do Threading in Delphi?
  13. Is non-blocking I/O really faster than multi-threaded blocking I/O? How?
  14. Python 2.7: streaming HTTP server supporting multiple connections on one port
  15. How to manage db connections on server?
  16. Can x86 reorder a narrow store with a wider load that fully contains it?
  17. Conditional Variable vs Semaphore
  18. Using HttpContext.Current in WebApi is dangerous because of async
  19. Max number of goroutines
  20. Asynchronous IO in Scala with futures
  21. How do I Understand Read Memory Barriers and Volatile
  22. Canonical way to generate random numbers in Cython
  23. Can a hyper-threaded processor core execute two threads at the exact same time?
  24. Prolog find all paths Implementation
  25. How do I run parallel threads of computation on a partitioned array?
  26. Do lock-free algorithms really perform better than their lock-full counterparts?
  27. anonymous struct and empty struct
  28. Can we have race conditions in a single-thread program?
  29. Can a multi-threaded program ever be deterministic?
  30. How to recover from concurrent map writes?

Leave a Comment