Java 5+ has all the tools you need for this kind of thing. You will want to:
- Put all your Producers in one
ExecutorService
; - Put all your Consumers in another
ExecutorService
; - If necessary, communicate between the two using a
BlockingQueue
.
I say “if necessary” for (3) because from my experience it’s an unnecessary step. All you do is submit new tasks to the consumer executor service. So:
final ExecutorService producers = Executors.newFixedThreadPool(100);
final ExecutorService consumers = Executors.newFixedThreadPool(100);
while (/* has more work */) {
producers.submit(...);
}
producers.shutdown();
producers.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
consumers.shutdown();
consumers.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
So the producers
submit directly to consumers
.