Use tqdm with concurrent.futures?

by

You can wrap tqdm around the executor as the following to track the progress:

list(tqdm(executor.map(f, iter), total=len(iter))

Here is your example:

import time  
import concurrent.futures
from tqdm import tqdm

def f(x):
    time.sleep(0.001)  # to visualize the progress
    return x**2

def run(f, my_iter):
    with concurrent.futures.ThreadPoolExecutor() as executor:
        results = list(tqdm(executor.map(f, my_iter), total=len(my_iter)))
    return results

my_iter = range(100000)
run(f, my_iter)

And the result is like this:

16%|██▏           | 15707/100000 [00:00<00:02, 31312.54it/s]

More Related Contents:

  1. Multiprocessing : use tqdm to display a progress bar
  2. Is having a concurrent.futures.ThreadPoolExecutor call dangerous in a FastAPI endpoint?
  3. tqdm in Jupyter Notebook prints new progress bars repeatedly
  4. Python parallel execution with selenium
  5. Jupyter Notebooks not displaying progress bars
  6. Python: concurrent.futures How to make it cancelable?
  7. Multiprocessing Share Unserializable Objects Between Processes
  8. Finding smaller numbers in an array
  9. Determine the type of an object?
  10. When to use cla(), clf() or close() for clearing a plot in matplotlib?
  11. How to state in requirements.txt a direct github source
  12. Python List vs. Array – when to use?
  13. Difference between class and instance methods
  14. Pandas read in table without headers
  15. How to split the definition of a long string over multiple lines?
  16. Key Listeners in python?
  17. ReactorNotRestartable error in while loop with scrapy
  18. What does ‘wb’ mean in this code, using Python?
  19. Calling Python 2 script from Python 3
  20. how do I calculate a rolling idxmax
  21. list python package dependencies without loading them?
  22. How to compile a Python package to a dll
  23. Using stdin in a child Process
  24. open url from pythonanywhere
  25. Animating “growing” line plot in Python/Matplotlib
  26. Vectorized spatial distance in python using numpy
  27. Python: iterating over list vs over dict items efficiency
  28. Select the max row per group – pandas performance issue
  29. Python super() arguments: why not super(obj)?
  30. Collectstatic error while deploying Django app to Heroku

Leave a Comment