How to use Python multiprocessing Pool.map to fill numpy array in a for loop

by

The following works. First it is a good idea to protect the main part of your code inside a main block in order to avoid weird side effects. The result of pool.map() is a list containing the evaluations for each value in the iterator list_start_vals, such that you don’t have to create array_2D before.

import numpy as np
from multiprocessing import Pool

def fill_array(start_val):
    return list(range(start_val, start_val+10))

if __name__=='__main__':
    pool = Pool(processes=4)
    list_start_vals = range(40, 60)
    array_2D = np.array(pool.map(fill_array, list_start_vals))
    pool.close() # ATTENTION HERE
    print array_2D

perhaps you will have trouble using pool.close(), from the comments of @hpaulj you can just remove this line in case you have problems…

More Related Contents:

  1. Multiprocessing.Pool makes Numpy matrix multiplication slower
  2. Can’t pickle when using multiprocessing Pool.map()
  3. How to print the full NumPy array, without truncation?
  4. Convert Python sequence to NumPy array, filling missing values
  5. Numpy array assignment with copy
  6. numpy matrix vector multiplication [duplicate]
  7. Combine Pool.map with shared memory Array in Python multiprocessing
  8. Repeating each element of a numpy array 5 times
  9. Filling a queue and managing multiprocessing in python
  10. Ambiguity in Pandas Dataframe / Numpy Array “axis” definition
  11. How to create a numpy array of arbitrary length strings?
  12. Numpy modify array in place?
  13. Subsetting a 2D numpy array
  14. How to normalize a 2-dimensional numpy array in python less verbose?
  15. Passing multiple parameters to pool.map() function in Python [duplicate]
  16. What is the difference between ndarray and array in NumPy?
  17. Removing elements from an array that are in another array
  18. check for identical rows in different numpy arrays
  19. pd.Timestamp versus np.datetime64: are they interchangeable for selected uses?
  20. Numpy: For every element in one array, find the index in another array
  21. Prevent numpy from creating a multidimensional array
  22. Inverting a numpy boolean array using ~
  23. Cannot understand numpy argpartition output
  24. Using numpy ‘module’ object has no attribute ‘array’ [duplicate]
  25. Python: Split NumPy array based on values in the array
  26. Python, OpenCV: Increasing image brightness without overflowing UINT8 array
  27. Sorting a python array/recarray by column
  28. Numpy Array Get row index searching by a row
  29. Resample a numpy array
  30. Convert list of lists with different lengths to a numpy array [duplicate]

Leave a Comment