Getting data from ctypes array into numpy

by

Creating NumPy arrays from a ctypes pointer object is a problematic operation. It is unclear who actually owns the memory the pointer is pointing to. When will it be freed again? How long is it valid? Whenever possible I would try to avoid this kind of construct. It is so much easier and safer to create arrays in the Python code and pass them to the C function than to use memory allocated by a Python-unaware C function. By doing the latter, you negate to some extent the advantages of having a high-level language taking care of the memory management.

If you are really sure that someone takes care of the memory, you can create an object exposing the Python “buffer protocol” and then create a NumPy array using this buffer object. You gave one way of creating the buffer object in your post, via the undocumented int_asbuffer() function:

buffer = numpy.core.multiarray.int_asbuffer(
    ctypes.addressof(y.contents), 8*array_length)

(Note that I substituted 8 for np.dtype(float).itemsize. It’s always 8, on any platform.) A different way to create the buffer object would be to call the PyBuffer_FromMemory() function from the Python C API via ctypes:

buffer_from_memory = ctypes.pythonapi.PyBuffer_FromMemory
buffer_from_memory.restype = ctypes.py_object
buffer = buffer_from_memory(y, 8*array_length)

For both these ways, you can create a NumPy array from buffer by

a = numpy.frombuffer(buffer, float)

(I actually do not understand why you use .astype() instead of a second parameter to frombuffer; furthermore, I wonder why you use np.int, while you said earlier that the array contains doubles.)

I’m afraid it won’t get much easier than this, but it isn’t that bad, don’t you think? You could bury all the ugly details in a wrapper function and don’t worry about it any more.

More Related Contents:

  1. Passing Numpy arrays to a C function for input and output
  2. How do I pass large numpy arrays between python subprocesses without saving to disk?
  3. How to use NumPy array with ctypes?
  4. Convert array dictionary to a dictionary python
  5. Plot a function in python
  6. How do I get indices of N maximum values in a NumPy array?
  7. How to convert a PIL Image into a numpy array?
  8. Fast Haversine Approximation (Python/Pandas)
  9. Index n dimensional array with (n-1) d array
  10. Roll rows of a matrix independently
  11. View onto a numpy array?
  12. how does multiplication differ for NumPy Matrix vs Array classes?
  13. Using Numpy Vectorize on Functions that Return Vectors
  14. how to handle an asymptote/discontinuity with Matplotlib
  15. Why is numpy.array so slow?
  16. Efficient evaluation of a function at every cell of a NumPy array
  17. Numpy individual element access slower than for lists
  18. How to turn a video into numpy array?
  19. AttributeError: ‘numpy.float64’ object has no attribute ‘log10’
  20. Why does numpy std() give a different result to matlab std()?
  21. generalized cumulative functions in NumPy/SciPy?
  22. Efficiently replace elements in array based on dictionary – NumPy / Python
  23. Function to determine if two numbers are nearly equal when rounded to n significant decimal digits
  24. How to generate a random normal distribution of integers
  25. Installing lapack for numpy
  26. Adding colors to a 3d quiver plot in matplotlib
  27. Why is B = numpy.dot(A,x) so much slower looping through doing B[i,:,:] = numpy.dot(A[i,:,:],x) )?
  28. Indexing numpy array with another numpy array
  29. Append 2D array to 3D array, extending third dimension
  30. WindowsError: [Error 126] when loading a DLL with ctypes

Leave a Comment