Passing C++ vector to Numpy through Cython without copying and taking care of memory management automatically

I think @FlorianWeimer’s answer provides a decent solution (allocate a vector and pass that into your C++ function) but it should be possible to return a vector from doit and avoid copies by using the move constructor.

from libcpp.vector cimport vector

cdef extern from "<utility>" namespace "std" nogil:
  T move[T](T) # don't worry that this doesn't quite match the c++ signature

cdef extern from "fast.h":
    vector[int] doit(int length)

# define ArrayWrapper as holding in a vector
cdef class ArrayWrapper:
    cdef vector[int] vec
    cdef Py_ssize_t shape[1]
    cdef Py_ssize_t strides[1]

    # constructor and destructor are fairly unimportant now since
    # vec will be destroyed automatically.

    cdef set_data(self, vector[int]& data):
       self.vec = move(data)
       # @ead suggests `self.vec.swap(data)` instead
       # to avoid having to wrap move

    # now implement the buffer protocol for the class
    # which makes it generally useful to anything that expects an array
    def __getbuffer__(self, Py_buffer *buffer, int flags):
        # relevant documentation http://cython.readthedocs.io/en/latest/src/userguide/buffer.html#a-matrix-class
        cdef Py_ssize_t itemsize = sizeof(self.vec[0])

        self.shape[0] = self.vec.size()
        self.strides[0] = sizeof(int)
        buffer.buf = <char *>&(self.vec[0])
        buffer.format="i"
        buffer.internal = NULL
        buffer.itemsize = itemsize
        buffer.len = self.v.size() * itemsize   # product(shape) * itemsize
        buffer.ndim = 1
        buffer.obj = self
        buffer.readonly = 0
        buffer.shape = self.shape
        buffer.strides = self.strides
        buffer.suboffsets = NULL

You should then be able to use it as:

cdef vector[int] array = doit(length)
cdef ArrayWrapper w
w.set_data(array) # "array" itself is invalid from here on
numpy_array = np.asarray(w)

---

Edit: Cython isn’t hugely good with C++ templates – it insists on writing std::move<vector<int>>(...) rather than std::move(...) then letting C++ deduce the types. This sometimes causes problems with std::move. If you’re having issues with it then the best solution is usually to tell Cython about only the overloads you want:

 cdef extern from "<utility>" namespace "std" nogil:
    vector[int] move(vector[int])