Python: how to store a numpy multidimensional array in PyTables?

by

There may be a simpler way, but this is how you’d go about doing it, as far as I know:

import numpy as np
import tables

# Generate some data
x = np.random.random((100,100,100))

# Store "x" in a chunked array...
f = tables.open_file('test.hdf', 'w')
atom = tables.Atom.from_dtype(x.dtype)
ds = f.createCArray(f.root, 'somename', atom, x.shape)
ds[:] = x
f.close()

If you want to specify the compression to use, have a look at tables.Filters. E.g.

import numpy as np
import tables

# Generate some data
x = np.random.random((100,100,100))

# Store "x" in a chunked array with level 5 BLOSC compression...
f = tables.open_file('test.hdf', 'w')
atom = tables.Atom.from_dtype(x.dtype)
filters = tables.Filters(complib='blosc', complevel=5)
ds = f.createCArray(f.root, 'somename', atom, x.shape, filters=filters)
ds[:] = x
f.close()

There’s probably a simpler way for a lot of this… I haven’t used pytables for anything other than table-like data in a long while.

Note: with pytables 3.0, f.createCArray was renamed to f.create_carray. It can also accept the array directly, without specifying the atom,

f.create_carray("https://stackoverflow.com/", 'somename', obj=x, filters=filters)

More Related Contents:

  1. Using numpy to build an array of all combinations of two arrays
  2. Understanding NumPy’s einsum
  3. Intuition and idea behind reshaping 4D array to 2D array in NumPy
  4. Why is numpy’s einsum faster than numpy’s built in functions?
  5. Calculate mean across dimension in a 2D array
  6. Selecting specific rows and columns from NumPy array
  7. Convert a 1D array to a 2D array in numpy
  8. Subsetting a 2D numpy array
  9. Check if two 3D numpy arrays contain overlapping 2D arrays
  10. Selecting Random Windows from Multidimensional Numpy Array Rows
  11. How to split a numpy array in fixed size chunks with and without overlap?
  12. How to find the index of a value in 2d array in Python?
  13. What is the difference between ndarray and array in NumPy?
  14. Access n-th dimension in python [duplicate]
  15. How does the axis parameter from NumPy work?
  16. Mapping a NumPy array in place
  17. How to convert a 3d numpy array to 2d
  18. Sorting arrays in NumPy by column
  19. Numpy matrix to array
  20. Replacing Numpy elements if condition is met
  21. How to make a multidimension numpy array with a varying row size?
  22. Force NumPy ndarray to take ownership of its memory in Cython
  23. How to normalize a NumPy array to within a certain range?
  24. How do I use np.newaxis?
  25. Sort multidimensional array based on 2nd element of the subarray
  26. Numpy: Creating a complex array from 2 real ones?
  27. Repeat each values of an array different times
  28. How to create 2d array with numpy random.choice for every rows?
  29. What is the difference between np.array() and np.asarray()?
  30. Good ways to “expand” a numpy ndarray?

Leave a Comment