Pandas can’t read hdf5 file created with h5py

by

I’ve worked a little on the pytables module in pandas.io and from what I know pandas interaction with HDF files is limited to specific structures that pandas understands. To see what these look like, you can try

import pandas as pd
import numpy as np
pd.Series(np.zeros((3,5),dtype=np.float32).to_hdf('test.h5','test')

If you open ‘test.h5’ in HDFView, you will see a path /test with 4 items that are needed to recreate the DataFrame.

HDFView of test.h5

So I think your only option for reading in NumPy arrays is to read them in directly and then convert these to Pandas objects.

More Related Contents:

  1. “Large data” workflows using pandas [closed]
  2. Optimal HDF5 dataset chunk shape for reading rows
  3. HDF5 – concurrency, compression & I/O performance [closed]
  4. which is faster for load: pickle or hdf5 in python
  5. Incremental writes to hdf5 with h5py
  6. How to read a v7.3 mat file via h5py?
  7. Is there an analysis speed or memory usage advantage to using HDF5 for large array storage (instead of flat binary files)?
  8. Pandas ParserError EOF character when reading multiple csv files to HDF5
  9. What is the difference between the two ways of accessing the hdf5 group in SVHN dataset?
  10. Convert DataFrame column type from string to datetime
  11. How to convert an XML file to nice pandas dataframe?
  12. df.append() is not appending to the DataFrame
  13. Add Leading Zeros to Strings in Pandas Dataframe
  14. Progress indicator during pandas operations
  15. pandas groupby without turning grouped by column into index
  16. How to convert a Scikit-learn dataset to a Pandas dataset
  17. pandas applying regex to replace values
  18. Redefining the Index in a Pandas DataFrame object
  19. pandas .plot() x-axis tick frequency — how can I show more ticks?
  20. Storing numpy sparse matrix in HDF5 (PyTables)
  21. Drop rows with all zeros in pandas data frame
  22. Boolean Series key will be reindexed to match DataFrame index
  23. how to merge two data frames based on particular column in pandas python?
  24. Input and output numpy arrays to h5py
  25. Anti-Join Pandas
  26. Get mapping of categorical variables in pandas
  27. Drop rows on multiple conditions in pandas dataframe
  28. Remove ends of string entries in pandas DataFrame column
  29. why is a sum of strings converted to floats
  30. Pandas: Create missing combination rows with zero values

Leave a Comment