numpy.genfromtxt produces array of what looks like tuples, not a 2D array—why?

by

What is returned is called a structured ndarray, see e.g. here: http://docs.scipy.org/doc/numpy/user/basics.rec.html. This is because your data is not homogeneous, i.e. not all elements have the same type: the data contains both strings (the first two columns) and floats. Numpy arrays have to be homogeneous (see here for an explanation).

The structured array ‘solves’ this constraint of homogeneity by using tuples for each record or row, that’s the reason the returned array is 1D: one series of tuples, but each tuple (row) consists of several fields, so you can regard it as rows and columns. The different columns are accessible as a['nameofcolumn'] e.g. a['Julian_Day'].

The reason that it returns a 2D array when removing the converters for the first two columns is that in that case, genfromtxt regards all data of the same type, and a normal ndarray is returned (the default type is float, but you can specify this with the dtype argument).

EDIT: If you want to make use of the column names, you can use the names argument (and set the skip_header at only three):

a2 = np.genfromtxt("input.txt", delimiter=",", skip_header=3, names = True, dtype = None,
                  usecols=[0, 1] + radii_indices, converters={0: date_conv, 1: time_conv})

the you can do e.g.:

>>> a2['Dateddmmyyyy']
array(['06/03/2006', '06/03/2006', '18/03/2006', '19/03/2006',
       '19/03/2006', '19/03/2006', '19/03/2006', '19/03/2006',
       '19/03/2006', '19/03/2006'], 
      dtype="|S10")

More Related Contents:

  1. How do I read CSV data into a record array in NumPy?
  2. Import Error: No module named numpy
  3. Using numpy.genfromtxt to read a csv file with strings containing commas
  4. Error “Import Error: No module named numpy” on Windows
  5. Using genfromtxt to import csv data with missing values in numpy
  6. Convert a 3D array to RGB valued array [closed]
  7. Unable to allocate array with shape and data type
  8. Replacing Numpy elements if condition is met
  9. How to make a multidimension numpy array with a varying row size?
  10. What does the term “broadcasting” mean in Pandas documentation?
  11. Save / load scipy sparse csr_matrix in portable data format
  12. subsampling every nth entry in a numpy array
  13. Do I need to import submodules directly?
  14. Numpy and line intersections
  15. Loading UTF-8 file in Python 3 using numpy.genfromtxt
  16. What does a colon and comma stand in a python list?
  17. convert string representation of array to numpy array in python
  18. Prevent Python from caching the imported modules
  19. How to share numpy random state of a parent process with child processes?
  20. numpy reverse multidimensional array
  21. How can I import from the standard library, when my project has a module with the same name? (How can I control where Python looks for modules?)
  22. How do I get Pylint to recognize NumPy members?
  23. Why doesn’t an import in an exec in a function work?
  24. Form a big 2d array from multiple smaller 2d arrays
  25. Sieve of Eratosthenes – Primes between X and N
  26. Is the order of a Python dictionary guaranteed over iterations?
  27. numpy arange: how to make “precise” array of floats?
  28. Vectorizing `numpy.random.choice` for given 2D array of probabilities along an axis
  29. How can I turn a flat list into a 2D array in python?
  30. Sort array’s rows by another array in Python

Leave a Comment