Python: sort function breaks in the presence of nan

by

The previous answers are useful, but perhaps not clear regarding the root of the problem.

In any language, sort applies a given ordering, defined by a comparison function or in some other way, over the domain of the input values. For example, less-than, a.k.a. operator <, could be used throughout if and only if less than defines a suitable ordering over the input values.

But this is specifically NOT true for floating point values and less-than:
“NaN is unordered: it is not equal to, greater than, or less than anything, including itself.” (Clear prose from GNU C manual, but applies to all modern IEEE754 based floating point)

So the possible solutions are:

  1. remove the NaNs first, making the input domain well defined via <
    (or the other sorting function being used)
  2. define a custom comparison function (a.k.a. predicate) that does
    define an ordering for NaN, such as less than any number, or greater
    than any number.

Either approach can be used, in any language.

Practically, considering python, I would prefer to remove the NaNs if you either don’t care much about fastest performance or if removing NaNs is a desired behavior in context.

Otherwise you could use a suitable predicate function via “cmp” in older python versions, or via this and functools.cmp_to_key(). The latter is a bit more awkward, naturally, than removing the NaNs first. And care will be required to avoid worse performance, when defining this predicate function.

More Related Contents:

  1. Python Math – TypeError: ‘NoneType’ object is not subscriptable
  2. Sorting list based on values from another list
  3. Sieve of Eratosthenes – Finding Primes Python
  4. Check if a number is a perfect square
  5. Scrapy Crawl URLs in Order
  6. How does the key argument in python’s sorted function work?
  7. Sorting Python list based on the length of the string
  8. Math operations from string [duplicate]
  9. Difference between the built-in pow() and math.pow() for floats, in Python?
  10. How can I use numpy.correlate to do autocorrelation?
  11. List sorting with multiple attributes and mixed order
  12. Sorting a dictionary with lists as values, according to an element from the list
  13. Exponentials in python: x**y vs math.pow(x, y)
  14. django – convert a list back to a queryset [duplicate]
  15. Pandas DataFrames with NaNs equality comparison
  16. How does python compare functions?
  17. How do I find the distance between two points? [closed]
  18. Sort list of strings by a part of the string
  19. scipy is not optimizing and returns “Desired error not necessarily achieved due to precision loss”
  20. Sorting by absolute value without changing to absolute value
  21. What algorithm does Python’s built-in sort() method use?
  22. Pythonic way to sorting list of namedtuples by field name
  23. Equivalent of `polyfit` for a 2D polynomial in Python
  24. Get all numbers that add up to a number
  25. Python’s and Numpy’s nan and set
  26. Sort list while pushing None values to the end
  27. What arguments does Python sort() function have?
  28. Sort strings by the first N characters
  29. Multi Index Sorting in Pandas
  30. Sort array’s rows by another array in Python

Leave a Comment