Python: sort function breaks in the presence of nan
The previous answers are useful, but perhaps not clear regarding the root of the problem.
In any language, sort applies a given ordering, defined by a comparison function or in some other way, over the domain of the input values. For example, less-than, a.k.a. operator <,
could be used throughout if and only if less than defines a suitable ordering over the input values.
But this is specifically NOT true for floating point values and less-than:
"NaN is unordered: it is not equal to, greater than, or less than anything, including itself." (Clear prose from GNU C manual, but applies to all modern IEEE754
based floating point)
So the possible solutions are:
- remove the NaNs first, making the input domain well defined via < (or the other sorting function being used)
- define a custom comparison function (a.k.a. predicate) that does define an ordering for NaN, such as less than any number, or greater than any number.
Either approach can be used, in any language.
Practically, considering python, I would prefer to remove the NaNs if you either don't care much about fastest performance or if removing NaNs is a desired behavior in context.
Otherwise you could use a suitable predicate function via "cmp" in older python versions, or via this and functools.cmp_to_key()
. The latter is a bit more awkward, naturally, than removing the NaNs first. And care will be required to avoid worse performance, when defining this predicate function.
I'm not sure about the bug, but the workaround may be the following:
sorted(
(2, 1, float('nan')),
lambda x,y: x is float('nan') and -1
or (y is float('nan') and 1
or cmp(x,y)))
which results in:
('nan', 1, 2)
Or remove nan
s before sorting or anything else.