Set iteration order varies from run to run

Why does the iteration order of a Python set (with the same contents) vary from run to run, and what are my options for making it consistent from run to run?

I understand that the iteration order for a Python set is arbitrary. If I put 'a', 'b', and 'c' into a set and then iterate them, they may come back out in any order.

What I've observed is that the order remains the same within a run of the program. That is, if my program iterates the same set twice in a row, I get the same order both times. However, if I run the program twice in a row, the order changes from run to run.

Unfortunately, this breaks one of my automated tests, which simply compares the output from two runs of my program. I don't care about the actual order, but I would like it to be consistent from run to run.

The best solution I've come up with is:

1. Copy the set to a list.
2. Apply an arbitrary sort to the list.
3. Iterate the list instead of the set.

Is there a simpler solution?

Note: I've found similar questions on StackOverlow, but none that address this specific issue of getting the same results from run to run.

The reason the set iteration order changes from run-to-run appears to be because Python uses hash seed randomization by default. (See command option -R.) Thus set iteration is not only arbitrary (because of hashing), but also non-deterministic (because of the random seed).

You can override the random seed with a fixed value by setting the environment variable PYTHONHASHSEED for the interpreter. Using the same seed from run to run means set iteration is still arbitrary, but now it is deterministic, which was the desired property.

Hash seed randomization is a security measure to make it difficult for an adversary to feed inputs that will cause pathological behavior (e.g., by creating numerous hash collisions). For unit testing, this is not a concern, so it's reasonable to override the hash seed while running tests.

Use the symmetric_difference (^) operator on your two sets to see if there are any differences:

In [1]: s1 = set([5,7,8,2,1,9,0])
In [2]: s2 = set([9,0,5,1,8,2,7])
In [3]: s1
Out[3]: set([0, 1, 2, 5, 7, 8, 9])
In [4]: s2
Out[4]: set([0, 1, 2, 5, 7, 8, 9])
In [5]: s1 ^ s2
Out[5]: set()

What you want isn't possible. Arbitrary means arbitrary.

My solution would be the same as yours, you have to sort the set if you want to be able to compare it to another one.

The set's iteration order depends not only its contents, but on the order in which the items were inserted into the set, and whether there were deletions along the way. So you can create two different sets, using different insertions and deletions, and end up with the same set at the end, but with different iteration orders.

As others have said: if you care about the order of the set, you have to create a sorted list from it.