Please Explain Kuratowski Definition of Ordered Pairs

I've seen this Kuratowski definition for ordered pairs, but can't fathom why it implies an order to $x$ and $y$

$(x,y):=\{\{x\}, \{x,y\}\}$

As I understand sets, $\{\{x\}, \{x,y\}\}$ is also $\{\{x,y\}, \{x\}\}$. Only when I think about the Axiom of Union does $\{\{x\}, \{x,y\}\}$ "collapse down" to $S = \{x, y\}$, but that doesn't give me much either. All I can see is some as yet hidden message in the set saying "I am the set $\{x,y\}$ and my order of $x$ first is indicated by having $\{x\}$ along for the ride."

You said:

I can't fathom why it implies an order to $x$ and $y$

It doesn't really. The $x$ and $y$ in the ordered pair $(x, y)$ don't really have an order. Who's to say that the $x$ is first and the $y$ is second? If you read right-to-left, you'd say that the $y$ was first and the $x$ was second.

The important thing isn't which is first. The important thing is that the set we pick to represent $(x,y)$ must be different from the set that represents $(y, x)$, because these are different pairs.

As you pointed out $\{x, y\}$ is the same set as $\{y, x\}$. But let's consider the Kuratowski pairs $(x,y)$ and $(y,x)$:

$$(x,y) = \{\{x\},\{x,y\}\}\\ (y,x) = \{\{y\},\{x,y\}\} $$

Hey look, they’re different sets. That's what we needed.

Kuratowski's definition was preceded by a number of others. The one by Felix Hausdorff may make you feel more comfortable:

$$(x,y) = \{\{x, 1\},\{y,2\}\}\\ (y,x) = \{\{y, 1\},\{x,2\}\} $$

Now the order you wanted is explicit.

But it is important to realize that the $1$ and $2$ here are completely arbitrary markers! It would have worked just as well for Hausdorff to have used some different markers to indicate which component was first:

$$(x,y) = \{\{x, \text{potato}\},\{y,\text{banana}\}\}\\ (y,x) = \{\{y, \text{potato}\},\{x,\text{banana}\}\}\\ $$

Now you can recognize the first component of the pair because it is associated with $\text{potato}$.

The point is that the details of the particular representation aren't important. We only care that the representation does what we need it to. For ordered pairs, we need to be able to form the pair $(x, y)$ for any $x$ and $y$; we need to be able to extract the components again, and crucially, we need $(x,y) $ to be equal to $(a, b)$ if and only if $x=a$ and $y=b$. Kuratowski’s definition and Hausdorff's both do this, and so do many other definitions.

Which definition we pick is not really important. What is important is that the objects we choose to represent ordered pairs must behave like ordered pairs. If we get that much, we are mathematically satisfied. The Kuratowski definition isn't used because it captures some basic essence of ordered pair-ness, but because it does that we need it to do, which is just enough.

Just how you define order pairs concretely is an "implementation detail". Whichever definition you adopt just has to meet a basic requirement: from $\langle x, y \rangle$, you must be able to uniquely recover each of $x$ and $y$ with (preferably simple) functions $first(z)$ and $second(z)$.

The Kuratowski construction meets this criterion.

$first((x,y)) = x$

Note that $\{x\} = \{x\} \cap \{x,y\} = \bigcap \{\{x\}, \{x,y\}\} = \bigcap z$ where $z = (x,y)$. Now, as for any set, $\bigcup\{x\} = x$. So if $z = (x,y)$ then $x = \bigcup \bigcap z$, so we can define $first$ as: $$ first(z) = \bigcup \bigcap z. $$

$second((x,y)) = y$

Taking the union gives $\{x,y\} = \{x\} \cup \{x,y\} = \bigcup \{\{x\}, \{x,y\}\} = \bigcup z$ where $z = (x,y)$. Consider $\{x,y\} \setminus \{x\}$, which in terms of $z$ is $\bigcup z \setminus \bigcap z$. It's equal to $\emptyset$ if $y = x$, and equal to $\{y\}$ otherwise. So in any case we can recover (return) $y$ by defining $$ second(z) = \begin{cases} \\ &first(z)&\quad\text{if $\bigcup z \setminus \bigcap z = \emptyset$}, \\ &\bigcup (\bigcup z \setminus \bigcap z) &\quad\text{otherwise}, \\ \end{cases}$$

These definitions meet the essential requirement: $$ z = (x, y) \iff [first(z) = x \text{ and } second(z) = y], $$ and all three functions have elementary definitions.

For any set $A$, $\bigcup A$ is the set of all things $z$ that are members of some thing $y$ in $A$: that is, $\bigcup A = \{ z\mid (\exists y\in A)\,z\in y\}$. In "union of a family of sets" notation, $\bigcup A = \bigcup_{a\in A} a$. So $\bigcup \{x\} = \{z\mid(\exists y\in \{x\})\,z\in y\} = \{z\mid(\exists y = x)\,z\in y\} = \{z\mid z\in x\} = x$.