What is symmetric about set symmetric difference?
Using $\Delta$ for set symmetric difference,
$A \Delta B$ is all the elements in exactly one of the sets but not all of them.
$A \Delta B \Delta C $ is all the elements in exactly one of the sets or all of them.
I appreciate there is an even number of sets in the first example and an odd number in the second (and associativity implies no order ambiguity), but what is symmetric about set symmetric difference?
A function in two variables $f(x,y)$ is called symmetric if $f(x,y)=f(y,x)$.
It is easy to see that $A\mathbin{\triangle}B=B\mathbin{\triangle}A$, exactly because being in exactly in one of $A$ or $B$ is the same as being exactly in one of $B$ and $A$.
This is in contrast to set difference, where $A\setminus B$ is generally not the same as $B\setminus A$.
Here is a Venn diagram for normal set difference:
Here is a Venn diagram for symmetric set difference:
Note the symmetry of the latter, and asymmetry of the former.
The term "symmetry" is used in math to refer to an operation that leaves some property unchanged. For instance, if you take a square and rotate it by 90 degrees, you will end up with the same shape. So a square has rotational symmetry.
Generalizing from geometrical symmetry in the previous section, we say that a mathematical object is symmetric with respect to a given mathematical operation, if, when applied to the object, this operation preserves some property of the object.
https://en.wikipedia.org/wiki/Symmetry
In common parlance, "symmetric" is often used to refer to reflectional symmetry, which is where one can reflect a shape about an axis and end up with a shape that is congruent to the original; this is a special case of the more general mathematical meaning of "symmetry".
In the case of functions, "symmetry" refers to the proper where you can switch two variables and end up with a function that evaluates to the same thing. For instance, the distance formula $\sqrt {x^2+y^2}$ is symmetric with respect to $x$ and $y$; if you replace every instance of $x$ with $y$, and every instance of $y$ with $x$, you get $\sqrt {y^2+x^2}$, which is the same thing. A commutative binary operator is symmetric with respect to it operands, but "symmetry" is a more general term, applied to function of more than two inputs, and functions that aren't operators.
Thus, the term "symmetric difference" implies that it doesn't matter what order the inputs are given: the symmetric difference of $A$ and $B$ is the same as the symmetric difference of $B$ and $A$, and the symmetric difference of $(A,B, \text{and } C)$ is the same as for $(A,C, \text{and } B)$, $(B,A, \text{and } C)$, $(B,C, \text{and } A)$, $(C,A, \text{and } B)$, or $(C,B, \text{and } A)$.