Questions regarding Riemann's rearrangement theorem

I have recently learned about Riemann's rearrangement theorem, and I have some questions regarding the theorem.

Let $$\sum a_n = a_1 + a_2 + a_3 + a_4 + \cdots$$ be conditionally convergent. If we switch two of the terms, for example $a_2$ and $a_3$, we get the new series $$a_1+a_3+a_2+a_4+\cdots.$$ Both the new and the old series have the same sum, and this will be the case no matter which two terms we switch. Then my (obviously wrong) claim is this: A rearrangment is just repeated switching of terms.

Why is my claim wrong? I can imagine that it has something to do with the fact that for many rearrangements one has to switch two terms infinitely many times. If that is the case, I guess that one can do any finite amount of switching without altering the sum. Is this correct?

I know that it was Riemann who first proved this theorem, but I can't find out when and where. Did he write a paper on it, and if he did, where can I find a copy of it?

I am also grateful for any related information that could be of interest.

Riemann's rearrangement theorem: If an infinite series is conditionally convergent, then its terms can be arranged in a permutation so that the series converges to any given value, or even diverges. (Wikipedia)

I appreciate all help.

Unlike finite symmetric groups, the infinite symmetric group $\text{Sym}(\mathbb{N})$ is not generated by transpositions. The problem is that an infinite composition of transpositions doesn't make sense: consider the composition of $(1 \; 2)$ followed by $(2 \; 3)$ followed by $(3 \; 4)$ and so on to infinity. Where does $1$ go? So yes, rearranging only finitely many terms necessarily doesn't affect the value of the series, but that doesn't imply anything about rearranging infinitely many terms.

I can imagine that it has something to do with the fact that for many rearrangements one has to switch two terms infinitely many times. If that is the case, I guess that one can do any finite amount of switching without altering the sum. Is this correct?

Yes and yes. The subgroup of $\text{Aut}(\mathbb{N})$ generated by transpositions is the group of bijections which fix all but finitely many points. There are countably many such bijections, but uncountably many bijections $\mathbb{N} \to \mathbb{N}$ (exercise), so this subgroup is not the entire group.

The theorem appears in the paper where Riemann also defines the Riemann integral: Ueber die Darstellbarkeit einer Function durch eine trigonometrische Reihe. It's at the beginning of section 3 (the integral comes in section 4). You can find it Riemann's collected works, which is freely available online, for example here; see p. 221 (which is p. 236 in the file).