The set of all permutations of indices such that the new series converges to the same limit forms a group?
Here is a reasonably simple counterexample. Let $\sigma\colon\mathbb{N}\to\mathbb{N}$ be the following permutation of the natural numbers: $$ \begin{array}{|c|c|c|c|c|c|c|c|c|c|c|c|c|c|} \hline n & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10 & 11 & 12 & \cdots \\ \hline \sigma(n) & \color{blue}2 & \color{blue}4 & \color{blue}6 & \color{red}1 & \color{blue}8 & \color{blue}{10} & \color{blue}{12} & \color{red}3 & \color{blue}{14} & \color{blue}{16} & \color{blue}{18} & \color{red}{5} & \cdots\\ \hline \end{array} $$ with three even numbers for every odd number. We will demonstrate a convergent series with the property that $\sigma$ does not change its value, but $\sigma^2$ does.
Let $\{a_n\}_{n=1}^\infty$ be the sequence $$ -1,2,-1,\;-\frac12,\frac22,-\frac12,\;-\frac13,\frac23,-\frac13,\;-\frac14,\frac24,-\frac14,\;\ldots $$ and let $\{b_n\}_{n=1}^\infty$ be the sequence whose odd terms are zero, and whose even terms satsify $b_{2n} = a_n$. Then the series $$ \sum_{n=1}^\infty b_n \;=\; 0 - 1 + 0 + 2 + 0 - 1 \;+\; 0 - \frac12 + 0 + \frac22 + 0 - \frac12 + 0 \;+\; \cdots $$ converges to $0$. Applying the permutation $\sigma$ to the terms of this series gives $$ \sum_{n=1}^\infty c_n \;=\; \color{blue}{-1+2-1} \color{red}{+ 0} \color{blue}{- \frac12 + \frac22 - \frac12} \color{red}{+ 0} \color{blue}{- \frac13 + \frac23 - \frac13} \color{red}{+ 0} + \cdots $$ which also converges to $0$. However, applying $\sigma$ again yields $$ \sum_{n=1}^\infty d_n \;=\; \color{blue}{2 + 0 + \frac22} \color{red}{- 1} \color{blue}{+ 0 + \frac23 + 0} \color{red}{-1} \color{blue}{+\frac24 + 0 + \frac25} \color{red}{- \frac12} + \cdots. $$ We claim that this series does not converge to $0$.
To prove this, observe that the $8N$'th partial sum of this series is $$ \sum_{n=1}^{8N} d_n \;=\; \left(\sum_{k=1}^{3N} \frac{2}{k} \right) - \left(2\sum_{k=1}^{N} \frac{1}{k} \right) \;=\; 2H_{3N} - 2H_N, $$ where $H_i$ denotes the $i$'th harmonic number. It is known that $$ H_n \;=\; \log(n) + \gamma + o(1) $$ where $\gamma$ is the Euler-Mascheroni constant, and therefore $$ \begin{align*} \sum_{n=1}^{8N} d_n \;&=\; 2H_{3N} - 2H_N \\ \;&=\; 2\bigl(\log(3N) + \gamma + o(1) \bigr) - 2\bigl(\log(N) + \gamma + o(1)\bigr) \\[2ex] \;&=\; \log(9) + o(1) \end{align*} $$ Since the terms of the series converge to zero, it follows that $$ \sum_{n=1}^\infty d_n \;=\; \log(9). $$
Edit:
By the way, I should mention how I made this example up. Suppose we are given a permutation $\sigma$ and a series $\sum_{n=1}^\infty s_n$ with the following properties:
The distance $|\sigma(n) - n|$ that $\sigma$ moves terms is unbounded.
The series is conditionally convergent.
Under these conditions, $\sigma$ will almost always change the value of the sum (or change the convergent series to a divergent one).
So all I did was pick the simplest permutation $\sigma$ that I could think of, and then I created a conditionally convergent series with $0$'s in the right places to be unchanged by $\sigma$.
Under these conditions, it will usually be the case that $\sigma^2$ changes the value of the sum. The only hard part is making sure that the original series converges, and that the original sum and the final sum can be evaluated explicitly.