On two special kind of invertible similar matrices with rational entries
Solution 1:
In a first step we show that the eigenvalues of $A$ are roots of unity. We denote the not necessarily distinct eigenvalues of $A$ by $\{\lambda_1,...,\lambda_N\}$, $N$ the size of $A$. As $BA^sB^{-1}=A^{s+1}$, the matrices $A^s$ and $A^{s+1}$ have the same eigenvalues. This means that there is a permutation $p$ such that $\lambda_i^s=\lambda_{p(i)}^{s+1}$ for all $i$. It follows inductively that $\lambda_i^{s^j}=\lambda_{p^j(i)}^{(s+1)^j}$ for all $j$. Now there is a positive integer $d$ such that $p^d=id$. Then we obtain that $\lambda_i^{(s+1)^d-s^d}=1$ for all $i$ and all eigenvalues are roots of unity.
In the second step we show that $A$ is diagonalisable. Otherwise, let $k+1>1$ denote the maximal length of a Jordan block of $A$. Now we write $A=S+N$ with a diagonalisable $S$ and a nilpotent $N$ which commute. We have $N^{k+1}=0$ and $N^{k}\neq0$. For positive integers $r$, we write $$A^r=S^r+\binom{r}1\,S^{r-1}N+\cdots+\binom{r}k S^{r-k}N^k.$$ As the eigenvalues of $A$ have modulus 1, there exist positive $a_1,b_1$ such that $a_1r^k\leq||A^r||\leq b_1r^k$ for all $r$. Here we use any submultiplicative matrix norm. This fact is proved at the end of the solution. Since $B$ is similar to $A$, there also are positive constants $a_2,b_2,a_3,b_3$ such that $a_2r^k\leq||B^r||\leq b_2r^k$ and $a_3r^k\leq||B^{-r}||\leq b_3r^k$ for all $r$. Now the equation $BA^sB^{-1}=A^{s+1}$ implies inductively that $$\tag{1}B^rA^{s^r}B^{-r}=A^{(s+1)^r}\mbox{ for all }r.$$ We obtain the inequalities $$a_1 (s+1)^{rk}\leq ||A^{(s+1)^r}||=||B^rA^{s^r}B^{-r}||\leq b_1b_2b_3r^{2k}s^{rk}$$ for all $r$. For sufficiently large $r$, this is false and we have a contradiction. Therefore $k=0$ and $A$ is diagonalizable.
Consider now the order $m$ of $A$ (and $B$), that is the smallest positive integer such
that $A^m=I$. It exists because the eigenvalues of $A$ are roots of unity and
$A$ is diagonalisable. We claim that $m=1$ and hence $A=B=I$.
Otherwise (1) implies for $r=m$ using $B^m=I$ that $A^{(s+1)^m-s^m}=I$.
Therefore $m$ divides $(s+1)^m-s^m$. Observe that $m$ must be coprime to $s$ and $s+1$ in this case.
Consider now the smallest prime divisor $p$ of $m$ and its multiplicity $e$.
Then by the binomial theorem $(s+1)^p\equiv s^p+1\mod p$ and inductively also
$(s+1)^{p^e}\equiv s^{p^e}+1\mod p$. In particular $(s+1)^{p^e}$ and $s^{p^e}$ are different
modulo $p$. Now the multiplicative group $({\mathbb Z}/p{\mathbb Z})^\times$ has order $p-1$ and
the quotient $u=m/p^e$ is coprime to $p-1$ because $p$ is the smallest prime divisor of $m$.
Therefore also $(s+1)^{up^e}$ and $s^{up^e}$ are different modulo $p$ and hence also modulo
$m$. Contradiction!
Appendix: Consider $A=S+N$ with a diagonalisable $S$ and a nilpotent $N$ which commute. Assume that all eigenvalues of $A$ have modulus 1 and that $k\geq0$ is such that $N^k\neq0$ and $N^{k+1}=0$. Consider any submultiplicative matrix norm $||.||$.
Then there exist positive constants $a,b$ such that for all positive integers $r$ one has $a\,r^k\leq||A^r||\leq b\,r^k$.
For the proof, we use that for positive integers $r$, we can write
$$\tag{2}A^r=S^r+\binom{r}1\,S^{r-1}N+\cdots+\binom{r}k S^{r-k}N^k.$$
Without loss of generality, we can assume that $S$ is diagonal. Now all norms on a finite dimensional vector space are equivalent and
$||S^r||_\infty=1$ for all $r$ and for the matrix norm
$$||(d_{ij})||_\infty=\max_{i}\sum_j|d_{ij}|$$
corresponding to the maximum norm for vectors.
Hence there are positive constants $c,d$ such that for all $r$ we have $d\leq||S^r||\leq c$.
This implies using (2) that $$||A^r||\leq c+c|||N|r+\cdots\binom rk c||N^k||\leq c(1+||N||+\cdots+||N^k||)r^k$$ for all positive integers $k$ as wanted.
We also have for all $r$
$$||A^r||\geq \binom rk d ||N^k||-c-\cdots -\binom r{k-1} c||N^{k-1}||.$$
Now if $r\geq2k$ then $\binom rk\geq \frac {r^k}{2^kk!}$. As above, we can find a positive constant $M$ such that
$$c+\cdots +\binom r{k-1} c||N^{k-1}||\leq Mr^{k-1}$$
for all positive integers $r$. Together, this gives $$||A^r||\geq r^k \left(\frac{d||N^k||}{2^kk!}-\frac Mr\right)$$ for all $r\geq 2k.$
Therefore if $r\geq 2k$ and $r\geq 2M\,2^kk!/(d||N^k||)$ we find that $||A^r||\geq\frac{d||N^k||}{2^{k+1}k!}r^k$.
Finally, if $a$ denotes the minimum of $\frac{d||N^k||}{2^{k+1}k!}$ and $||A^r||/r^k$ for $1\leq r\leq \max(2k,2M\,2^k k!/(d||N^k||)$ then we have
$||A^r||\geq a r^k$ for all positive $r$ as wanted.
Solution 2:
Let be the multiset of generalised eigenvalues over $\mathbb{C}$ of $A$ be $\lambda_i$. Then since $A^s$ and $A^{s+1}$ are similar (via $B$), we have that the multisets of $\lambda_i^s$ and $\lambda_i^{s+1}$ are equal. So for each $i$, we have $$\lambda_{\sigma(i)}^{s+1}=\lambda_{i}^s$$ for some permutation $\sigma$ of the multisets.
Now $\sigma$ has finite order $n\geq 2$, so consider the $s^n$-th power of $\lambda_i$:
$$\lambda_i^{s^n}=(\lambda_i^s)^{s^{n-1}}=(\lambda_{\sigma(i)}^{s+1})^{s^{n-1}}=(\lambda_{\sigma(i)}^{s^{n-1}})^{s+1}=(\lambda_{\sigma^2(i)}^{s^{n-2}})^{(s+1)^2}=...=(\lambda_{\sigma^n(i)}^{s^{n-n}})^{(s+1)^n}=\lambda_i^{(s+1)^n}$$
So we see that $\lambda^{(s+1)^n-s^n}=1$, so each $\lambda_i$ is a root of unity.
Lets now show that $A$ and $B$ have finite order, that is, their unipotent part is trivial. Let $U$ denote the unipotent part of $A$, and note that we have: $$BU^sB^{-1}=U^{s+1}$$
So applying the power series for log (which is just a polynomial since $U$ is unipotent), we obtain: $$B s \ln (U)B^{-1}=(s+1)\ln (U)$$
So if $\ln(U)$ is nonzero, it is an eigenvector of value $\frac{s+1}{s}$ of the conjugation linear map $\text{Ad}_B$. But the eigenvalues of $\text{Ad}_B$ are all roots of unity, since they are given by $\{\lambda_i \lambda_j^{-1}\}_{i,j}$. So $\ln(U)=0$, and $U=I$ since log and exp are bijective when restricted to uni/nilpotent matrices.
So now we know that $A$ and $B$ have finite order $m$, coprime to the integers $s$ and $s+1$. In particular, we now know that $m$ must be odd.
Within $GL_n(\mathbb{Q})$, we have the cyclic group generated by $A$, and within this, the finite subgroup generated by $A^s$. Conjugation by $B$ yields as isomorphism to the cyclic subgroup generated by $A^{s+1}$, so since these groups have equal order in the ambient cyclic group $\langle A \rangle$, they are thus equal, since a cyclic group has one subgroup of any given size. In particular, $A$ is in the subgroup generated by $A^s$, since $A^s$ and $A^{s+1}$ are.
Now that we know $A^s$ and $A^{s+1}$ generate the same cyclic subgroup as $A$, lets call it $G$, and note that $B$ normalises $G$, and acts by multiplication by $k$, where $k$ is the unique element of $\mathbb{Z}/m\mathbb{Z}$ with $ks=s+1$.
Now note that $k$ is the image of an additive generator of $\mathbb{Z}/m\mathbb{Z}$, is a unit, and has $(k-1)s=1$, so $k-1$ is also a unit. So to prove the final contradiction, it remains to prove the following:
If $m$ is odd, and integers $k$ and $k-1$ are both units in $(\mathbb{Z}/m\mathbb{Z})^*$, then the order of $k$ in $(\mathbb{Z}/m\mathbb{Z})^* $ doesn't divide $m$.
For this, consider the smallest prime $p$ dividing $m$ with exponent $e$. Then the automorphism group of $G$ breaks up into $\text{Aut}(\mathbb{Z}/p^e\mathbb{Z})\times \text{Aut}'$, and since $p$ was our smallest prime, we have $p-1$ is coprime to $m$. So if an element $x$ has order dividing $m$, its component in the automorphism group of $\mathbb{Z}/p^e\mathbb{Z}$ lies in the $p$ sylow of this group, which is precisely the image (of reduction mod $p^e$) of those units with $x-1$ divisible by $p$. Thus, we conclude that $x-1$ is divisible by the smallest prime dividing $m$, so in particular, it is not coprime to $m$, so can't be a unit, giving the lemma.