How to prove that $a_{n}$ must be of the form $a^2+b^2$?
let $a_{1}=1,a_{2}=2,a_{3}=5$,and $$a_{n}=3a_{n-1}a_{n-2}-a_{n-3}$$
show that $a_{n}=a^2+b^2,a,b\in N$
while $a_{1}=0^2+1^2,a_{2}=2=1^2+1^2,a_{3}=5=2^2+1^2,a_{4}=29=5^2+2^2,a_{5}=433=17^2+12^2$ and so on
Solution 1:
There are many such properties involving the Markov numbers, see also OEIS, which together make up a tree rather than a single sequence. The numbers are any of the variables (required to be positive integers) in $$ x^2 + y^2 + z^2 = 3 x y z. $$ Your sequence $1,2,5,29,433,37666,48928105,\ldots$ is the fastest growing branch in the tree, in that you are replacing (jumping out) the smallest number in the current triple. The slowest growing branch is the Fibonacci numbers with odd index $1,2,5,13,34,89,233,610,1597,\ldots,$ in that the middle-sized number in the triple is replaced. See jpeg far below. Oh, if you replace the largest number in the triple, you jump back closer to the root. Travel within the tree is one of the more serious uses of Vieta Jumping. The other serious use, with recent articles in the AMS Bulletin by Kontorovich and by E. Fuchs, is Apollonian Gaskets. I would like to put, at a minimum, Wikipedia links at Vieta Jumping and Apollonian Gaskets pointing to each other, but I have had little luck with Wikipedia and the relationship has not been noticed in print.
Your property (sum of two squares) is true for all Markov numbers.
The most striking property, for me, is that if $m$ is a Markov number, then $$9m^2 - 4 $$ is the sum of two squares.
The shortest proof that $m$ itself is the sum of two squares is probably this: in $$ x^2 + y^2 + z^2 = 3 x y z $$ the initial solution is $(1,1,1).$ So the greatest common divisor is $1.$ There is a proof by induction that $\gcd (x,y,z)=1$ whenever $(x,y,z)$ is such a Markov triple; we are going to move to a new triple with $z' = 3 x y - z.$ If there is any prime $p$ that divides all three $3xy-z,x,y$ Then it also divides $(3xy) -(3xy-z)=z,$ causing $\gcd (x,y,z) \neq 1,$ violating the induction assumption.
Now, $$ 3xyz - z^2 = x^2 + y^2, $$ $$ z(3xy - z) = x^2 + y^2. $$ IF there were any prime $q \equiv 3 \pmod 4,$ such that $q |z,$ it would also be true that $q|(x^2 + y^2).$ However, it is a standard fact that if such $q|(x^2 + y^2),$ then $q|x$ and $q|y.$ So, if this happened, $q$ would divide all three, violating $\gcd (x,y,z)=1.$ Note that the same trick works to show that $z$ cannot be divisible by 4, otherwise all three numbers would be even. As a result, any Markov number $m$ is not divisible by 4 or by any prime $q \equiv 3 \pmod 4,$ so we can write it primitively as the sum of two squares, $m = u^2 + v^2$ with $\gcd(u,v)=1.$
Proof of standard fact: (it seems DonAntonio has this; oh, well...) If $q \equiv 3 \pmod 4$ and $$ x^2 + y^2 \equiv 0 \pmod q, $$ ASSUME $y \neq 0 \pmod q.$ Then $$ x^2 \equiv - y^2 \pmod q, $$ but $y$ has a multiplicative inverse in this field, and $$ x^2 / y^2 \equiv - 1 \pmod q, $$ $$ (x / y)^2 \equiv - 1 \pmod q, $$ which violates the fact that $-1$ is not a quadratic residue for primes $q \equiv 3 \pmod 4.$
=================
This is page 19 from Cusick and Flahive
=============
Solution 2:
Hopefully you already know the very nice theorem from number theory that states that a natural number can be written as the sum of two squares iff every prime divisor of the number which is equal to $\,3\pmod 4\,$ appears to an even power (primes which are $\;1\pmod 4\;$ or $\;2\;$ make no problems).
You can now use some induction, proving/pointing out that:
-- If $\,p\,,\,q\,$ are two primes $\,p,q=1\pmod 4\implies pq=1\pmod 4\,$
-- The only primes dividing $\,a_n\,,\,\,n\ge 2\;$ , are $\,1\pmod 4\;$ or $\,2\,$