Real matrix $A_{3\times 3}$ such that $\operatorname{tr(}A)=0$ and $A^2+A^T=I$?
Solution 1:
Let $a,b,c$ be the roots of the characteristic polynomial of $A$ so that $\det (xI-A)=(x-a)(x-b)(x-c)$ with $a,b,c \in \mathbb{C}$. Since $tr(A)=0$, we have $a+b+c=0$. Since $tr(A^2)=3$ as you found, we have $a^2+b^2+c^2=3$. Therefore $$2ab+2ac+2bc=(a+b+c)^2-(a^2+b^2+c^2)=-3.$$ Moreover, it follows that $\det(A^2)=\det(I-A^T)=\det(I-A)= (1-a)(1-b)(1-c)=1-3/2-abc=-0.5-abc$. On the other hand $\det(A^2)=\det(A)^2=(abc)^2$. So for $y=abc$ we have the equation $y^2+y+1/2=0$ which does not have real roots. We have proved that such a real matrix $A$ does not exist.
Solution 2:
If $A$ exists, then $A^\ast=A^T=I-A^2$ is a polynomial in $A$. Hence $A$ is normal and $A,A^T$ are simultaneously unitarily diagonalisable over $\mathbb C$. It follows that for each eigenvalue $\lambda$ of $A$, we have $$ \lambda^2+\bar{\lambda}=1.\tag{$\ast$} $$
Now, if all eigenvalues of $A$ are real, each of them must be equal to $\frac12(-1\pm\sqrt{5})$, but then $\operatorname{tr}(A)\ne0$.
Therefore $A$ must have some non-real eigenvalues. Since $A$ is real and traceless, its three eigenvalues are $\lambda_1=x+iy,\ \lambda_2=x-iy$ and $\lambda_3=-2x$ for some real numbers $y\ne0$ and $x$. By inspecting the imaginary part of $(\ast)$ for $\lambda_1$, we get $2ixy-iy=0$. Hence $2x=1$. But then $\lambda_3=-2x=-1$ would violate $(\ast)$. So, we conclude that $A$ does not exist.
Solution 3:
We have $A^T = I-A^2$ , then $A = (A^T)^T = I^T - (A^2)^T = I - (A^T)^2 = I - (I-A^2)^2$.
Therefore $A^4 - 2A^2 + A = 0$. SO $A(A-I)(A^2+A-I) = 0$
Let's call $a,b,c$ our eigenvalues. We have $a+b+c = 0$ and $a,b,c \in \{ 0,1 , \frac{-1+\sqrt{5}}{2} , \frac{-1-\sqrt{5}}{2} \}$.
The only possibility is $\{a,b,c \} = \{ 1, \frac{-1+\sqrt{5}}{2} , \frac{-1-\sqrt{5}}{2} \}$.
But $a^2 + b^2 + c^2 = tr(A^2) = 3- tr(A^T) = 3$ , which is absurd.