Optimization of the area of a cross inscribed in a circle

Without loss of generality we may assume that the radius is $1$: we can scale area by $r^2$ later.

By the formula you quoted, the area of the triangle enclosed by the two lines that form the angle $\theta$ is $\frac{1}{2}\sin\theta$. The area covered by one of the two full arms of the cross is therefore $4$ times this, which is $2\sin\theta$.

Double this to get the sum $4\sin\theta$ of the areas covered by the two full arms. Unfortunately, this sum counts the area of the middle square twice. So we will need to subtract the area of that square.

Note that by trigonometry, the horizontal segment at the top of the cross has length $2\sin(\theta/2)$. Thus the middle square has area $4\sin^2(\theta/2)$. Using the trigonometric identity $\cos 2\phi=1-2\sin^2\phi$, we find that the middle square has area $2-2\cos \theta$.

So the area of the cross is $4\sin\theta +2\cos\theta-2$. Maximizing should be straightforward.

Remarks: $1.$ We don't really need calculus. Look at the equivalent problem of maximizing $4\sin2\theta+2\cos\theta$. Rewrite this as $$2\sqrt{5}\left(\frac{2}{\sqrt{5}}\sin \theta+\frac{1}{\sqrt{5}}\cos\theta\right),$$ and let $\psi$ be the angle whose cosine is $2/\sqrt{5}$ and whose sine is $1/\sqrt{5}$. Then our expression becomes $2\sqrt{5}\sin(\theta+\psi)$. The maximum possible value of the sine function is $1$. So the maximum area is $2\sqrt{5}-2$.

$2.$ We can use the above calculation to answer your question about $x$. Alternately, set the derivative equal to $0$, as you did. Manipulation will yield an explicit expression for the root.

$3.$ At a certain stage you were maximizing $8x\sqrt{r^2-x^2}-4r^2+4x^2$. Let $x=r\sin t$. We want to maximize $8r^2\sin t\cos t-4r^2+4r^2\sin^2 t$. Forget about the $r^2$ part, it is a constant multiplier. Now simplify to $8\sin t\cos t-4\cos^2 t$, differentiate. (Using double angle identities to simplify first is a good idea.) We can think of this as just a technical device to ensure we end up with a simple equation.