What is the Maximum Area of a Quadrilateral with sides of length a,b,c,d (in sequence).

Solution 1:

You will find that the optimal solution is for the opposite angles to add to $\pi$ and, thus, for the quadrilateral to be concyclic (inscribed in a circle).

Solution 2:

Notice that on the right hand side of Bretschneider's formula, everything is constant except $\alpha$ and $\gamma$. So the area is maximized if and only if $\cos^2 (\frac{\alpha+\gamma}{2})$ is minimized. But $\alpha$ and $\gamma$ are not independent. From the figure one can see that $\alpha$ and $\gamma$ are constrained by applying the law of cosines to two of the triangles in the picture: $$(*) \qquad b^2 + c^2 - 2bc \cos(\gamma) = a^2 + d^2 - 2ad \cos(\alpha) $$ So now you have a constrained optimization problem, and you can try to solve it using Lagrange's method: find the minimum of $\cos^2 (\frac{\alpha+\gamma}{2})$ subject to the constraint $(*)$ (and restricted to $\alpha,\gamma \in [0,\pi]$).

You might be able to simplify the problem even further using some symmetry. For example: if you restrict to $\alpha,\gamma \in [0,\pi/2]$, so that $\cos(\alpha+\gamma) \ge 0$, then $\cos^2(\frac{\alpha+\gamma}{2})$ is minimized if and only if $\cos(\frac{\alpha+\gamma}{2})$ is also minimized.