How to find minimum value of : $\frac{7x^2 - 2xy + 3y^2}{x^2 - y^2}$?
Solution 1:
You had the right-ish idea with your substitution, however you only found values on the unit circle. In a slightly easier form, consider the substitution
$$\begin{cases}x = r\cosh \tau \\ y = r\sinh\tau \end{cases} \implies f(\tau) = 7\cosh^2\tau-\sinh 2\tau+3\sinh^2\tau$$
where the $r$'s cancel. Then
$$f'(\tau) = 7\sinh 2\tau - 2 \cosh 2\tau+3 \sinh 2\tau = 0 \implies \tanh 2\tau = \frac{1}{5}$$
This makes an easier back substitution with the original function in terms of $2\tau$'s
$$f(\tau) = 5\cosh 2\tau - \sinh 2\tau + 2 $$
Then use $\tanh^2z + \operatorname{sech}^2z = 1$ to obtain
$$\begin{cases}\cosh 2\tau = \frac{1}{\sqrt{1-\frac{1}{5}^2}} = \frac{5}{\sqrt{24}} \\ \sinh 2\tau = \frac{1}{\sqrt{24}}\end{cases} \implies f(\tau) \geq \sqrt{24} + 2$$