Problem in defining a trigonometric equation
Solution 1:
Let's say the center of the circle with radius $r$ is $(x_P, y_P)$ in Cartesian coordinates. Each coordinate is a sum of two parts related to point $(x_\phi, y_\phi)$, and also part of the triangle with angle $\beta$ and hypotenuse $r+r_k$:
$$ \begin{align*} x_P &= x_\phi + r \sin \alpha = (r+r_k) \cos \beta \\ y_P &= y_\phi + r \cos \alpha = (r+r_k) \sin \beta \end{align*} $$
We can then rearrange this to a set of $2$ linear equations in $2$ unknowns $r$ and $r_k$, and solve.
$$ \begin{align*} r (\cos \beta - \sin \alpha) + r_k \cos \beta &= x_\phi \\ r (\sin \beta - \cos \alpha) + r_k \sin \beta &= y_\phi \end{align*} $$
$$ \det \left( \begin{array}{cc} \cos \beta -\sin \alpha & \cos \beta \\ \sin \beta -\cos \alpha & \sin \beta \end{array} \right) = \det \left( \begin{array}{cc} -\sin \alpha & \cos \beta \\ -\cos \alpha & \sin \beta \end{array} \right) = -\sin \alpha \sin \beta + \cos \alpha \cos \beta = \cos(\alpha+\beta) $$
$$ \left( \begin{array}{cc} \cos \beta -\sin \alpha & \cos \beta \\ \sin \beta -\cos \alpha & \sin \beta \end{array} \right)^{-1} = \frac{1}{\cos(\alpha+\beta)} \left( \begin{array}{cc} \sin \beta & -\cos \beta \\ -\sin \beta + \cos \alpha & \cos \beta - \sin \alpha \end{array} \right) $$
$$ \begin{align*} r &= \frac{x_\phi \sin \beta - y_\phi \cos \beta}{\cos(\alpha+\beta)} \\ r_k &= \frac{x_\phi(\cos \alpha - \sin \beta) + y_\phi(\cos \beta - \sin \alpha)}{\cos(\alpha+\beta)} \end{align*} $$
Solution 2:
Here's an alternative proof with the same result as the method of simultaneous equations (see other answer).
Consider the triangle $\triangle ABC$. Side $AB$ opposite angle $\theta$ has length $x_1 = x_\varphi - y_\varphi \cot\beta.$ Side $BC$ opposite angle $\beta$ has length $r.$ By the law of sines, $$ \frac{x_1}{\sin\theta} = \frac{r}{\sin\beta}.$$ But $\theta = \frac\pi2 - \alpha - \beta$ and $\sin\theta = \cos(\alpha + \beta),$ so $$ \frac{x_\varphi - y_\varphi \cot\beta}{\cos(\alpha + \beta)} = \frac{r}{\sin\beta},$$ whence $$ r = \frac{x_\varphi \sin\beta - y_\varphi \cos\beta}{\cos(\alpha + \beta)}.$$
Now consider the triangle $\triangle OCD$. We have side $x_2$ opposite angle $\theta$ and side $r_k + r$ opposite angle $\psi = \frac\pi2 + \alpha.$ By the law of sines, $$ \frac{x_2}{\sin\theta} = \frac{r_k + r}{\sin\psi}.$$ But $\sin\psi = \cos\alpha$ and $x_2 = x_\varphi - y_\varphi \tan\alpha,$ so $$ \frac{x_\varphi - y_\varphi \tan\alpha}{\cos(\alpha + \beta)} = \frac{r_k + r}{\cos\alpha},$$ whence $$ r_k + r = \frac{x_\varphi \cos\alpha - y_\varphi \sin\alpha}{\cos(\alpha + \beta)}. $$