How can I convert an axis-angle representation to a Euler angle representation

In the following, we adjust $\delta$ in the range specified by $(2)$, $\sigma_B=\pm1$, and $\sigma_C=\pm1$ until we get the required rotation.

Angles $\alpha$, $\beta$, and $\gamma$

Let $\alpha$, $\beta$, and $\gamma$ be the angles from the $X$, $Y$, and $Z$ axes to the axis of rotation, $A$. These three angles are related by the Pythagorean Theorem: $$ \cos^2(\alpha)+\cos^2(\beta)+\cos^2(\gamma)=1\tag{1} $$ These angles, along with the angle $\delta$, will be assumed to be in the range $[0,\pi]$.

Angles $\rho_X$, $\rho_Y$, and $\rho_Z$

Consider the diagram on the sphere

$\hspace{3.2cm}$

For each value of $\delta$ so that $$ \sin^2(\delta)\ge\max\left(\cos^2(\alpha),\cos^2(\gamma)\right)\tag{2} $$ the circle of radius $\delta$ about $Y$ intersects both the circle of radius $\alpha$ about $X$ and the circle of radius $\gamma$ about $Z$ at one or two points.

Using the Spherical Law of Cosines, we can compute $$ \angle AXY=2\tan^{-1}\left(\frac{\cos(\gamma)}{\sin(\alpha)+\cos(\beta)}\right)\tag{3} $$ For each $\alpha$ and $\delta$, there are one or two solutions for $B$. To account for this, let $\sigma_B=\pm1$. Then $$ \begin{align} \rho_X&=\angle AXY-\sigma_B\cos^{-1}\left(\frac{\cos(\delta)}{\sin(\alpha)}\right)\tag{4}\\ \angle YBX&=\pi-\sigma_B\cos^{-1}(\cot(\alpha)\cot(\delta))\tag{5}\\ \angle XYB&=\sigma_B\cos^{-1}\left(\frac{\cos(\alpha)}{\sin(\delta)}\right)\tag{6} \end{align} $$ Using the Spherical Law of Cosines, we can compute $$ \angle YZA=2\tan^{-1}\left(\frac{\cos(\alpha)}{\sin(\gamma)+\cos(\beta)}\right)\tag{7} $$ For each $\gamma$ and $\delta$, there are one or two solutions for $C$. To account for this, let $\sigma_C=\pm1$. Then $$ \begin{align} \rho_Z&=\angle YZA-\sigma_C\cos^{-1}\left(\frac{\cos(\delta)}{\sin(\gamma)}\right)\tag{8}\\ \angle ZCY&=\pi-\sigma_C\cos^{-1}(\cot(\gamma)\cot(\delta))\tag{9}\\ \angle CYZ&=\sigma_C\cos^{-1}\left(\frac{\cos(\gamma)}{\sin(\delta)}\right)\tag{10} \end{align} $$ For consistency, define $\sigma_A=\mathrm{sgn}(\cos(\beta))$. Since $\angle XYZ=\frac\pi2$, $$ \begin{align} \rho_Y&=\frac\pi2-\angle XYB-\angle CYZ\tag{11}\\ \angle XAZ&=\pi-\sigma_A\cos^{-1}(\cot(\alpha)\cot(\gamma))\tag{12} \end{align} $$

Angle of Rotation

The rotations $\rho_X$, $\rho_Y$, and $\rho_Z$ computed above will fix the axis $A$. Accounting for parallel transport, the rotation about the axis $A$ is equal to the total geodesic curvature $$ \rho_X\cos(\alpha)+\rho_Y\cos(\delta)+\rho_Z\cos(\gamma)\tag{13} $$ minus the area of the light red deltoid region (since it is traversed clockwise).

Using Girard's Theorem, the areas of the purple triangles are $$ \begin{align} |\triangle A|&=\angle ZXA+\angle AZX+\angle XAZ-\pi\\ |\triangle B|&=\angle BXY+\angle XYB+\angle YBX-\pi\\ |\triangle C|&=\angle YZC+\angle CYZ+\angle ZCY-\pi \end{align}\tag{14} $$ The areas of the white sectors are $$ \begin{align} |\unicode{x2AA6} X|&=\rho_X\,(1-\cos(\alpha))\\ |\unicode{x2AA6} Y|&=\rho_Y\,(1-\cos(\delta))\\ |\unicode{x2AA6} Z|&=\rho_Z\,(1-\cos(\gamma)) \end{align}\tag{15} $$ The area of the light red deltoid is $$ \frac\pi2-|\triangle A|-|\triangle B|-|\triangle C|-|\unicode{x2AA6} X|-|\unicode{x2AA6} Y|-|\unicode{x2AA6} Z|\tag{16} $$ Note that depending on the signs of $\sigma_B$ and $\sigma_C$, the light red deltoid may have one or two lunes attached.

Subtracting $(16)$ from $(13)$ yields a rotation of $$ |\triangle A|+|\triangle B|+|\triangle C|+\rho_X+\rho_Y+\rho_Z-\frac\pi2\tag{17} $$ Four applications of Girard's Theorem reduces $(17)$ to $$ \angle XBY+\angle YCZ+\angle ZAX-2\pi\tag{18} $$

Example:

Suppose $\alpha=\gamma=\frac\pi3$ and $\beta=\frac\pi4$. Note that $\cos^2(\alpha)+\cos^2(\beta)+\cos^2(\gamma)=1$.

By $(2)$, we can choose any $\delta$ so that $\sin^2(\delta)\ge\frac14$. Let $\delta=\frac\pi6$.

$(3)$: $\angle AXY=0.615479708670388$

$(4)$: $\rho_X=0.615479708670388$

$(5)$: $\angle YBX=\pi$

$(6)$: $\angle XYB=0$

$(7)$: $\angle YZA=0.615479708670388$

$(8)$: $\rho_Z=0.615479708670388$

$(9)$: $\angle ZCY=\pi$

$(10)$: $\angle CYZ=0$

$(11)$: $\rho_Y=\frac\pi2$

$(12)$: $\angle XAZ=1.91063323624902$

$(18)$ says that the rotation is $1.91063323624902$.

Note that the angles specified in your question do not satisfy $(1)$: $$ \cos^2(\alpha)+\cos^2(\beta)+\cos^2(\gamma)=2.25134287511099\ne1 $$ There is no point that is $-17^\circ$, $+40^\circ$, and $-30^\circ$ from the coordinate axes.