Bessel function integral and Mellin transform
Gradshteyn&Ryzhik 6.635.3 provides the following integral, with the usual constraints on $\nu,\alpha,\beta$, $$\int\limits_0^\infty \exp\left(-\frac{\alpha}{x}-\beta x\right)J_\nu(\gamma x)\frac{\mathrm{d}x}{x}= 2J_\nu\left(\sqrt{2\alpha(\sqrt{\beta^2+\gamma^2}-\beta)}\right) K_\nu\left(\sqrt{2\alpha(\sqrt{\beta^2+\gamma^2}+\beta)}\right).$$ The two previous equations give the integral $$\int\limits_0^\infty \exp\left(-\frac{\alpha}{x}\right)J_\nu(\beta x)\frac{\mathrm{d}x}{x}= 2J_\nu\left(\sqrt{2\alpha\beta}\right) K_\nu\left(\sqrt{2\alpha\beta}\right)$$ and the same integral with $J_\nu(...)$ replaced by $Y_\nu(...)$ on both sides of the equation (Bessel function of the first kind replaced by the second kind). The same simple substitution sometimes works and sometimes doesn't work in other integrals in Gradshteyn&Ryzhik. Is there a general rule as to when this simple substitution works, and in particular does it work for the first integral given above, 6.635.3? Alternatively, how can a computation of the above integral (with the Bessel function of the second kind) be approached? [I would prefer to understand this better than just to know what answer Mathematica gives. To save anyone the trouble, Wolfram definite integral balks at it.]
It's a more involved question, but is there a more interesting way to understand the connection between $\exp\left(-\frac{\alpha}{x}-\beta x\right)$ and the Bessel functions that is reflected by the integrals above and by Gradshteyn&Ryzhik 3.471.9, $$\int\limits_0^\infty x^{\nu-1}\exp\left(-\frac{\alpha}{x}-\beta x\right)\mathrm{d}x= 2\left(\frac{\alpha}{\beta}\right)^{\frac{\nu}{2}} K_\nu(2\sqrt{\alpha\beta})$$ than the bare statement that this integral is a Mellin transform?
[As may be obvious, I learned what was necessary to formulate the second question while formulating the first. This question is related to a question about quantum field theory on physics.SE. Thanks in advance.]
EDIT: For the first question, because of the definition $Y_\nu(z)=\frac{cos(\nu\pi) J_\nu(z)-J_{-\nu}(z)}{sin(\nu\pi)}$, for noninteger $\nu$, $|\arg(z)|<\pi$, and because of the identity $K_{-\nu}(z)=K_\nu(z)$, the noninteger case is embarrassingly obvious, then continuity of the Bessel functions w.r.t. order is enough (for me) for the integer case, $$\int\limits_0^\infty \exp\left(-\frac{\alpha}{x}-\beta x\right)Y_\nu(\gamma x)\frac{\mathrm{d}x}{x}= 2Y_\nu\left(\sqrt{2\alpha(\sqrt{\beta^2+\gamma^2}-\beta)}\right) K_\nu\left(\sqrt{2\alpha(\sqrt{\beta^2+\gamma^2}+\beta)}\right).$$ Those observations seem to be enough to give a reasonable idea of when it's straightforward to replace $J_\nu(...)$ by $Y_\nu(...)$ (not often, ultimately). Of course all such integral equivalences eventually derive from the differential equations and the boundary conditions satisfied by the Bessel functions of various orders. The second question, still open, clearly needs considerably more knowledge and, I take it, a more abstract mindset.
3.471.9
I will start with 3.471.9, since it is much simpler. The key ingredient
Slater theorem, saying that $$ \int_0^\infty t^{\alpha-1} f(t) g\left(\frac{x}{t}\right) \mathrm{d} t = \frac{1}{2 \pi i} \int_{\gamma - i \infty}^{\gamma + i \infty} \mathcal{M}_f(s+\alpha) \mathcal{M}_g(s) x^{-s} \mathrm{d} s = \mathcal{M}^{-1} \left\{ \mathcal{M}_f(s+\alpha) \mathcal{M}_g(s) \right\}(x) $$ where $\mathcal{M}_f(s) = \int_0^\infty t^{s-1} f(t) \mathrm{d} t$ is the Mellin transform of $f$, valid within some strip $\min < \Re(s) < \max $, and $\gamma$ is a real constant, such that $\gamma$ is within the strip of validity of the Mellin transform of $g$, and $\gamma + \Re(\alpha)$ in within the strip of the Mellin transform of $f$.
The proof is easy: $$ \begin{eqnarray} \int_0^\infty t^{\alpha-1} f(t) g\left(\frac{x}{t}\right) \mathrm{d} t &=& \int_0^\infty t^{\alpha-1} f(t) \left( \frac{1}{2 \pi i} \int_{\gamma - i \infty}^{\gamma+\infty} \mathcal{M}_g(s) x^{-s} t^s \mathrm{d} s \right) \mathrm{d} t \\ &=& \frac{1}{2 \pi i} \int_{\gamma - i \infty}^{\gamma+\infty} \left( \int_0^\infty t^{s+\alpha-1} f(t) \mathrm{d} t \right) \mathcal{M}_g(s) x^{-s} \mathrm{d} s \\ &=& \frac{1}{2 \pi i} \int_{\gamma - i \infty}^{\gamma+\infty} \mathcal{M}_f(s+\alpha) \mathcal{M}_g(s) x^{-s} \mathrm{d} s \end{eqnarray} $$
We now apply this theorem to 3.471.9, using Cahen-Mellin integral $\mathcal{M}_{\exp(-\bullet)}(s) = \Gamma(s) $: $$\begin{eqnarray} \int_0^\infty t^{\alpha-1} \exp\left(-y t\right) \exp\left( - \frac{x}{t}\right) \mathrm{d} t &=& \frac{1}{2 \pi i} \int_{\gamma - i \gamma}^{\gamma + i \infty} \Gamma(s+\alpha) \Gamma(s) y^{-\alpha -s} x^{-s} \mathrm{d}s \\ &=& y^{-\alpha} \left( \frac{1}{2 \pi i} \int_{\gamma - i \gamma}^{\gamma + i \infty} \Gamma(s+\alpha) \Gamma(s) \left( x y \right)^{-s} \mathrm{d}s \right) \\ &=& y^{-\alpha} \left( 2 (x y)^{\alpha/2} K_{\alpha}(2 \sqrt{x y} ) \right) \\ &=& 2 \left( \frac{x}{y} \right)^{\alpha/2} K_\alpha\left(2 \sqrt{x y}\right) \end{eqnarray} $$ where DMLF 10.43.19 was used.
6.635.3
A formula of kin is worked out in Bateman, Erdelyi et al, "Higher Transcendental Functions", chapter 7, section 7.7.6 on Macdonald's and Nochilson's formulas, formula (37). It says:
$$
\int_0^\infty \exp\left(-\frac{t}{2} - \frac{x^2+X^2}{2 t} \right) I_\nu \left( \frac{x X}{t} \right) \frac{\mathrm{d} t}{t} = \cases{2 I_\nu\left( x \right) K_\nu\left( X \right) & x < X \\ 2 I_\nu\left( X \right) K_\nu\left( x \right) & x > X }
$$
By doing an analytic continuation $x \to \mathrm{e}^{i \pi/2} x$ of the (the only possible) first branch, we get
$$
\int_0^\infty \exp\left(-\frac{t}{2} - \frac{X^2-x^2}{2 t} \right) J_\nu \left( \frac{x X}{t} \right) \frac{\mathrm{d} t}{t} = 2 J_\nu(x) K_\nu(X)
$$
Now, performing reparameterization $X = \sqrt{2 \alpha \left( \sqrt{\beta^2 + \gamma^2}+ \beta \right)}$, $x = \sqrt{2 \alpha \left( \sqrt{\beta^2 + \gamma^2} - \beta \right)}$, accompanied by a change of variables $t \to \left(2 \alpha t\right)^{-1}$ we arrive at 6.635.3.
Derivation of the point of departure formula relies on two auxiliary results: $$ \int_0^\infty J_{\nu}(x u) J_{\nu}(X u) \exp\left(-\frac{t u^2}{2}\right) u \mathrm{d} u = \frac{1}{t} \exp\left(-\frac{x^2+X^2}{2t}\right) I_\nu\left( \frac{x X}{t}\right) $$ which is obtained by expanding the exponential in series and integrating term-wise. Then, using the above integral representation in the formula under consideration, and carrying out simple integration with respect to $t$, we get $$ \int_0^\infty \frac{\mathrm{e}^{-t/2}}{t} \exp\left(-\frac{x^2+X^2}{2t}\right) I_\nu\left( \frac{x X}{t}\right) \mathrm{d} t = \int_0^\infty J_\nu(x u) J_\nu(X u) \frac{2 u \mathrm{d} u}{1+u^2} $$ The resulting integral is of Sonine-Gegenabuer-type, and equals to $2 I_\nu( \min(x,X)) K_\nu(\max(x,X))$.