Laplace, Legendre, Fourier, Hankel, Mellin, Hilbert, Borel, Z...: unified treatment of transforms?
I understand "transform methods" as recipes, but beyond this they are a big mystery to me.
There are two aspects of them I find bewildering.
One is the sheer number of them. Is there a unified framework that includes all these transforms as special cases?
The second one is heuristic: what would lead anyone to discover such a transform in the course of solving a problem?
(My hope is to find a unified treatment of the subject that simultaneously addresses both of these questions.)
Solution 1:
The essential idea of many transforms is to change the basis in the space of functions with the hope that in the new basis the problem will simplify.
Let me give a finite-dimensional example. Suppose we have a $2\times2$ matrix $A$ and we want to compute $A^{1000}$. Direct approach would not be very wise. However, if we first diagonalize $A$ as $PA_dP^{-1}$ (i.e. rotate the basis by $P$), the calculation becomes much easier: the answer is given by $PA_d^{1000}P^{-1}$ and computing powers of diagonal matrix is a very simple task.
A somewhat analogous infinite-dimensional example would be the solution of the heat equation $u_t=u_{xx}$ using Fourier transform $u(x,t)\rightarrow \hat{u}(\omega,t)$. The point is that in the Fourier basis the operator $\partial_{xx}$ becomes diagonal: it simply multiplies $\hat{u}(\omega,t)$ by $-\omega^2$. Therefore, in the new basis, our partial differential equation simplifies and becomes ordinary differential equation.
In general, the existence of a transform adapted to a particular problem is related to its symmetry. The new basis functions are chosen to be eigenfunctions of the symmetry generators. For instance, in the above PDE example we had translation symmetry with the generator $T=-i\partial_x$. In the same way, e.g. Mellin transform is related to scaling symmetry, etc.
Solution 2:
There are various ways one can find connections between the Fourier, Laplace, Z, Mellin, Legendre, et al transforms.
The rationale is to change the representation of a problem (e.g a differential equation) in order to simplify and thus solve easier.
For example, as stated before, most of the transforms stem directly from trying to solve specific Sturm-Liouville problems (e.g Fourier, Laplace etc..) so in this sense the different conditions of the problem specify a transform to use. For instance by diagonalising (or de-coupling in physics parlance) the (differential) operator of a system description. This process then defines the kernel of the integral transform, which itself describes the type of integral transform (e.g Laplace, Fourier and so on..).
The way this works and a certain transform is applied is the following:
Given that the differential operator $d/dx$ has as eigen-vector (eigen-function) the exponential function $e^x$ (i.e $de^x/dx = e^x$), it is natural to express the (solution of) a certain differential equations in terms of (or as if expanded) the eigen-functions of the (differential) operator. This then derives naturaly some of the known transforms, like Laplace and Fourier.
An eigen-vector/eigen-function of an operator is that function which is left as is by the application of the operator. Or in other words these functions on which the operator has the simplest effect. It is natural and easier to express solutions w.r.t these functions as they will have the simplest interaction with the (differential) operator which describes the system under study
Now one sees how analysing the (differential) operator in terms of its own eigen-functions, simplifies (the solution of) the problem. These eigen-functions define the integral transform (its kernel) and this field of study (of which integral transforms is a part) is refered as Spectral Theory (of Operators).
For example, every integral transform is a linear operator, since the integral is a linear operator, and in fact if the kernel is allowed to be a generalized function then all linear operators are integral transforms (a properly formulated version of this statement is the Schwartz kernel theorem).
from wikipedia, Integral Transform
Another way to acquire a unified view of the transforms is to think as one transform that changes with respect to change of the underlying domain of the problem. For example the Mellin transform can be seen as the Fourier transform when doing a change of variables from $x \to log(y)$. The Laplace transform can be seen as the Fourier transform on a line instead of on a circle by change of variables $i\omega \to s$ or $e^{i\omega} \to e^s$
For discrete applications, the Discrete Fourier transform when changing the variables $e^{i\omega} \to e^{i\omega_k}=e^{i\omega_0k}=e^{2{\pi}k/n}$ or $\omega_k \to 2{\pi}k/n$ for $k=0,..,n-1$ (sampling the unit circle at regular angles, or sampling the frequency at regular $n$ intervals)
The Z transform can be seen either as the dicrete analog of the Laplace transform or as discrete fourier transform extended beyond the unit circle i.e $e^{i\omega_0k} \to z^k$
The Legendre transform can be seen as the Fourier transform on an idempotent semiring instead of a ring
Apart from that, most transforms have a direct geometrical meaning and also a group-theoretic meaning and characterization
If you want to generalise more, one can go into the non-linear domain and generalise the Fourier transform, one such generalisation is the Inverse Scattering Transform which is used to solve non-linear differential equations
Again, the rationale, is to simplify the representation of the problem and/or express it in other known terms. Only in this case (at least for IST) the problem is not of the Sturm-Liouville type but rather of the more general Riemann-Hilbert type
Integral Transforms
Parseval's Theorem for Fourier and Legendre Transforms
Solution 3:
The closest thing to a general theory that leads to MANY of the above (though not all) is Sturm–Liouville theory. Basically, many of these transforms have come about from the study of physical phenomena via linear differential equations, where, as previous answers have noted, specific transforms diagonalize the differential operator. It turns out that MANY physical phenomena of interest obey second order differential equations of the Sturm-Liouville type. The same logic really applies for other differential equations (or difference equations in the case of the z-transform). Once you know what functions fundamentally solve a linear differential equation, you want to make up more functions that solve the problem by an integral or sum over these fundamental solutions; this idea leads to many of the transform above. Spectral theory of operators and ideas from Hilbert spaces generalize this for higher order operators. Each one of these equation types naturally appear in physical models of the world. I'll outline some of the differential equations I mean, the associated transform, and the physical applications in which they came about.
Linear constant coefficient ODEs with zero boundary conditions before t=0. The function $e^{st}$ solves these for some values of $s$. Superposing these leads to the Laplace transform. Mellin is closely related. Equations model kinematics, circuits.
Linear constant coefficient ODEs or PDEs in unbounded domains. Plane waves $e^{jkr}$ in multiple dimensions solve these for a continuum of $k$ values. Superposing these leads to the (multidimensional) Fourier transform. In bounded domains some of these dimensions reduce to summations instead of integrals. In certain cylindrical symmetry the solutions are Bessel and Hankel functions, reducing to the Hankel transform. Equations model wave mechanics, heat conduction, potential theory, etc.
Linear constant coefficient difference equations in the variable $n$. The function $z^n$ will solve these equations for some particular values of $z$. Superposing these leads to the z transform. Linear recurrences appear in the math of sequences and series, digital filters, generating functions in probability.
Some of the methods you mention are not from this family of naturally arising from differential equations, namely the Legendre and Hilbert transforms. The Hilbert has a similar form of a linear integral transform, and could be considered unified with the rest. The Legendre transform is something else entirely however.
Solution 4:
There is actually a unified treatment of transform theory. This is what chapter seven of Keener's book is about.
There is a relationship between Linear differential equations, spectrum, Green functions, contour integration, Dirac Deltas, and Transforms.
The story goes like this. Given a linear operator (differential) $L$, form the operator $L - \lambda = 0$, with given boundary conditions, find its Green function $G(x, \xi, \lambda)$. Then
\begin{equation} \delta(x - \xi) = -\frac{1}{2 \pi \mathrm{i}} \int_{C_{\infty}} G(x, \xi , \lambda) \, d \lambda, \end{equation}
where the contour $C_{\infty}$ is a circle having all the spectrum of $L$ inside.
This particular Dirac delta representation is the product of the direct and inverse transforms.
Here are a few examples:
-
Sine transform pair: Functions continuously differentiable in $[0,1]$. \begin{equation} Lu = - u'' \quad , \quad u(0)=u(1)=0. \end{equation}
result:
\begin{eqnarray*} U_k &=& 2 \int_0^1 d \xi \sin( k \pi \xi) u( \xi ) \\ u(x) &=& \sum_{k=1}^{\infty} U_k \sin k \pi x. \end{eqnarray*}
-
Cosine transform pair: \begin{equation} Lu = - u'' \quad , \quad u'(0)=u'(1)=0. \end{equation}
result:
\begin{eqnarray*} U_k &=& 2 \int_0^1 d \xi \cos( k \pi \xi) u( \xi ) \\ u(x) &=& \sum_{k=1}^{\infty} U_k \cos k \pi x. \end{eqnarray*}
-
Sine transform integral
Functions in $L^2[0, \infty)$. \begin{equation} Lu = - u'' \quad , \quad u(0)=0 \quad , \quad \lim_{x \to \infty} u(x) = 0. \end{equation}
result:
\begin{eqnarray*} U(\mu) &=& \frac{2}{\pi} \int_0^{\infty} dx u(x) \sin \mu x \\ u(\xi) &=& \ \int_0^{\infty} d \mu U(\mu) \sin \mu x \\ \end{eqnarray*}
-
cosine transform integral \begin{equation} Lu = - u'' \quad , \quad u'(0)=0 \quad , \quad \lim_{x \to \infty} u(x) = 0. \end{equation}
result:
\begin{eqnarray*} U(\mu) &=& \frac{2}{\pi} \int_0^{\infty} dx u(x) \cos \mu x \\ u(\xi) &=& \ \int_0^{\infty} d \mu U(\mu) \cos \mu x \\ \end{eqnarray*}
-
Fourier transform Functions in $L^2(-\infty, \infty)$. \begin{equation} Lu = - u'' \quad , \quad , \quad \lim_{x \to \pm \infty} u(x) = 0. \end{equation}
result:
\begin{eqnarray*} U(\mu) &=& \int_{-\infty}^{\infty} dx \; u(x) \; \mathrm{e}^{\mathrm{i} \mu \xi} \\ u(\xi) &=& \frac{1}{2 \pi} \int_{-\infty}^{\infty} U(\mu) \mathrm{e}^{-\mathrm{i} \mu \xi} d \mu. \end{eqnarray*}
The list follows: Mellin, Hankel, etc. Keener shows the linear operators with boundary conditions for these.
I do not know which operator and boundary conditions generate the Laplace transform. I opened this question in StackExchange Mathematics for this particular problem.
I am writing some notes about this matter here.
UPDATE: I solved the problem of connecting the Laplace transform to an ODE with boundary conditions. Please see here
Thanks.