Intuition behind convolution
I always wondered about the idea behind convolution. I get what the definition of the convolution does (and I saw all the animations), but what I don't understand is how it relates to so many topics in physics. It seems to me that it is not really an intuitive concept. I guess my question is - what were the ideas, thoughts of the guy that discovered convolution?
About two years ago I read a blog post about convolution that explains it pretty intuitively, but I forgot the name of the website.
Well, here I want to give you just an intuition not a whole proof and abstract notions (convolution products are defined in abstract algebra), so it is a good way to start with Fourier transform. Let's begin with this theorem
$\hat{f}.\hat{g}=\mathcal{F}[\frac{1}{2\pi}f*g]$
where $\hat{f}$ and $\hat{g}$ are the Fourier transform of the functions $f$ and $g$ respectively and $\mathcal{F}$ is the Fourier transform symbol.
$\hat{f}=\frac{1}{2\pi}\int_{-\infty}^{\infty}f(x)e^{i\omega x}dx$
$\hat{g}=\frac{1}{2\pi}\int_{-\infty}^{\infty}g(x)e^{i\omega x}dx$
We just want to know "Where did convolution formula come from?" So we just need to accept the identity above and extend it to see what happens?
$\hat{f}.\hat{g}=\frac{1}{2\pi}\int_{-\infty}^{\infty}f(x)e^{i\omega x}dx .\frac{1}{2\pi}\int_{-\infty}^{\infty}g(x)e^{i\omega x}dx$
$=\frac{1}{4\pi^2}\int_{-\infty}^{\infty}f(x)e^{i\omega x}dx\int_{-\infty}^{\infty}g(x)e^{i\omega x}dx$
Now if we extend the right hand side of the first equation mentioned, we obtain:
$\frac{1}{2\pi}(\frac{1}{2\pi}\int_{-\infty}^{\infty}(f*g)e^{i\omega x}dx) = \frac{1}{4\pi^2}\int_{-\infty}^{\infty}(f*g)e^{i\omega x}dx$
In the equality the term $\frac{1}{4\pi^2}$ is omitted from both side and we obtain:
$\int_{-\infty}^{\infty}f(x)e^{i\omega x}dx\int_{-\infty}^{\infty}g(x)e^{i\omega x}dx=\int_{-\infty}^{\infty}(f*g)e^{i\omega x}dx$
Now we can ask "How can we have the product of two different integrals in only one integral?" (Note that by accepting the convolution and the theorems of Fourier transform we got here!).
Let's make it easier by writing $\int_{\Omega}\phi d\mu(x) \int_{\Omega}\psi d\mu(x)$ and how can we make them into one integral. We know from real analysis that we can write the integral as:
$\int \phi d\mu = \sum_{1}^{n}a_j\mu(E_j)$ and $\int \psi d\mu = \sum_{1}^{n}b_j\mu(E_j)$
where $E_j$ are elementary sets of the domain. For easier manipulation (this is just an intuition not a proof!) we suppose that for all $E_j$ we have $\phi(E_j)=a_j$ and $\psi(E_j)=b_j$ and with assumption that $\mu(E_j)=\mu(E_i)=1$ for all $i$ and $j$ ($\mu$ here is the measure of the sets $E_j$) then we can write
$\int \phi d\mu\int \psi d\mu = \sum_{1}^{n}a_j\mu(E_j)\sum_{1}^{n}b_j\mu(E_j)=$
$(a_1+a_2+\dots+a_n)(b_1+b_2+\dots+b_n)=$ $a_1(b_1+b_2+\dots+b_n)+a_2(b_1+b_2+\dots+b_n)+\dots+a_n(b_1+b_2+\dots+b_n)$
For the first term $a_1(b_1+b_2+\dots+b_n)$, we see that $a_1$ is multiplied in all values of the function $g$ and this process is repeated for other terms. This is like translating the function $f$ (compute its values in $E_j$s) and then multiply it by function of $g$ then sum them (integrate the function $f(x-t)g(x)$ over the translation domain).
Now we can understand the meaning of convolution: it is a way to take the product of two integrals into only one integral, and for doing that, you need to translate one function (like $f$) then multiply it by the other ($g$) and then sum the multiplications (integrate them).
Any useful comment to complete the answer is appreciated.