Convolution intuition: clarifying Terence Tao's "blurring"/"fuzz" interpretation

On this math.MO post, "What is convolution intuitively?", Terence Tao's answer (in the case where one function is a bump function) involves "blurring" and "fuzz."

Could someone clarify his interpretation more explicitly? The intuition still escapes me. Thanks!

Solution 1:

Consider a function $f_1$: $$f_1(x)=\delta(x)$$ where $\delta(x)$ is Dirac delta, and a Gaussian function $g(x)$: $$g(x)=\frac1{\sigma\sqrt{2\pi}}\exp\left(-\frac{x^2}{2\sigma^2}\right).$$ One of main properties of Dirac delta is this: $$\int_{-\infty}^\infty q(x)\delta(x-x_0)dx=q(x_0).\tag1$$ Thus, convolution of $f_1$ and $g$ will equal $g$. Let's see how it looks ($f_1$ is red, $f_1*g$ is blue):

You can see the 'blurring' effect of gaussian convolved with a Dirac delta. Now consider more complex function: $$f_2(x)=\sum_{i=-N}^N a(x_i) f_1(x-x_i)\Delta x_i.\tag2$$ $\Delta x_i$ here is step in $x$, i.e. $\Delta x_i=x_i-x_{i-1}$. We'll need it later. We'll take $\Delta x_i=\Delta x_j\; \forall x,j\in\mathbb{Z}$, then it can be viewed as just a coefficient before sum.

Here's a plot of $f_2$ and $f_2*g$ ($f_2$ is red, $f_2*g$ is blue):

Here you can also see that the result of convolving a gaussian with a sum of Dirac deltas gives you blurred version of original function. It's easy to generalize $f_2$ to a sum of infinite number of deltas with $N\to\infty$: $$f_{2a}(x)=\sum_{i=-\infty}^\infty a(x_i)f_1(x-x_i)\Delta x_i.$$

$f_{2a}(x)$ resembles a Riemann sum. Let's use this and take the limit $\Delta x_i\to0$: $$f_3(x)=\lim_{\Delta x_i\to 0}f_{2a}(x)=\int_{-\infty}^\infty a(t)f_1(x-t)dt\equiv\int_{-\infty}^\infty a(t)\delta(x-t)dt.\tag3$$ From $(1)$, $$f_3(x)=a(x).$$ In $(3)$ after taking limit $x_i$ becomes $t$, and $\Delta x_i$ becomes $dt$. $\Delta x_i$ is needed to gradually attenuate Dirac delta amplitude as $\Delta x_i\to 0$, so that in the limit it becomes finite, resulting in a finite function.

Here's a plot of $f_3$ and $f_3*g$ ($f_3$ is red, $f_3*g$ is blue):

This is finally the convolution of a usual function with a gaussian, which still does resemble the blurring, but is somewhat less obvious.

Also note that since $p*q=q*p$, you could as well start from a Dirac delta, represent $g(x)$ as a sum of scaled&translated $\delta(x)$'s and get the same result by summing $a(x)$'s scaled by $g(x_i)$ instead of $g(x)$'s scaled by $a(x_i)$.

Solution 2:

I'll just answer this comment, since the rest seems to have been addressed satisfactorily.

First we need to understand the statement "functions are fuzzy versions of points". What this is saying is that the support of a function is not a single point, but many. Consider $f$ being a distribution of compact support, which we write $f\in \mathcal{E}'$ (if you are not familiar with distribution theory, just think locally integrable functions with compact support for now). Since the coordinate projections $(x_1,\ldots,x_n)\mapsto x_i$ are $C^\infty$ functions, we have that the pairing $$ \langle f,x_i\rangle \tag{CoM}$$ is well defined. (In terms of locally integrable functions, think of equation (CoM) as $\int_{\mathbb{R}^n} x_i f(x) \mathrm{d}x$. Since $f$ has compact support the integral converges.)

This allows as to define the center-of-mass for each $f\in \mathcal{E}'$. That is, let $\mu: \mathcal{E}' \to \mathbb{R}^n$ be the mapping $$ \mu(f) = \frac{1}{\langle f,1\rangle} \left( \langle f,x_1\rangle, \langle f,x_2\rangle,\ldots,\langle f,x_n\rangle\right)$$

Quite obviously $\mu$ is not injective: multiple functions/distributions can have the same center of mass. It is in this sense that functions are "fuzzy versions" of points: think of $f\in \mathcal{E}'$ with mass $\langle f,1\rangle = 1$ as something that is like the point $\mu(f)$, but spread out a bit in space.

Now let $f,g\in \mathcal{E}'$. Their convolution is well-defined. (See the Wikipedia link above.) We have the characterisation that for any smooth function $\phi$ the convolution satisfies $$ \langle f*g,\phi\rangle = \langle f,\psi\rangle $$ where $$ \psi(y) = \langle g,\tau_{-y} \phi\rangle $$ In terms of integrable functions this just says that $$ f*g(x) = \int f(x-y)g(y) \mathrm{d}y $$ the usual definition.

Now, a direct computation shows that $$ \mu(f*g) = \mu(f) + \mu(g) $$ This you can easily check using the integrable functions definition also: $$ \int x_i f*g(x) \mathrm{d}x = \iint x_i f(x-y) g(y) \mathrm{d}y\mathrm{d}x = \iint (z_i + y_i) f(z) g(y) \mathrm{d}z \mathrm{d}y = \int g(y) \mathrm{d}y \int f(z)z_i \mathrm{d}z + \int f(z) \mathrm{d}z \int y_i g(y) \mathrm{d}y $$

That is to say: when you convolve two functions/distributions, you add their center of mass. So if you treat $f\in \mathcal{E}'$ as fuzzy versions of points, the convolution is their natural law of "vector addition".

Lastly know that the Dirac delta function $\delta_x\in \mathcal{E}'$. They correspond to the actual sharp (non-fuzzy) points!